Global Volumes are now in beta for Serverless endpoints

What happened

Runpod has introduced Global Volumes in beta for its Serverless endpoints, enabling users to store model weights once and access them from any data center with compatible GPU types. This eliminates the need to pin endpoints to a single region, allowing workers to start in any available data center. The feature is designed for read-heavy workloads and offers usage-based billing.

Why it matters

Global Volumes provide significant flexibility for AI developers and machine learning engineers by allowing model weights to be accessed from multiple data centers without duplication. This reduces the need for region-specific storage and simplifies model deployment and updates, making it easier to scale AI workloads across different regions.

• Global Volumes allow model weights to be stored once and accessed from any data center with compatible GPUs.
• They are ideal for read-heavy workloads like model weights, LoRAs, and tokenizers.
• Usage-based billing and no upfront sizing make them cost-effective for scalable AI deployments.
• Global Volumes are not a replacement for high-performance regional storage like network volumes.
• Switching between Global Volumes and network volumes does not require changes to handler code.

The bigger picture

Global Volumes reduce the complexity and cost of managing model storage across multiple regions, enabling more efficient deployment and scaling of AI workloads. They are particularly beneficial for applications that require frequent model updates or serve users in multiple geographic locations.

Runpod's Global Volumes in beta offer a flexible and scalable solution for AI developers to store and access model weights across multiple data centers, reducing the need for region-specific storage and simplifying deployment processes.

Technical details

• Global Volumes are region-independent and elastic, with no fixed size requirements.
• They are mounted at /runpod-volume on Serverless endpoints, the same path as network volumes.
• Model weights and other artifacts are stored once and accessed across multiple data centers.
• Usage-based billing applies, with different rates for reads and writes.
• Global Volumes are backed by object storage and lack full file system semantics like file locking.
• They are not suitable for write-heavy workloads or applications requiring high write throughput.

FAQ