Private GPU pools and reserved GPU capacity, explained

Reserved GPU capacity is for teams that already know a workload can run and need predictable access to the GPUs powering it. The decisions are how much capacity to hold every day, which workloads can share it, and which traffic should stay on demand.

This guide explains when a private GPU pool makes sense, when on-demand GPUs are still the better choice, how bursting above a reservation works, and what data to bring before you commit. The core rule is simple: reserve the capacity you can use consistently. Let on-demand capacity handle the demand you can’t predict.

What is a private GPU pool?

A private GPU pool is reserved GPU capacity assigned to one customer. During the commitment term, that capacity is held for your organization instead of being returned to shared on-demand supply. Nobody else runs workloads on it.

Runpod can also reserve CPU compute in some cases, but this guide focuses on GPU capacity because that is the more common reservation request.

You may hear related terms:

• Reserved GPU capacity means the commercial commitment.
• GPU capacity reservation means the same thing in procurement or planning language.
• Dedicated GPU capacity describes the fact that the reserved compute is assigned to your organization.
• Private GPU pool describes the hardware pool your workloads can use.

On Runpod, reserved capacity can support the same products teams already use, including Pods, Serverless endpoints, and Clusters. That matters because a team can test a workload on self-serve infrastructure, prove its usage pattern, and then move predictable baseline usage into a commitment without rebuilding the workload around a different platform.

How reserved GPU capacity is billed

Reserved capacity bills for the committed compute capacity, not for each second of active work. If you reserve capacity, you pay for that commitment through the term whether it is fully used, partly used, or idle.

That is the tradeoff. On-demand GPU capacity gives you elasticity: you pay only while compute is active, but capacity may not be available exactly when you need it. Reserved capacity gives you predictable access and committed-use pricing, but idle time still costs money.

Current pooled usage is billed at the pool’s committed rate. If several workloads share one pool, Runpod may not distinguish workload-level rates inside that reserved capacity.

When a private GPU pool makes sense

A private GPU pool makes sense when at least one of these is true:

• You have a measurable baseline of GPU usage that runs most days.
• You know which GPU type, region, or machine class the workload needs.
• Running out of capacity would affect customers, revenue, deadlines, or launches.
• You can use the reserved capacity across more than one workload.
• Finance needs a more predictable monthly GPU commitment.
• Your team has enough usage history to size the commitment from real data.

The best candidates are not necessarily the teams with the highest peak traffic. They are the teams with the most predictable floor.

For example, a product with 4 active GPUs most of the day and occasional spikes to 20 GPUs may be a better reservation candidate than a batch pipeline that jumps to 40 GPUs once a week and sits idle the rest of the time.

When on-demand GPUs are still the better choice

Stay on demand when usage is hard to forecast or when the workload changes too often for a commitment to be useful.

Common on-demand cases include:

• early model experiments;
• short-lived development environments;
• demos with uncertain follow-up demand;
• one-time migrations or backfills;
• batch jobs that can wait for capacity;
• workloads that need different GPU types from week to week; and
• products before they have enough usage history to size a baseline.

If your team cannot explain what will run on the reserved capacity next month, do not buy the reservation yet. Run on demand until the floor becomes visible.

Sizing rule: start with the baseline, then plan for growth

Start with current usage, then layer in known growth. Historical data shows the capacity you already use consistently; forecasted demand helps decide whether to reserve extra room ahead of launches, customer commitments, or expected production growth.

If you have access to them, bring the clearest usage signals you can:

• recent GPU usage or spend;
• average daily usage;
• steady or recurring usage;
• peak concurrent GPUs by workload;
• common GPU types and regions; and
• upcoming launches, customer commitments, or growth assumptions.

Then compare that usage with the capacity you would reserve.

For an eight-GPU reserved pool, a rough utilization formula is:

90-day GPU hours / (90 days x 24 hours x 8 GPUs)

If the result is 75%, the reserved capacity would have been busy three-quarters of the time. If the result is 20%, the reservation would have been idle most of the time.

Use historical utilization as the baseline, then decide how much extra capacity is worth reserving for growth. A larger commitment can make sense when launches, customer demand, or supply risk justify the extra room; it should still be tied to a concrete workload plan.

Two reasons to reserve even when the discount is close

Price is not the only reason to reserve GPU capacity. A private GPU pool can still make sense when the pure rate comparison is close.

The first reason is supply certainty. If your workload depends on a scarce GPU type in a specific region, on-demand capacity can become the operational risk. A lower hourly price does not help if the GPU is unavailable during a launch, customer demo, training run, or production incident.

The second reason is workload flexibility. A shared reserved pool can support more than one kind of work. A team might run production inference during the day, scheduled evaluation jobs overnight, and fine-tuning jobs during lower-traffic windows. That does not make the reservation free, but it raises utilization and makes the commitment easier to defend.

Before reserving for either reason, write down the failure case. What happens if the capacity is not available? If the answer is “the job waits,” on demand may be fine. If the answer is “customer requests fail,” “a launch gets delayed,” or “an internal deadline slips,” reserved capacity may be worth more than the discount alone shows.

What a private GPU pool is not

A private GPU pool is not the same as bare metal access. The reserved capacity is dedicated to your organization, but workloads still run inside Runpod’s managed, containerized environment.

That distinction matters. Dedicated capacity gives you isolation and predictable access. The managed environment keeps deployment portable across reserved capacity and on-demand burst capacity, so the same container image can run in either place.

A private GPU pool also does not mean you should reserve every GPU your workload might ever need. The pool should cover the steady baseline. Capacity above that baseline can burst to on-demand supply when the workload and agreement allow it.

Minimums, terms, machine shapes, and available regions depend on the GPU type, product, and agreement. Treat the first sizing conversation as a planning exercise, not a checkout flow.

What happens when traffic exceeds the pool?

Runpod will try to place workloads on reserved capacity first. When the pool is full, additional demand may burst into on-demand capacity when bursting is enabled. Usage caps or hard-stop behavior should be confirmed during setup, because that control is still being finalized and may initially depend on API support.

Example: You reserve one eight-GPU machine for a Serverless endpoint. The first eight active workers will aim to be placed first on your reserved capacity. If traffic needs 12 workers and bursting is enabled, the next four workers will be placed outside of your pool on on-demand capacity and bill separately above the commitment.

Bursting is useful for customer-facing inference because it lets the product absorb temporary spikes without sizing the reservation for the worst hour of the month.

For cost-controlled batch jobs, internal tools, and experiments, ask whether the workload can queue, wait, or use an available cap instead of spilling into on-demand spend. Do not assume hard-stop behavior is available for every private pool configuration yet.

Decide this by workload:

Why stopped on-demand workloads behave differently

Stopping an on-demand Pod releases the GPU back to be made available to other customers. When you restart, the exact GPU capacity may no longer be available.

Runpod documents this as the reason a stopped Pod can restart with zero GPUs: the storage may remain on the physical machine, but the GPU slot can be rented by someone else after you stop the Pod.

Reserved capacity works differently. Terminating a workload or scaling an endpoint down releases the workload, not the reservation. The capacity stays assigned to your organization for the term.

That does not mean every user in the organization always sees idle capacity. If multiple teams or workloads share the same private pool, one workload can consume capacity that another workload expected to use.

That difference matters when the GPU type, region, or launch window is part of the requirement.

How this compares with hyperscaler GPU reservations

Most GPU capacity reservations follow the same basic logic: commit to baseline capacity, receive better pricing or supply assurance, and use on-demand capacity for overflow.

The practical question is how much engineering work the reservation creates.

In a general-purpose cloud, a GPU reservation can come with procurement lead time, region constraints, quota negotiations, service-specific architecture, and migration work. If the pilot ran on one system and the reservation requires a different deployment path, the capacity agreement is also a replatforming project.

Runpod’s model is meant to reduce that migration burden, not eliminate planning. Teams can often start self-serve, run real workloads, and work with Runpod on reserved capacity once usage becomes predictable. The reservation still needs to match GPU type, region, availability, and the way the customer’s workloads actually run.

How reserved capacity fits into Runpod

Runpod positions reserved capacity as part of its enterprise model: a reserved baseline at committed-use pricing, usage-based burst above the baseline, post-paid billing, contractual SLAs, and priority support.

For prospects, the important question is not “Should every workload move into a private pool?” The useful question is “Which part of our GPU usage is predictable enough to reserve, and which part should stay elastic?”

That split can look different by team:

• An inference team may reserve the baseline needed for normal production traffic and burst during launch spikes.
• A computer vision team may reserve the GPU type and region needed for daily processing, then run occasional backfills on demand.
• A research team may stay on demand until experiments settle into a recurring training or evaluation schedule.
• A platform team may reserve shared capacity across several internal teams once aggregate usage becomes predictable.

Customer examples can support this section, but they should not be treated as private-pool proof unless the case study says so. Gendo’s case study supports portability from existing containers and faster scaling. Segmind’s case study supports elastic scaling with near-zero idle compute cost. Coframe’s case study supports fast scale-up for launch traffic. Those examples show why elastic-to-committed infrastructure matters; they do not, by themselves, prove that a private pool is the right choice for every workload.

What to bring to a sizing conversation

Bring usage data, workload constraints, and the business reason for reserving. The first conversation should answer how much capacity to reserve, where it needs to run, and what should happen when demand exceeds the pool.

Start with whatever you can gather:

• recent GPU usage or spend;
• peak concurrent GPUs by workload;
• any visible baseline or recurring usage;
• GPU types and memory requirements;
• required regions or data-residency constraints;
• latency requirements for inference workloads;
• launch dates, deadlines, or seasonal demand patterns;
• whether each workload can queue, wait, fail, or burst;
• expected growth over the commitment term; and
• current on-demand spend for the same workloads.

Then ask these questions:

• What baseline can we use at least 60% to 70% of the time?
• Which workloads can share the same GPU type and region?
• What happens if the pool is full?
• Which workloads should burst, and which should use an available cap, queue, or wait?
• How will we track reserved utilization after the agreement starts?
• Who owns renewal, resizing, and expiration planning?

If little usage is predictable, keep most usage on demand and revisit the commitment later. If most usage is predictable and capacity risk is expensive, a private GPU pool is worth pricing.

Ready to explore reserved capacity? Talk to our sales team or visit runpod.io/enterprise to learn more.

Frequently asked questions

Is a private GPU pool the same as reserved GPU capacity?

They are closely related. Reserved GPU capacity is the agreement that commits you to a baseline amount of GPU infrastructure. A private GPU pool is the dedicated capacity your workloads use under that agreement.

What is the minimum size of a private GPU pool?

Minimums depend on the GPU type, product, region, and agreement. In practice, planning often starts at the machine level rather than an arbitrary fraction of a machine. Confirm the minimum pool size during the sizing conversation.

How long is a typical GPU capacity reservation?

Commitment length depends on the agreement and workload. Longer commitments can offer better pricing, but they require careful planning to make sure the reservation stays aligned with your usage. Size the first commitment from recent usage, then revisit it before renewal.

Does a private GPU pool give me bare metal access?

No. A private pool gives your organization dedicated capacity, but workloads still run in Runpod’s managed container environment.

Can one reservation cover Serverless, Pods, and Clusters?

Reserved capacity can support multiple Runpod products, including Pods, Serverless, and Clusters, depending on the agreement and workload requirements. Confirm the exact scope before signing so the reservation matches how your team actually uses GPUs.

What happens if my workload exceeds the reserved pool?

If bursting is enabled, excess demand can run on on-demand capacity and bill separately above the reserved commitment. If you need workloads to cap, queue, or fail instead of bursting, confirm the available controls during setup; hard-stop behavior is still being finalized and may initially depend on API support.

Should I size the pool for average demand or peak demand?

Neither by itself. Start with the steady baseline you can use consistently, then add growth capacity only when launches, customer commitments, or supply risk justify the extra room. Use on-demand capacity for peaks that are occasional or hard to predict.

How do I know whether reserved capacity will save money?

Compare the committed rate against your historical utilization, not against perfect utilization. A lower reserved rate can still cost more if the capacity sits idle. The reservation is strongest when utilization, capacity assurance, and operational predictability all matter.

What happens when the reservation expires?

The reserved capacity is no longer held for your organization unless the agreement is renewed or replaced. Review utilization and growth before the end of the term so you can renew, resize, or move workloads without a production surprise.