News icon

Batch Inference is now generally available on Runpod Serverless

NVIDIA RTX 3090 for AI: 24GB specs and what it still runs

The GeForce RTX 3090 launched in September 2020, and it is still in the Runpod lineup six years later. That is unusual for a consumer card, and the reason is simple: 24 GB of VRAM at one of the lower hourly rates in the lineup.

This guide covers what the card actually is, what fits in 24 GB, whether the 48 GB version people keep searching for exists, and where a 2020 GPU still earns its place against newer silicon.

RTX 3090 specs

The RTX 3090 is built on NVIDIA's Ampere architecture, using the GA102 die, with 24 GB of GDDR6X on a 384-bit bus.

SpecificationGeForce RTX 3090
ArchitectureAmpere (GA102)
GPU memory24 GB GDDR6X
Memory interface384-bit
Memory bandwidth936 GB/s
CUDA cores10,496
Tensor coresThird generation
RT coresSecond generation
Total board power350 W
LaunchedSeptember 2020

The number that matters for AI work is the 936 GB/s of memory bandwidth, not the CUDA core count. Token generation during LLM inference is largely memory-bandwidth-bound, because the GPU spends most of its time moving weights rather than doing arithmetic. On that measure a 2020 card holds up better than its age suggests.

What fits in 24 GB

24 GB is the dividing line that decides which models run on one card. The rough arithmetic is two bytes per parameter at 16-bit, one at 8-bit and half at 4-bit, before you add the key-value cache.

  • 7B and 8B models at 16-bit fit with room for a working context. This is the comfortable case.
  • 13B models at 8-bit fit. At 16-bit they do not: the weights alone come to roughly 26 GB, which is past the card before you account for any context at all.
  • 30B-class models at 4-bit fit, with the quality tradeoff that quantization implies.
  • 70B models do not fit on a single 24 GB card at any precision you would want to serve. That is the ceiling, and it is the reason to look at a 48 GB or 96 GB card.
  • Image generation is comfortable. SDXL and most ComfyUI workflows run inside 24 GB without offloading.

Treat these as capacity guidance rather than benchmarks. We have not published measured throughput for this card, and we are not going to imply numbers we have not run.

Is there a 48 GB RTX 3090?

No. NVIDIA never produced one, and 24 GB is the only memory configuration the RTX 3090 shipped with. This comes up often enough to be worth stating plainly, because modified cards with doubled memory circulate on secondary markets and are not an NVIDIA product.

If you need 48 GB on a single GPU, the cards that actually provide it are the RTX A6000 at $0.69/hr, the L40S at $1.09/hr and the RTX 6000 Ada at $1.09/hr. Above those, the RTX PRO 6000 Blackwell carries 96 GB.

Is the RTX 3090 still worth using for AI?

For a card released in 2020, more often than you would expect. Three situations where it is still the sensible pick:

  • Memory-bound inference on models that fit. If your model lives comfortably in 24 GB, what you are paying for is bandwidth and capacity. The 3090 is not the cheapest 24 GB card on Runpod, the RTX A5000 is, but it carries more memory bandwidth than the Ampere workstation cards, which is the specification that governs token generation.
  • Development and experimentation. Prototyping, fine-tuning small models and testing pipelines rarely need current-generation silicon.
  • Image and video generation workflows that fit in 24 GB and are not latency-critical.

Where it stops making sense is equally clear. It has no FP8 support, so the quantization paths that newer architectures use for throughput are not available. Its 350 W draw is high for the work it does by current standards. And anything that needs more than 24 GB needs a different card, not a second 3090, unless you are prepared to handle the multi-GPU overhead.

Where the RTX 3090 sits against other 24 GB cards

The 3090 is one of several 24 GB options, and which one wins depends on the workload rather than on raw specs. Rather than restate the numbers here, we maintain side-by-side comparisons that stay current:

The one worth reading first is the 5090 comparison, because it is the same consumer line two generations on, and the RTX 5090 guide covers what Blackwell adds.

Running the RTX 3090 on Runpod

Deploy the RTX 3090 from the GPU models page at $0.50/hr, billed per second with no commitment. Current rates for every card are on the pricing page.

RTX 3090 FAQs

How much VRAM does the RTX 3090 have?

24 GB of GDDR6X on a 384-bit interface, with 936 GB/s of memory bandwidth.

Is there a 48 GB version of the RTX 3090?

No. NVIDIA only ever shipped the RTX 3090 with 24 GB. Modified cards with doubled memory exist on secondary markets but are not an NVIDIA product. For 48 GB on one GPU, the RTX A6000, L40S and RTX 6000 Ada are the cards that provide it.

How many CUDA cores does the RTX 3090 have?

10,496, alongside third-generation Tensor cores and second-generation RT cores.

Can I run a 70B model on an RTX 3090?

Not on a single card. 70B models do not fit in 24 GB at a precision worth serving. A 48 GB or 96 GB card is the single-GPU answer.

Can I run a 13B model on an RTX 3090?

At 8-bit, yes. At 16-bit, no: the weights alone are roughly 26 GB, which exceeds the card before any context is allocated.

Is the RTX 3090 good for Stable Diffusion?

Yes. SDXL and most ComfyUI workflows fit inside 24 GB without offloading, which is the main requirement for image generation work.

What is the power draw of the RTX 3090?

350 W total board power. That is high relative to the work it does by current standards, though on rented infrastructure it affects the hourly rate rather than your electricity bill.

Does the RTX 3090 support FP8?

No. FP8 arrived with later architectures, so quantization paths that depend on it are not available on Ampere consumer cards.

Related guides

Author profile: The Runpod Team

Purple glow background

Related articles

View All
No items found.

Build what’s next.

Build, train, and scale AI workloads on Runpod with cloud GPUs, Serverless, and Clusters.

Star field background