News icon

Batch Inference is now generally available on Runpod Serverless

ReportFall 2026

State of AI Compute

Insights from the latest data around AI deployment, infrastructure demand, and model scaling trends.

Cover of the Fall 2026 State of AI Compute report

Get the full report

Download the PDF

PDF, 8.7 MB

What’s inside

VRAM tier by workload type

0%10%20%30%40%50%38.424.736.9FINE-TUNING37.133.829.1SERVING ONLY31.544.124.4INFERENCE35.440.324.3TRAINING49.834.515.7BATCH PROCESSING<24 GB25-48 GB80 GB+

Usage share among top audio models

0%20%40%60%80%NOVDECJANFEBMARAPRMAYJUNJULWHISPER62.7%OTHER TTS26.0%PYANNOTE10.3%QWEN AUDIO7.4%

Three fleets double by early December

1.0X1.5X2.0X2.5X3.0X3.5XB3003.4X TODAY’S FLEETH200 SXM2.5X TODAY’S FLEETB2002.1X TODAY’S FLEETH100 SXM1.5X TODAY’S FLEETRTX PRO 60001.3X TODAY’S FLEETGROWTH BY EARLY DECEMBER, AS A MULTIPLE OF TODAY’S FLEET
Fleet plans track the P50 demand forecast, 3.5-month horizon (Dec 7), August 2026. Multiples of currently verified supply.