Qwen moves fast enough that "how to run Qwen 3" is really two questions: which Qwen can you actually self-host, and what does it need to run? Alibaba has shipped several generations since the original Qwen3 release, and not all of them come with downloadable weights.
This guide sorts out what is open, what is API-only, and how to deploy the open ones on Runpod. If you want the short answer: the Qwen3.6 family is the current open-weights line, and the 27B dense model runs on a single 48 GB card.
Which Qwen models have open weights?
This is the part most guides get wrong, because the answer changed several times in 2026.
- Qwen3 (original family): open weights under Apache 2.0, spanning 0.6B dense up to 235B-A22B MoE. Still perfectly usable and widely deployed.
- Qwen3.5 (February 2026): open weights, Apache 2.0.
- Qwen3.6-27B (April 2026): a dense open-weights model that outperforms considerably larger MoE models on agentic coding benchmarks. Apache 2.0.
- Qwen3.6-35B-A3B: a sparse MoE variant with roughly 3B active parameters. Apache 2.0.
- Qwen3.7 (May 2026): API-only. No open weights.
- Qwen3.8-Max (previewed July 2026): a 2.4-trillion-parameter multimodal model. Preview access only, weights promised but not released at the time of writing.
So if the goal is running Qwen on your own infrastructure, the practical shortlist is Qwen3, Qwen3.5, and Qwen3.6. The Qwen3.6-27B dense model is the sweet spot for most teams: strong on coding and agentic tasks, small enough to serve on one GPU, and permissively licensed.
Why run Qwen on Runpod?
The size range is enormous. Qwen spans 0.6B to 235B-A22B. Picking the right one means trying several, and buying hardware for the largest before you know which you need is backwards.
Apache 2.0 means you can actually ship it. Unlike some open-weights licenses, Apache 2.0 puts no restrictions on commercial deployment, which makes self-hosting a real option rather than an evaluation exercise.
Serverless suits variable traffic. If Qwen is powering a feature rather than a batch job, endpoints that scale from zero cost nothing between requests.
GPU requirements by model size
Rough guidance for bf16 inference. Quantization brings each tier down substantially.
- 0.6B to 4B: RTX A5000 (24 GB) from $0.16/hr, or an RTX 4090 from $0.34/hr
- 7B to 14B: RTX 4090 (24 GB) from $0.34/hr, or RTX 5090 (32 GB) from $0.69/hr
- 27B dense (Qwen3.6-27B): L40S or RTX 6000 Ada (48 GB) from $0.74/hr
- 32B to 35B-A3B: L40S (48 GB) from $0.79/hr, comfortable on an H100 (80 GB) from $1.99/hr
- 235B-A22B: multi-GPU. H200 (141 GB) from $3.59/hr per card, or B200 from $5.98/hr
The MoE variants deserve a note: Qwen3.6-35B-A3B holds 35B parameters but activates only about 3B per token. You still need memory for the full parameter set, but generation speed behaves like a much smaller model.
Step 1: Create a Runpod account
Go to runpod.io and sign up with email, GitHub, or Google. Add credits or a payment method so you can deploy a GPU.
Step 2: Deploy with the vLLM worker
The fastest route to a working endpoint is the vLLM worker in the Runpod Hub, which supports the Qwen family directly.
- Open the vLLM worker and click Deploy.
- In the Model field, enter the Hugging Face path for your chosen Qwen model.
- Expand Advanced and set Max Model Length to the context you need. 8192 is a reasonable starting point.
- Select a GPU tier matching the table above, then click Create Endpoint.
For interactive work or fine-tuning instead, deploy a pod from the Pods section with a PyTorch template.
Step 3: Send a request
curl -X POST "https://api.runpod.ai/v2/YOUR_ENDPOINT_ID/runsync" -H "Authorization: Bearer YOUR_API_KEY" -H "Content-Type: application/json" -d '{"input": {"prompt": "Refactor this function to use async iteration and explain the tradeoffs."}}'Pro tips
Start smaller than you think you need. Qwen's smaller models are unusually strong for their size. Test a 7B before assuming you need a 32B, because the cost difference over a month of serving is substantial.
Check the license on the specific model. Most Qwen open-weights releases are Apache 2.0, but verify for the exact checkpoint you deploy rather than assuming it inherits from the family.
Use a network volume across experiments. If you are comparing three Qwen sizes, keeping all three on a volume avoids re-downloading each time you switch.
Stop your pod when you are done. Runpod bills by the second while a pod runs.
Wrapping up
Qwen gives you the broadest size range of any open-weights family, from models that run on a 24 GB card to multi-GPU MoE deployments, and Apache 2.0 licensing on most of it. The main thing to get right is picking the version that still ships weights, because the newest Qwen is not always the one you can download.
Ready to try it? Deploy the vLLM worker with a Qwen3.6 model, or launch a pod for fine-tuning.
FAQ
Is Qwen 3 open source?
The Qwen3, Qwen3.5, and Qwen3.6 families ship open weights, most under Apache 2.0, which permits commercial use. Qwen3.7 is API-only, and Qwen3.8-Max was in preview with weights promised but not released at the time of writing. Check the license on the specific checkpoint you plan to deploy.
Which Qwen model should I run?
For most teams, Qwen3.6-27B. It is dense rather than MoE, so it is straightforward to serve, it performs strongly on agentic coding, and it fits on a single 48 GB GPU from $0.74/hr. Step down to a 7B if latency and cost matter more than capability.
What GPU do I need for Qwen 3?
It depends entirely on size. Models under 4B run on a 24 GB card from $0.16/hr. The 27B dense model wants 48 GB. The 235B-A22B MoE needs multiple high-VRAM GPUs.
What does 35B-A3B mean?
Thirty-five billion total parameters with roughly three billion active per token. You need memory for all 35B, but generation speed resembles a 3B model, which is the point of the architecture.
Can I fine-tune Qwen on Runpod?
Yes. LoRA and QLoRA fine-tuning of the smaller Qwen models runs comfortably on a single 24 to 48 GB GPU. Deploy a pod with a PyTorch template and attach a network volume for your datasets and checkpoints.
