Meta
Llama 3.2 3B Instruct
Meta's 3B-parameter open language model, ready to serve on Runpod Serverless with vLLM or on a Pod.
Llama 3.2 3B Instruct
Specifications
Provider
Meta
Modality
Language
Task
Text generation
Parameters
3B
Active parameters
Context
Precision
BF16
Weights
6
License
llama3.2
Released
2024-09-17
Endpoint ID
Weights on Hugging Face
meta-llama/Llama-3.2-3B-Instruct
Frequently asked questions
How do I deploy Llama 3.2 3B Instruct on Runpod?
Deploy the vLLM worker from the Runpod Hub as a Serverless endpoint and set MODEL_NAME to meta-llama/Llama-3.2-3B-Instruct. The endpoint is OpenAI-compatible and scales to zero when idle. You can also run vLLM on a Pod.
What GPU do I need to run Llama 3.2 3B Instruct?
Start with RTX 4090 (24 GB VRAM). The weights take about 6 GB; longer contexts and more concurrent requests need extra memory for the KV cache.
How much does Llama 3.2 3B Instruct cost on Runpod?
Serverless endpoints and Pods bill per second for the GPU you choose. A Serverless endpoint costs nothing while scaled to zero.
meta-llama-llama-3-2-3b-instruct
meta-llama/Llama-3.2-3B-Instruct
Language
2024-09-17 17:00
RTX 4090