Skip to main contentBlog
Run MoonshotAI's Kimi-K2-Instruct on Runpod Clusters using H200 SXM GPUs and a 2TB shared network volume for seamless multi-node training. This guide.
1. Create Network Storage (2TB), Use the CA-MTL-4 region (recommended for now).
2. Spin Up a Pod using Runpod official Pytorch template and mount the network volume you just created, once the pod is running, connect to Jupyter Lab
3. Download the Model
4. Launch the Instant Cluster
1. installation on Node 0 with a shared volume
2. Node 1 Instructions
3. You should see the following:
4. Run on node with ip as host.
Known IssuesCurrently as of July 21st, vllm library is not up to date so need to build from nightly builds.
https://github.com/MoonshotAI/Kimi-K2/issues/19
uv environment on a Network Volume is slow to initialize ray, recommend any python environments be ran on the machine itself instead of on the Network Volume.
Author profile: Brendan McKeag
Blog Posts
.jpeg)
Whether you're already running production endpoints on Runpod or you're sizing us up for the first time, here's a plain-language tour of what Runpod Serverless does today, why it's faster and cheaper than it was six months ago, and how to deploy your first endpoint in minutes.
Shift from stateless inference to stateful architectures to resolve infrastructure bottlenecks like memory management, concurrency limits, and runaway jobs in production AI agents.
We raised a $100 million Series A. Here's what it means for you.
