How to run Kimi K3 on Runpod's Public Endpoint

Kimi K3 is Moonshot Labs’ latest open-source model. It’s been making news lately for its massive 1-million-token context window and 2.8-trillion-parameter scale. Its uses include advanced coding, knowledge work, and reasoning. 

We now have Kimi K3 available on our public endpoints. This blog post walks you through what’s new and exciting about Kimi K3 from a technical perspective and how to get started using it on Runpod. 

What makes Kimi K3 notable

Kimi K3 breaks new ground on three axes at once. Architecturally, it's the most prominent demonstration that hybrid linear attention plus extreme MoE sparsity can hold the frontier. Strategically, it's the first time weights of this class have been open, extending Moonshot's streak of setting the upper bound of open-model sizes for nine of the past twelve months while collapsing the moat between closed API access and self-hosted deployment

Kimi K2 already proved Moonshot could train enormous sparse models. K3 goes further by rethinking the attention stack itself. The model is built on Kimi Delta Attention (KDA) combined with Attention Residuals (AttnRes): of its 93 layers, 69 use KDA and only 24 use gated Multi-head Latent Attention. In other words, roughly three-quarters of the network runs on a linear-attention-style mechanism, with a minority of full-attention layers preserving global recall. AttnRes, meanwhile, selectively retrieves representations across model depth rather than accumulating them uniformly; an architectural answer to information flow in very deep networks.

Native multimodality and a 1M-token window in one open model

K3 understands text, images, and video in a single model, using a 401M-parameter MoonViT-V2 vision encoder. Open models with strong vision exist, and open models with long context exist, but K3 combines frontier-level versions of both: it posts 91.1 on OmniDocBench for document understanding and 90.0 on Video-MME, while supporting a full 1,048,576-token context.

The case studies are the real headline

Benchmarks compress what K3 can do; Moonshot's demonstrations of long-horizon autonomy are more telling. In a single 48-hour autonomous run using open-source EDA tools, K3 designed, optimized, and verified a chip on the Nangate 45nm library, closing timing at 100 MHz and sustaining over 8,700 tokens/s decode throughput in simulation, purpose-built to serve a nano model based on K3's own architecture. A chip built by a model, for a model.

The sleeper feature: native MXFP4

The detail that should excite anyone who runs their own inference is quantization-aware training. From the SFT stage onward, K3 was trained natively in MXFP4 weights with MXFP8 activations. This isn't a post-hoc quantization that shaves accuracy off a BF16 checkpoint: 4-bit is the model's native format, chosen for broad hardware compatibility.

When you’re deciding what hardware to deploy this on, this matters. At MXFP4, 2.8T parameters occupy on the order of 1.5TB rather than the ~5.6TB a BF16 version would need. Combined with only 104B active parameters per token, per-token compute is closer to a mid-sized dense model than to anything resembling 2.8T. This is the difference between needing to requisition a whole cluster or managing to fit it into just a pod with enough B300’s which could drastically simplify your level of technical overhead, not to mention GPU spend.

How to get started with Kimi K3 on Runpod

To learn more about our support for Moonshot Kimi, including Kimi K3, and the available parameters, or to see an example using JavaScript, be sure to take a look at our documentation. 

One thing to note is that the moonshot-kimi is an endpoint ID that applies to all models in the kimi family, so you will need to include the model ID kimi-k3 to specify the right model.

You can also start experimenting and testing Kimi K3 in the Runpod Hub playground.

Prerequisites

• An API key for Runpod
• Your machine needs outbound HTTPS (port 443) to api.runpod.ai. On a corporate or restricted network, confirm a proxy or firewall isn't blocking it.
• curl installed. Most macOS, recent Windows, and all Linux systems ship with it. Verify with curl --version. 
• jq installed. This is needed to pretty-print or filter the JSON response. 
• Install via brew install jq for macOS or apt install jq (Debian/Ubuntu)
• Python 3.8 or higher to run the Python example. 

Curl request 

To get started making requests with this endpoint, the easiest way is to use curl. Be sure to replace YOUR_API_KEY with your own API Key.

curl -X POST https://api.runpod.ai/v2/moonshot-kimi/run  \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -d '{
    "input": {
      "messages": [
        { "role": "system", "content": "You are Kimi." },
        { "role": "user", "content": "What is Runpod?" }
      ],
      "sampling_params": {
        "max_tokens": 2048,
        "temperature": 1
      },
      "model": "kimi-k3"
    }
  }'

Python 

The Python version of that Curl command you just ran is as follows:

import requests

response = requests.post(
    "https://api.runpod.ai/v2/moonshot-kimi/runsync",
    headers={
        "Authorization": f"Bearer {YOUR_API_KEY}",
        "Content-Type": "application/json",
    },
    json={
        "input": {
            "messages": [
                {"role": "system", "content": "You are Kimi."},
                {"role": "user", "content": "What is Runpod?"},
            ],
            "sampling_params": {
                "max_tokens": 2048,
                "temperature": 1,
            },
            "model": "kimi-k3",
        }
    },
)

result = response.json()
print(result["output"])

Using Kimi K3 with OpenAI’s Python client

Because this model is compatible with OpenAI’s API format, you can use the OpenAI Python client to call it.

from openai import OpenAI

client = OpenAI(
    api_key=RUNPOD_API_KEY,
    base_url="https://api.runpod.ai/v2/moonshot-kimi/openai/v1",
)

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[
        {
            "role": "system",
            "content": "You are Kimi.",
        },
        {
            "role": "user",
            "content": "What is Runpod?",
        },
    ],
    max_tokens=2048,
    temperature=1,
    reasoning_effort="max", # set for adding reasoning
    top_p=0.9,
)

print(response.choices[0].message.content)

For streaming responses, add stream=True

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[
        {"role": "system", "content": "You are Kimi."}, 
        {"role": "user", "content": "What is Runpod?"}
    ],
    max_tokens=2048,
    stream=True,
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Integrating with Kimi Code 

Another way to get started using Kimi with Runpod is to set Runpod as the model provider for Kimi Code, the coding assistant from Kimi.

First, you need to download Kimi. In macOS and Linux type the following into your terminal:

curl -fsSL https://code.kimi.com/kimi-code/install.sh | bash

In Windows and PowerShell, use the following command:

irm https://code.kimi.com/kimi-code/install.ps1 | iex

In the editor of your choosing, you will want to edit ~/.kimi-code/config.toml to match the following (create it if it doesn't already exist):

default_model = "runpod/kimi-k3"

[providers.runpod]
type = "openai"
base_url = "https://api.runpod.ai/v2/moonshot-kimi/openai/v1"
api_key = "YOUR_API_KEY"

[models."runpod/kimi-k2.6"]
provider = "runpod"
model = "kimi-k2.6"
max_context_size = 262144
capabilities = [ "thinking", "tool_use", "image_in" ]
display_name = "Kimi K2.6 (Runpod)"

[models."runpod/kimi-k2.7-code"]
provider = "runpod"
model = "kimi-k2.7-code"
max_context_size = 262144
capabilities = [ "thinking", "tool_use" ]
display_name = "Kimi K2.7 Code (Runpod)"

[models."runpod/kimi-k3"]
provider = "runpod"
model = "kimi-k3"
max_context_size = 1048576
capabilities = [ "thinking", "always_thinking", "tool_use" ]
support_efforts = [ "low", "high", "max" ]
default_effort = "high"
display_name = "Kimi K3 (Runpod)"

[thinking]
enabled = true
effort = "high"
keep = "all"

This config sets up a coding tool to use Moonshot's Kimi models served through Runpod's API, defaulting to Kimi K3 with high-effort "thinking" enabled, and defines three selectable models (K2.6, K2.7 Code, and K3), each with its own context sizes, capabilities, and display names.

After saving your configuration file you can now type the following into your terminal:

kimi

Now, you should see Kimi Code up and running in your terminal. 

Conclusion

Kimi K3 is Moonshot Labs’ latest open-source model that can be used for including advanced coding, knowledge work, and reasoning. It is currently available as a public endpoint. Be sure to let us know on Discord, in the #built-on-runpod channel, if you build anything with it.