How to Build and Deploy a GPU-Powered MCP Server on Runpod

Model Context Protocol has gone from an Anthropic spec to the thing every AI tooling conversation eventually turns into. Ship an MCP server, and the next question is always the same: where does it actually run?

We've covered MCP here before. Runpod Just Got Native in Your AI IDE walks through Runpod's own MCP server, which lets Claude Code, Cursor, or any MCP client manage your Pods and endpoints straight from your editor. That post is about using MCP to control Runpod.

This one's the other direction: using Runpod to run an MCP server. Specifically, one whose tools need a GPU to do their job, instead of just calling someone else's REST API.

What you'll build:

• A minimal MCP server (Python) with one real tool: GPU-accelerated text-to-image generation
• A Docker image that runs it
• A Runpod Serverless endpoint that scales to zero when nobody's using it
• A live connection from Claude to that endpoint

Why your MCP server might need a GPU behind it

Most MCP servers out in the wild right now are thin. They turn a tool call like listing GitHub issues or posting to Slack channels into a REST request. That's a fine workload for a stdio process on your laptop or a lightweight container.

A different category of tool does the work itself instead of calling out for it: generate an image from a prompt, transcribe an audio file, embed a batch of documents, run inference against an open-weight model. Those tools need a GPU attached to the process handling the request, and they need to be reachable over the internet if you want anything other than your own machine to use them.

That's a hosting problem, not a protocol problem. MCP's Streamable HTTP transport handles "reachable over the internet" just fine. It's a regular HTTPS endpoint. What it doesn't give you is a GPU, autoscaling, or a way to avoid paying for that GPU between calls. That's where Runpod Serverless comes in: specifically, Load Balancing endpoints, which route HTTP requests directly to a pool of GPU workers instead of putting them behind a job queue.

What you'll need

• A Runpod account with billing or credits set up
• Docker, and somewhere to push an image (Docker Hub, GHCR, etc.)
• Python 3.10+
• Node.js (only needed later, to test the connection from Claude Desktop)

Step 1: Launch a Pod

In the Runpod console, go to Pods, then Deploy.

• Template: search for and select the official Runpod Pytorch template.
• GPU: SDXL-Turbo is light by diffusion-model standards. A 16 GB card (RTX 4000 Ada, A4000, L4, or similar) has plenty of headroom. Check current availability and pricing for your region.
• Container/Volume Disk: bump this up to at least 20 GB so there's room for the model weights.
• Exposed HTTP Ports: the template exposes 8888 (Jupyter) by default. Add 8000 to that list as well.
• Deploy On-Demand.

Step 2: Write and run the server

Click Connect on your Pod, then open up a terminal. The Jupyter Notebook terminal is the most easily accessible option that’s robust against interruption.

Install the few packages the template doesn't already have (PyTorch and CUDA are already there, so this is quick):

pip install mcp fastapi "uvicorn[standard]" diffusers transformers accelerate pillow

Then create the server file:

cat > server.py << 'PYEOF'
import contextlib
from io import BytesIO

import torch
from diffusers import AutoPipelineForText2Image
from fastapi import FastAPI
from mcp.server.fastmcp import FastMCP, Image
from mcp.server.transport_security import TransportSecuritySettings

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/sdxl-turbo",
    torch_dtype=torch.float16,
    variant="fp16",
).to("cuda")

mcp = FastMCP(
    name="gpu-image-gen",
    stateless_http=True,
    transport_security=TransportSecuritySettings(
        allowed_hosts=[
            "ro0aoohi5fvfx4-8000.proxy.runpod.net",
            "localhost:8000",
            "127.0.0.1:8000",
        ],
    ),
)

@mcp.tool()
def generate_image(prompt: str) -> Image:
    """Generate an image from a text prompt using a GPU-accelerated diffusion model."""
    result = pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0.0).images[0]
    buffer = BytesIO()
    result.save(buffer, format="PNG")
    return Image(data=buffer.getvalue(), format="png")

@contextlib.asynccontextmanager
async def lifespan(app: FastAPI):
    async with mcp.session_manager.run():
        yield

app = FastAPI(lifespan=lifespan)

@app.get("/ping")
async def health_check():
    return {"status": "healthy"}

app.mount("/", mcp.streamable_http_app())
PYEOF

A few things worth calling out:

• stateless_http=True tells FastMCP not to pin session state to a single request cycle, which is good practice for any HTTP-transport MCP server, Pod or not.
• generate_image loads stabilityai/sdxl-turbo once, at import time, so the model stays resident on the GPU across requests instead of reloading on every call.
• /ping isn't part of MCP. Iit's a plain health-check route, useful for your own sanity checks (and required if you move this to a Runpod Serverless Load Balancing endpoint later).
• mcp.streamable_http_app() mounts the actual MCP endpoint at /mcp by default, sitting right next to /ping in the same app, on the same port.

Run it in the background so it keeps going after you close the terminal tab.

You should see something like [1] 1605 in the output.

Then, run:

tail -f server.log

You should see the server running:

Loading weights: 100%|██████████| 517/517 [00:00<00:00, 6554.55it/s]5.59it/s]
Loading weights: 100%|██████████| 196/196 [00:00<00:00, 10745.35it/s].44it/s]
Loading pipeline components...: 100%|██████████| 7/7 [00:01<00:00,  6.25it/s]
INFO:     Started server process [1605]
INFO:     Waiting for application startup.
[07/01/26 08:19:58] INFO     StreamableHTTP session manager started                                                        streamable_http_manager.py:131
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to q

Give it a minute before you see anything as that first run has to actually download SDXL-Turbo's weights and load them onto the GPU, so there'll be a pause before the log fills in. You're waiting for Application startup complete on its own line. Once you see it, Ctrl+C just exits the tail, it won't touch the server (it's still running in the background as PID 1605).

If you see a traceback instead:

• ModuleNotFoundError → the pip install line didn't fully complete, worth re-running it
• Anything mentioning CUDA/out of memory → the GPU on this Pod is too small, or something else is already using it
• Nothing at all after a couple minutes → check curl http://localhost:8000/ping in a second terminal tab; if that hangs too, the process likely died — run jobs to see if [1] is still listed

Step 3: Test it

From inside the Web Terminal, a quick local check:

curl http://localhost:8000/ping

Then from outside, using Runpod's proxy, find your Pod ID in the console and swap it in:

curl https://POD_ID-8000.proxy.runpod.net/ping

Both should return {"status": "healthy"}. Your MCP endpoint itself lives at:

https://POD_ID-8000.proxy.runpod.net/mcp

(One thing worth knowing: Runpod's proxy cuts off requests that run past 100 seconds. SDXL-Turbo's single-step generation finishes in a couple of seconds, so this specific tool is well clear of that. But that is worth remembering if you swap in something heavier later.)

Step 4: Connect it to Claude

Unlike a Runpod Serverless endpoint, a Pod's proxy URL isn't gated by your Runpod API key — so this connects the normal way, no extra auth wiring needed.

In Claude (web, Desktop, or Cowork), go to Settings → Connectors, click +, then Add custom connector. Paste in:

https://POD_ID-8000.proxy.runpod.net/mcp

Click Add, then enable the connector for your conversation and try:

"Generate an image of a lighthouse at sunset, watercolor style."

Before you leave this running

A couple of things worth knowing:

• Pods bill by the hour while they're on. There's no scale-to-zero here the way there is with Serverless. Stop or terminate the Pod when you're done testing.
• The proxy URL has no auth of its own beyond being hard to guess. Fine for a quick personal test; add your own auth check in the app, or move to an OAuth-protected Serverless deployment, before sharing the URL or leaving it up for long.
• We turned off the MCP SDK's Host-header check to get around Runpod's proxy rewriting it. That check guards against a specific attack (a malicious webpage using DNS rebinding to reach a service that's meant to be localhost-only) — since this server is intentionally public, not localhost-only, turning it off doesn't hand anything real away here. It would matter if you later ran the same code somewhere that doesn't rewrite the Host header on the way in; worth revisiting if you move this off Runpod's proxy.

Where to go from here

This gets you a working MCP server fast, but it's a dev setup, not a production one. The Pod runs whether or not anyone's calling your tool, and there's no autoscaling. Once the tool does what you want, the natural next step is to wrap the same server.py in a Dockerfile and deploy it to a Runpod Serverless Load Balancing endpoint instead. It’s the same /ping and /mcp routes, but with scale-to-zero billing and autoscaling across GPU workers instead of one Pod running around the clock. Happy to write that up as a follow-up if it's useful.

And if you haven't already set up Runpod's own MCP server, that's the natural next read. Same protocol, opposite direction.

‍

‍