Letta (formerly MemGPT) gives agents persistent long-term memory that survives across sessions, stored in Postgres rather than reconstructed from scratch each conversation. Like most agent orchestration frameworks, Letta itself doesn't need a GPU: it manages memory, tool calls, and conversation state, and hands off the actual token generation to whatever model provider you configure, including a self-hosted one on Runpod.
1. Letta doesn't need a GPU. Your model does.
The Letta server is a Python application with a Postgres backend. Run it on a CPU pod. If you want Letta's agents talking to a model you host yourself instead of paying OpenAI or Anthropic per token, point it at a separate Runpod-hosted vLLM or TGI endpoint, the same split-deployment pattern used across every agent framework in this series: cheap, always-on orchestration on CPU, expensive GPU inference behind its own scalable endpoint.
2. Launching the Letta server on Runpod
From Pods > CPU, deploy with:
letta/letta:latest
Attach a Network Volume mounted at /var/lib/postgresql/data so agent memory, conversation history, and archival data survive pod restarts. Letta's Docker image bundles Postgres and runs its schema migrations automatically on first boot, which takes one to two minutes; don't restart the pod during that window.
Under Expose Ports, add port 8283. Runpod's proxy serves the REST API at https://{POD_ID}-8283.proxy.runpod.net/v1.
Set authentication before exposing the endpoint beyond local testing:
SECURE=true LETTA_SERVER_PASSWORD=a-strong-generated-password
Without SECURE=true, the server accepts unauthenticated requests from anyone who has the proxy URL.
3. Pointing Letta at a self-hosted model on Runpod
Letta connects to LLM backends via environment variables at server startup. For a vLLM or TGI endpoint you're running elsewhere on Runpod that exposes an OpenAI-compatible API, set:
OPENAI_API_BASE=https://{OTHER_POD_ID}-8000.proxy.runpod.net/v1
OPENAI_API_KEY=placeholderThis makes your self-hosted endpoint show up to Letta the same way a real OpenAI deployment would, since Letta's OpenAI provider path just expects an OpenAI-compatible API at the configured base URL. For providers with native SDKs (Anthropic, Ollama), Letta also accepts their standard environment variables directly, so you can mix a self-hosted model with cloud providers on the same server.
4. Embedding model requirement
Letta uses vector embeddings for archival memory search, so every agent needs an embedding model configured, not just a chat model. When running via Docker, you must specify one explicitly when creating an agent; there's no silent default. If you're keeping everything self-hosted, point this at an embedding model served from the same or a separate Runpod endpoint rather than defaulting to a cloud embedding API.
5. Connecting the Agent Development Environment
Letta's ADE is a web interface for creating, testing, and observing agents against a running server, and it works against a self-hosted server the same way it works against Letta Cloud. Point the ADE at your Runpod proxy URL, and if SECURE=true is set, provide the server password when prompted.
6. Testing the deployment
curl https://{POD_ID}-8283.proxy.runpod.net/v1/health \
-H "Authorization: Bearer your-server-password"Once the server responds healthy, create an agent via the REST API or the Python/TypeScript SDK, specifying both the chat model and embedding model your server has access to.
7. Backups
Because agent memory lives in Postgres on your Network Volume, back it up the same way you'd back up any production database, not just rely on the volume's persistence. A simple scheduled dump:
docker exec letta-container pg_dump -U letta -d letta | gzip > backup-$(date +%Y%m%d).sql.gz
8. Quick-start checklist
- Deploy a CPU Pod with
letta/letta:latest, a Network Volume at/var/lib/postgresql/data, and port 8283 exposed - Set
SECURE=trueandLETTA_SERVER_PASSWORDbefore exposing the endpoint - Point
OPENAI_API_BASEat a Runpod-hosted vLLM or TGI endpoint if you want a self-hosted model - Configure an embedding model; Letta has no silent default
- Set up scheduled Postgres backups since agent memory is stateful data worth protecting
