Skip to main contentBlog
Our team’s insights on building better and scaling smarter.
This guide walks through setting up SillyTavern—a powerful, customizable roleplay frontend—on a Runpod instance. It covers port exposure, GitHub installation, whitelist config, and connecting to models like Oobabooga or KoboldAI.
Want to upgrade from basic chat UIs? SillyTavern offers a more interactive interface for AI conversations. Here’s how to install it on your own Runpod instance.
Runpod now supports Panchovix’s 16k-token context models, allowing for much deeper context retention in long-form generation. These models require higher VRAM and may trade off some performance, but are ideal for extended sessions like roleplay or complex Q&A.
Runpod announces a partnership with Defined.ai to offer ethically sourced speech and text datasets to AI developers, starting with a pilot program to fine-tune LLMs and accelerate NLP research.
Large language models like Guanaco 65B can run on Runpod with the right optimizations. Learn how to handle quantization, memory, and GPU sizing.
New 8k context models from TheBloke—like WizardLM, Vicuna, and Manticore—allow longer, more immersive text generation in Oobabooga. With more room for character memory and story progression, these models enhance AI storytelling.
Starting with runpod-python 0.10.0, you can launch a local API server for testing your worker handler using --rp_serve_api. This feature improves the development workflow by letting you simulate interactive API requests before deploying to serverless.
