While there are many useful front ends for prompting Stable Diffusion, in some ways it can be easier to simply it directly within Jupyter Notebook, which comes pre-installed within many RunPod templates. Once you spin up a pod you get instant access to Jupyter as well, allowing you to directly communicate and create images without even having to install a front end at all. Let's go through some easy to use code examples that you can copy and paste straight into your notebook and modify to your needs.

Getting started

Go ahead and deploy a pod with the official RunPod Stable Diffusion template to get started. If you need a refresher on how to do that, feel free to refer to our docs here. Once you've got the pod created, then go to your Pods page and then connect to the pod on port 8888. You'll be able to copy and paste any of the below Python examples directly into your notebook.

Stable Diffusion 1.5

This will automatically download a model from Huggingface and allow you to send a prompt to the diffuser models, which will be placed in the same directory as your notebook. There are four major steps within this process:

• Downloads the model from Huggingface if necessary
• Passes parameters to the scheduler
• Pipes the prompt, scheduler type, and any additional variables to the model, such as CFG and number of denoising steps
• Saves the image

Stable Diffusion XL

Here's another example for Stable Diffusion XL that will do the same while utilizing the refiner model, which adds an extra step:

So why is this relevant? Isn't it just easier to use a UI?

Not always. User interfaces are great at simplifying the creation process and displaying all available options available, but where they're generally not so useful is something that requires an iterative process.

Let's say you have an assignment where you want to generate a version of the same image, but you want to use the same seed and you want to save a version at every fifth denoising step. You could still do this manually, but it would be a huge pain. You'd need to move the slider, click generate, move the slider, click generate, etc. And while this may be feasible at some low number of iterations, its feasibility depends on there just not being a high number of iterations, and that is not a great position to be in, because sooner or later it's going to become an issue again.

Let's rerun the SDXL with refiner code to accomplish this goal, with an assist from the Pillow library to stitch all the resultant images together into a single version for comparison.

More than half of this code block is just using Pillow to create the final demonstration image. To complete the actual task set out, it only took a few more lines. So you can see how quickly you can accomplish the actual task with just a few lines of code, rather than sitting there for minutes, waiting on a generation to finish so you can adjust a slider and then click the button again. All it takes is a pretty elementary loop to get the job done.

For reference, this is the final result, and you can see where an assignment like this has value. Do you want the more ethereal looking lower step image, or the more defined higher step image? Here, you can explicitly see them all together for a side by side comparison. How many options might you have missed by just playing with a slider, generating a few images, and calling it a day?

What about CivitAI?

You can absolutely download a model from CivitAI if you prefer. The only thing to note is that you need to set up a free API key to download models from there. Here is the code modified to download a model from there instead, just feed in the API key that you created on your Account page.

Start up a Stable Diffusion Pod

Conclusion

Although the code above looks intimidating, in truth it was generated in large part by coding assistants. Using a code copilot in conjunction with Jupyter while running your pod may actually save you time, and that's something we'll get into shortly in a new article. Now it's your turn - create something new and show us what you made on our Discord, where we'll also be happy to answer any questions you might have on the process!

‍