Need Inspiration? Let Disco Diffusion Dream For You

BY ZHEN LU - JULY 1, 2022
Have you ever really wanted to generate some art using disco diffusion, but you aren't sure what to put in? I think we've all been there! Thankfully, there's a "prompt salad" feature in our version of disco diffusion that can help with your artists block!

UPDATE: We've switched to the discoart container and will aim to add prompt salad to it soon!

The easiest way to get started with using the "prompt_salad" feature is to use the RunPod Disco Diffusion GUI notebook. You can find a quick start guide here. When you start this notebook and run the first cell, you'll get a bunch of defaults that you can change to whatever you want. Go ahead and copy your settings if you want to save them for later before we start changing them. You can just copy/paste them into a new file for safekeeping.

What prompt salad allows you to do is to generate a template prompt that can randomly generate prompts for you based on randomized replacements. You can use the built-in default word replacement lists, or you can add your own! Here's an example config:

RN101: false
RN50: true
RN50x16: false
RN50x4: false
RN50x64: false
ViTB16: true
ViTB32: true
ViTL14: false
ViTL14_336: true
batch_name: prompt_salad_example
prompt_salad: true
prompt_salad_template: a {adjectives} {locations} spanning across a river of {of_something}, art by {progrock/artist}, trending on artstation
width_height:
- 2048
- 512
prompt_salad_amount: 100
n_batches: 1
steps: 250
set_seed: 8675309

This will generate prompts randomly that look like "a futuristic building spanning across a river of ice, art by Aldus Manutius, trending on artstation" . There are two settings that I should call out here:

prompt_salad_amount: 100
n_batches: 1

The prompt_salad_amount setting is the total amount of prompts you want it to generate before it stops. The n_batches setting is how many images it should generate for each prompt.

You can find all of the existing word lists that you can use for substitutionunder the /workspace/disco-diffusion-1/prompt_salad folder. If you don't like them, you can always edit them, or make your own! In order to make your own word list, you can create a new text file under prompt_salad/custom. For example, I could make a new word list file called lucky_charms.txt that looks like this:

Hearts
Stars
Horseshoes
Clovers
Blue Moons
Unicorns
Rainbows
Red Balloons

You can then use this like:

RN101: false
RN50: true
RN50x16: false
RN50x4: false
RN50x64: false
ViTB16: true
ViTB32: true
ViTL14: false
ViTL14_336: true
batch_name: prompt_salad_example
prompt_salad: true
prompt_salad_template: a {adjectives} {custom/lucky_charms} spanning across a river of {of_something}, art by {progrock/artist}, trending on artstation
width_height:
- 2048
- 512
prompt_salad_amount: 100
n_batches: 1
steps: 250
set_seed: 8675309

This is really awesome for just throwing a bunch of words at DD and seeing what luck can make for you. When you stumble on something great, then you can let your creative juices flow and iterate on it until you get that perfect result you've been looking for.

Good luck!

Thanks for /u/entmike for creating this awesome fork of DD

Zhen is one of the co-founders of RunPod. He holds a PhD in Computational Chemistry and has lots of experience teaching, coding, and talking to people.

Want to join our growing community?
Chat with us on our Discord Server.
Prefer email? Contact us at support@runpod.io