Use Mutations to Automate Your Disco Diffusion Studies

BY ZHEN LU - JULY 1, 2022
Have you seen those cool prompt/settings studies that people have done? Maybe you wanted to see what the different effects of cut_ic_power are on your final product, or play around with clip_guidance_scale, or to simply try a bunch of text prompt variations. You can automate generation of these exploratory studies using config files. Read on to find out how!

UPDATE: We've switched to the discoart container and will be publishing how to accomplish this with their code soon!

The easiest way to get started with using the "multipliers" feature is to use the RunPod Disco Diffusion GUI notebook. You can find a quick start guide here. When you start this notebook and run the first cell, you'll get a bunch of defaults that you can change to whatever you want. Go ahead and copy your settings if you want to save them for later before we start changing them. You can just copy/paste them into a new file for safekeeping.

Let's get started. There's a feature in this version of DD that is called "multipliers". What it does is generate combinations of DD render jobs based on the settings you choose. Let's look at a simple example:

# These settings are base settings for this job set.  All other settings are the
# "Notebook defaults" that a typically set.
batch_name: explore
set_seed: 54322
n_batches: 1
steps: 150
# Multipliers create a Cartesian Product of all combinations to create a set of
# jobs
multipliers:
  text_prompts:
  - 0:
        - cybernetic organism, artstation, Art by Beksinski, unreal engine
  clip_guidance_scale:
  - 500
  - 1000
  - 1500
  - 2000

This input will take the text_prompt "cybernetic organism, artstation, Art by Beksinski, unreal engine" and generate 4 jobs with 1 image in each job batch using the clip_guidance_scale values of 500, 1000, 1500, and 2000. Any settings that you put before the "Multipliers" section will be standard for all jobs. Anything you put inside the multipliers section will generate combinations of jobs. This particular config would be great for seeing what the effect of different cgs values would be on your final output.

Let's mix things up a bit. The following config is a little more complicated:

# These settings are base settings for this job set.  All other settings are the
# "Notebook defaults" that a typically set.
batch_name: explore
set_seed: 54322
n_batches: 1
steps: 150
# Multipliers create a Cartesian Product of all combinations to create a set of
# jobs
multipliers:
  text_prompts:
  - 0:
        - cybernetic organism, artstation, Art by Beksinski, unreal engine
  - 0:
        - cybernetic organism, artstation, Art by Beksinski, blender
  clip_guidance_scale:
  - 500
  - 1000
  - 1500
  - 2000
  cut_ic_pow:
  - 1
  - 5
  - 10

This config file will generate 2(prompts) x 4(clip_guidance_scale) x 3(cut_ic_pow) = 24 jobs, 12 for each prompt. This will give you all possible combinations of the settings under multipliers. This is great if you just want to mix and match everything to see what all the effects will be. Be careful though, this could generate a LOT of jobs!

After you're satisfied with your configuration, just run the next cell by selecting it and pressing enter while holding down shift (shift+enter). This will start your run. Your pictures will start generating in the images_out folder along with the specific settings that were used to generate them, so you can tell what pictures went with what settings. You can also copy the settings if you have a good output and you want to run more batches with that config. If you want to edit your settings, you can just hit the stop button to stop your running job, and go back to editing the config cell. Repeat until you've got awesome art to show everyone!

Thanks for /u/entmike for creating this awesome fork of DD

Zhen is one of the co-founders of RunPod. He holds a PhD in Computational Chemistry and has lots of experience teaching, coding, and talking to people.

Want to join our growing community?
Chat with us on our Discord Server.
Prefer email? Contact us at support@runpod.io