Stable Diffusion runtime error - how to fix CUDA out of memory error

If the Stable Diffusion runtime error is preventing you from making art, here is what you need to do.

Stable Diffusion is one of the best AI image generators out there. Unlike DALL-E and MidJourney AI, Stable Diffusion is available to the public and anyone with a powerful machine can generate images from texts.

However, Stable Diffusion might sometimes run into memory issues and stop working. If you are experiencing the Stable Diffusion runtime error, try the following tips.

How To Fix Runtime Error: CUDA Out Of Memory In Stable Diffusion

So you are running Stable Diffusion locally on your PC, maybe trying to make some NSFW images and bam! You are hit by the infamous RuntimeError: CUDA out of memory.

The error is accompanied by a long message that basically looks like this. The amount of memory may change but the content is the same.

RuntimeError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 6.00 GiB total capacity; 5.16 GiB already allocated; 0 bytes free; 5.30 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

It appears you have run out of GPU memory. It is worth mentioning that you need at least 4 GB VRAM in order to run Stable Diffusion. If you have 4 GB or more of VRAM, below are some fixes that you can try.

• Restarting the PC worked for some people.
• Reduce the resolution. Start with 256 x 256 resolution. Just change the -W 256 -H 256 part in the command.
• Try this fork as it requires a lot less VRAM according to many Reddit users.

If the issue persists, don't worry. We have some additional troubleshooting tips for you to try. Keep reading!

Other Troubleshooting Tips

So you have tried all the simple and quick fixes but the runtime error seems to have no intention to leave you, huh? No worries! Let's dive into relatively more complex steps. Here you go.

• As mentioned in the error message, run the following command first: PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6, max_split_size_mb:128. Then run the image generation command with: --n_samples 1.
• Call the optimized python script. Use the following command: python optimizedSD/optimized_txt2img.py --prompt "a drawing of a cat on a log" --n_iter 5 --n_samples 1 --H 512 --W 512 --precision full
• You can also try removing the safety checks aka NSFW filters, which take up 2GB of VRAM. Just replace scripts/txt2img.py with this:
  https://github.com/JustinGuese/stable-diffusor-docker-text2image/blob/master/txt2img.py

Hopefully, one of the suggestions will work for you and you will be able to generate images again. Now that the Stable Diffusion runtime error is fixed, have a look at how to access Stable Diffusion using Google Colab.