Stable Diffusion 2.1 is here!

[mykeehu]

It was announced on the Stable Diffusion Discord channel that an improved version of 2.0 is expected this week.
I'm curious to see what they improve on it...

[mykeehu]

Version 2.1 is out!

Here's the announcement and here's where you can download the 768 model and here is 512 model

"New stable diffusion model (Stable Diffusion 2.1-v, HuggingFace) at 768x768 resolution and (Stable Diffusion 2.1-base, HuggingFace) at 512x512 resolution, both based on the same number of parameters and architecture as 2.0 and fine-tuned on 2.0, on a less restrictive NSFW filtering of the LAION-5B dataset. Per default, the attention operation of the model is evaluated at full precision when xformers is not installed. To enable fp16 (which can cause numerical instabilities with the vanilla attention module on the v2.1 model) , run your script with ATTN_PRECISION=fp16 python <thescript.py>"
https://github.com/Stability-AI/stablediffusion

The new models work fine with the old 2.0 .yaml files, I tested them.

[Miraculix200]

xformers

I don't have them installed where can I get them ?

It will install them automatically when adding --xformers to cmdline. Worked for me (before it was producing black images). Can also confirm it has less ultra hardcore christian NSFW filters now. It can produce actual booba in bikinis

[Miraculix200]

Which model did you use, pruned 786 or non ema 786?

https://huggingface.co/stabilityai/stable-diffusion-2-1/blob/main/v2-1_768-ema-pruned.ckpt

[amarand]

xformers

I don't have them installed where can I get them ?

It will install them automatically when adding --xformers to cmdline. Worked for me (before it was producing black images). Can also confirm it has less ultra hardcore christian NSFW filters now. It can produce actual booba in bikinis

Are there any pros/cons to using --xformers ? This is the first time I've needed to use them since I started using AUTO1111.

[wywywywy]

Xformers should be slightly faster and lower VRAM, but the result is probabilistic not deterministic, so every time you generate the result will look slightly different, even with the same seed.

[amarand]

Ahh okay, I had noticed that little "artifact." I don't like the fact that it can be wildly different between seeds. Was driving me nuts.

I have a 3090 Ti with 24GB of memory, so likely not needing VRAM optimizations. I also don't need it to be faster - I came from the generation that would start a job and walk away for a cup of coffee.

So the question is: it completely fails (all black images) without --xformers enabled. Is there a way to fix that bug? I've been running without xformers from the beginning.

[Kilvoctu]

Did a quick test, 2.1 seems a bit better at cars accuracy (mine, at least)

2.0

2.1

[Infinitusvoid]

Looks lot better xD

[ghost]

Apparently they heavily filtered 2.0, not just for the NSFW stuff but also images involving celebrities and the work of artists. AKA any images that could have potential copyright claims.

I guess the idea is they wanted to mitigate any potential copyright claims involving the training data, that way they can continue to release improved updated versions without liability.

It would be up to the individual to train the model from that point if they wanted XYZ person, character, or art style. They probably also plan to make it so people with less VRAM can train images more easily.

2.1 I'm guessing added some selective amount of the removed samples back in after a legal screening.

--- This is just what I heard from other commentators and its hearsay, but it makes sense in my opinion ---

[5KilosOfCheese]

Having had spent time on discord exploring 2.0 collectively and talking with Stability devs, 2.1 improved basically based on the feedback given on discord. What they spent a lot of time on was removing the issue of watermarks and copyright notices from the pictures. These were are MASSIVE problems in 2.0. So it isn't like 2.0 didn't have copyrighted material in it.

Now important to note is that 2.1 is only a fine tuned version of 2.0 base model. It is not fully trained as another model nor is it 2.0 trained more. It is a fine tuned version of 2.0.
However what was a major change in 1.x to 2.x is the change from awful black box CLIP to OpenCLIP. They also spent time training their implementation of OpenCLIP for the model. This is actually the part where artists and celebs were filtered out. Now from the image training but the words.

This is obvious from the fact that in 2.0 and 2.1 you can still call up just about any celebrity and artists with enough effort and clever prompting. However a great improvement that kept improving in 2.1 is that art styles when prompted are more defined and purer. As in if you call up renessaince painters, you get renessaince painters and style and it isn't mixing it with modern version of those paintings or mimicry of style. This is very clear in art styles like Art Deco and Art Nouveau, Dutch Golden Age. Reducing the weight of modern artists improved the overall quality of all styles.

[ghostsquad]

I just updated, I've done the following:

1. deleted venv directory
2. downloaded v2-1_512-ema-pruned.ckpt to models/Stable-diffusion/v2-1_512-ema-pruned.ckpt
3. downloaded the yaml file to models/Stable-diffusion/v2-1_512-ema-pruned.yaml
4. uninstalled python 3.8
5. installed python 3.10
6. updated the webui repo (commit hash 8eef9d8e782aa0655241e43f67059aa7bef3bdca)
7. added set ATTN_PRECISION=fp16 to my webui-user.bat
8. relauched webui-user.bat (args are --api --listen)

It's producing solid brown images. GPU is a 3060 (12GB).

Anyone know what might be going on? I've found a couple conflicting things talking about --xformers and --no-half and the set ATTN_PRECISION=fp16 environment variable. Any clarification would be appreciated!

[ghostsquad]

I solved the problem. I was using the wrong yaml file (there's one for the 512 and one for the 768).
512 == https://github.com/Stability-AI/stablediffusion/blob/main/configs/stable-diffusion/v2-inference.yaml
768 == https://github.com/Stability-AI/stablediffusion/blob/main/configs/stable-diffusion/v2-inference-v.yaml