New Stable Diffusion 2.0-v at 768x768 resolution - all download links

[AugmentedRealityCat]

https://github.com/Stability-AI/stablediffusion
https://huggingface.co/stabilityai/stable-diffusion-2
https://huggingface.co/stabilityai/stable-diffusion-2-base
https://huggingface.co/stabilityai/stable-diffusion-2-depth
https://huggingface.co/stabilityai/stable-diffusion-2-inpainting
https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler/tree/main

768 model download link on HuggingFace: https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/768-v-ema.ckpt
512 base model download link: https://huggingface.co/stabilityai/stable-diffusion-2-base/blob/main/512-base-ema.ckpt
512 depth model download link: https://huggingface.co/stabilityai/stable-diffusion-2-depth/blob/main/512-depth-ema.ckpt
512 inpainting model download link: https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/blob/main/512-inpainting-ema.ckpt
new x4 upscaler download link: https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler/blob/main/x4-upscaler-ema.ckpt

These models are not compatible with the current release (2022-11-23) as far as I can tell - it might be a local problem but so far I haven't heard about any success. Please comment if you find a solution.

New stable diffusion model (Stable Diffusion 2.0-v) at 768x768 resolution. Same number of parameters in the U-Net as 1.5, but uses OpenCLIP-ViT/H as the text encoder and is trained from scratch. SD 2.0-v is a so-called v-prediction model.

The above model is finetuned from SD 2.0-base, which was trained as a standard noise-prediction model on 512x512 images and is also made available.

Added a x4 upscaling latent text-guided diffusion model.

New depth-guided stable diffusion model, finetuned from SD 2.0-base. The model is conditioned on monocular depth estimates inferred via MiDaS and can be used for structure-preserving img2img and shape-conditional synthesis.

d2i

A text-guided inpainting model, finetuned from SD 2.0-base.

We follow the original repository and provide basic inference scripts to sample from the models.

[kalkal11]

so... I imagine new code will be required to use this?

[AugmentedRealityCat]

Indeed. I tried it and it gave me an error message before I could test anything. I've posted a feature request about it.
The upscaler is using a ckpt format, so I guess it's not compatible either, but I haven't really tried it.

[bbecausereasonss]

Nothing is compatible the code needs slight adjustment.

[bbecausereasonss]

So excited. Is Automatic still alive?

[hemangjoshi37a]

Long live automatic https://hjlabs.in

[eypacha]

Are you spamming? Seriously?

[Kinglink]

considering he merged a pull request 4 days ago, I would say it looks likely he's alive.

Will he continue supporting this Web UI? will he support V2? Will he take on the workload and continue maintaining this repo? Or will someone fork the repo and make it work?... these are all questions we don't have answers to.. Stay tuned, same bat time same bat channel!

[lordcasey]

You need to switch the branch to the SD 2.0 branch

Does this fix the issue for you? I'm still getting this error on the latest version of the sd-2.0 branch:

size mismatch for model.diffusion_model.output_blocks.11.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).

[slymeasy]

Does this fix the issue for you? I'm still getting this error on the latest version of the sd-2.0 branch:

No. Coincidently, I saw Cheeze in the Stable Diffusion discord, by chance and asked him about that error. He said something like "check the commits" and that maybe there were some other forks that I may have a better chance with. He put that up for the people who can figure it out, not for it to be super easy like what Auto did.

[AugmentedRealityCat]

You need to switch the branch to the SD 2.0 branch

Does this fix the issue for you? I'm still getting this error on the latest version of the sd-2.0 branch:

size mismatch for model.diffusion_model.output_blocks.11.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).

One thing you should try is keeping only the 768 model in your model folder as the code will try to load the first model it sees in the list, and if you have all of them it will be a 512x512 model that has is not functioning yet with this repo.
See comment over here: #5008 (reply in thread)

[Atomizer74]

not tried yet but https://github.com/MrCheeze/stable-diffusion-webui/commits/sd-2.0

This worked pretty good for me, cloned the repo using the github windows client, swapped to make sure I was on the SD-2.0 branch, ran the webui-user.bat, let it download all the requirements, put the 768 model in the model folder, used the SD2 demo to generate a prompt, then successfully replicated the same prompt within my local copy(Using Euler sampler since DDIM threw an error, even though I remembered seeing something about DDIM in the release info).
Have yet to take a look at any of the other ckpt of SD2 myself, but the 768 model is at least in a working state within the webui.

Edit: Just wanted to add I guess with just these changes old models are broken, but I guess that makes sense with such a change in the underlaying data structure.

[dminGod]

@Miraculix200 thanks for the resolution. Https://hjlabs.in

Do not spam here

[ghost]

wakey wakey, AUTOMATIC1111

[kestrel99]

This is not a good way to motivate people who are working for free to help you.

[orlop]

Agreed. On the other hand, I wonder how many people keep refreshing the page and hoping AUTOMATIC1111 is in insane productivity mode right now :D

Thanks for all the hard work!

[ghost]

I dont know how many times I thanked him here and explained that his work should not be taken for granted, so I will allow myself this comment for now. I agree with the "thumbs down" count on my post completely.

[kybercore]

I dont know how many times I thanked him here and explained that his work should not be taken for granted, so I will allow myself this comment for now. I agree with the "thumbs down" count on my post completely.

Wholesome github moment

[gwbyrd]

wakey wakey, AUTOMATIC1111

I took this comment as entirely light-hearted LOL. If you had said, "Hey, get to work!", that would have been a different story! And, yes, c'mon, we all know A1111 rules! My goodness, what an amazing interface to SD! Can't wait to see it updated :)

[Miraculix200]

Looks like someone made it work with the webui on this Colab notebook. Didn't test it yet. It will break some things
https://colab.research.google.com/drive/1PvNyEWIhDU_D-i15DzpPjqDQkbYv_6Hu

https://www.reddit.com/r/StableDiffusion/comments/z3e2fw/stable_diffusion_20_on_google_colab/

[camenduru]

https://github.com/camenduru/stable-diffusion-diffusers-colab/blob/main/pytorch_stable_diffusion_2.ipynb

[Kallamamran]

Yeah runs in Colab on A100 if you're lucky. Does not run locally on GTX/RTX or in A1111

[githubUser01946]

I think git clone -b sd-2.0 https://github.com/MrCheeze/stable-diffusion-webui.git should work. I did that on free colab and it worked. But it crashed because of not enough system RAM

Ok did that but the commit hash is different, I dont think it has installed the SD2.0 version.

The commit hash for what was installed is: 3a33064
The hash for the SD file one is: 069591b

[Miraculix200]

Sorry, on Colab I actually added this piece of code to the notebook just before !python launch.py:
import os
os.environ["STABLE_DIFFUSION_REPO"] = "https://github.com/Stability-AI/stablediffusion"
os.environ["STABLE_DIFFUSION_COMMIT_HASH"] = "33910c386eaba78b7247ce84f313de0f2c314f61"

!rm -rf /content/stable-diffusion-webui/repositories/stable-diffusion/
!mkdir -p /content/stable-diffusion-webui/repositories/stable-diffusion/
%cd /content/stable-diffusion-webui/repositories/
!git clone https://github.com/Stability-AI/stablediffusion stable-diffusion

But with the local installation that wasn't necessary. I thought it was because the repositories folder is on Google Drive (symlink)
Anyway, if you get commit hash issues, you may want to try something similar locally.

[ifiwasakid]

okay gals and pals, "found" something that might be interesting ... possibly even helpful, who knows ... so when trying to verify i had switched branches (my initial git clone request was done "normally" and i used the "git checkout -b sd-2.0" method and althought i have no reason to assume that this was the reason for the failure, i did -- like many other people -- get the "size mismatch" error when doing it this way ... i should also add that both attempts so far were done using mklink, and i am trying again now without a linked ckpt file as i DO have reason to believe -- based on the CLI output that cited the link with TWO SLASHES in every position (ie. my//files//like//this/.ckpt) ... anyhow ... in trying to verify the branch switch ... I FOUND THE FOLLOWING "NEW" BRANCHES ... NOT SURE WHAT TO DO with them but i am currently trying to switch to one of them (sd-2.0-automatic-2 -> origin/sd-2.0-automatic-2) by performing the following line:

C:\Users\nik\stable-diffusion-webui-2>git branch --set-upstream-to=origin/sd-2.0-automatic-2
branch 'sd-2.0-automatic-2' set up to track 'origin/sd-2.0-automatic-2'

OKAY SO ... I GOT IT RUNNING!
I want to be VERY CLEAR... i have NO idea if changing branches had ANYthing to do with my success because, right before that, and without launching to verify, i removed the linked files and added the 726 file from a FULL COPY ... and left it first in the directory ... i changed the others to start with "zzz" for now ... any how ... START WITH HARD COPIES OF THE NEW CKPT and if that doesn't work, look to these new branches ...

remote: Counting objects: 100% (198/198), done.
remote: Compressing objects: 100% (49/49), done.
Receiving objects: 100% (238/238), 124.34 KiB | 24.87 MiB/s, done. 40

Resolving deltas: 100% (160/160), completed with 38 local objects.
From https://github.com/MrCheeze/stable-diffusion-webui
 * [new branch]      dev2-cheeze        -> origin/dev2-cheeze
 * [new branch]      sd-2.0-automatic   -> origin/sd-2.0-automatic
 * [new branch]      sd-2.0-automatic-2 -> origin/sd-2.0-automatic-2
Already up to date.

'C:\Users\nik\stable-diffusion-webui-2>git checkout -b sd-2.0-automatic-2
Switched to a new branch 'sd-2.0-automatic-2'

C:\Users\nik\stable-diffusion-webui-2>git pull
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details.

    git pull <remote> <branch>

If you wish to set tracking information for this branch you can do so with:

    git branch --set-upstream-to=origin/<branch> sd-2.0-automatic-2

[ifiwasakid]

the output i got for "success":

C:\Users\nik\stable-diffusion-webui-2>for / %i in (extensions\*) do @git -C "%i" pull --recurse-submodules
venv "C:\Users\nik\stable-diffusion-webui-2\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 3a330644ecdd27906fdaa4af98de7faf1a65dee6
Installing requirements for Web UI
Launching Web UI with arguments: --config repositories/stable-diffusion/configs/stable-diffusion/v2-inference-v.yaml
No module 'xformers'. Proceeding without it.
LatentDiffusion: Running in v-prediction mode
DiffusionWrapper has 865.91 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Downloading: 100%|████████████████████████████████████████████████████████████████| 3.94G/3.94G [05:52<00:00, 11.2MB/s]
Loading weights [2c02b20a] from C:\Users\nik\stable-diffusion-webui-2\models\Stable-diffusion\768-v-ema.ckpt
Global Step: 140000
Applying cross attention optimization (Doggettx).
Model loaded.
Loaded a total of 0 textual inversion embeddings.
Embeddings:
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

NOT SURE ABOUT THE "MISSING XFORMERS MODULE" line though ...
hesitant to try installing it immediately after getting her working, lol

[Kilvoctu]

If you're unaware, this repo main branch has the v2.0 support: ce69111 (well, the current commit at the moment is b5050ad)

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#stable-diffusion-20

[Koningsbruggen]

I love you AUTOMATIC1111, I use your software every day with great pleasure, I would be happy to contribute to a Patreon.

[Nsanity03]

I look forward to the web ui update as if it were Christmas, I am curious to try the new version, thanks for the free work you are doing for us noobs :)

[AugmentedRealityCat]

So you're waiting for Santa Clau Tomatic 1111 ;)

[Barbieroalex]

[DarkAlchy]

Not selecting a licence is partly to blame, and as Emad said they cannot officially support Automatic1111 until he sets a licence, which he refuses to do.

[githubUser01946]

Tried to install Mr Cheeze version via the instruction given to me above, it installed the wrong version so I have deleted the folder (hopefully that will uninstall it) and will wait for Automatic to do something.

[MancV21]

I'm not one of these "Where's the update" people, but does anyone know if AUTO is even going to update his UI? I've not seen any official word from him yet that he's going to bother at all. Willing to wait as long as it takes but if he's not, I'll start to look elsewhere, would prefer to carry on using his webUI though.

[Miraculix200]

The necessary changes aren't very much work. So it may be updated in one weekend

[eypacha]

You have every right to look everywhere, no one owes anyone anything. This is an open source project. We can help developing the feature and make a pull request if we don't want to wait. If we don't want to or can't, we just have to be patient

[MancV21]

You have every right to look everywhere, no one owes anyone anything. This is an open source project. We can help developing the feature and make a pull request if we don't want to wait. If we don't want to or can't, we just have to be patient

Absolutely, which is what I said. I'm just asking if anyone had heard officially if Auto was going to update his WebUI. :)

[eypacha]

Hi and sorry, maybe I misunderstand (I don't speak English natively). The phrase "he's going to bother at al" sounded like a claim to me. It's not even a week since the new version came out. A1111 is a famous anonymous now, but maybe they works, they has a family. I think the community is so big that dozens of different branches will be born.

I hope it comes out soon, I'm looking forward to web-user.bating and bringing a new pull.

[MancV21]

Hi and sorry, maybe I misunderstand (I don't speak English natively). The phrase "he's going to bother at al" sounded like a claim to me. It's not even a week since the new version came out. A1111 is a famous anonymous now, but maybe they works, they has a family. I think the community is so big that dozens of different branches will be born.

I hope it comes out soon, I'm looking forward to web-user.bating and bringing a new pull.

No, asking if someone is going to "bother" doing something is just how we ask if someone is going to attempt something in the north of England. There is no claims made by me or any pressure at all, I was just asking if Auto had said he was going to work on the update. The choice is totally his, if he doesn't want to, then all we can do is thank him for his work that he has done for the community (which is more than most I'd say) and respect his wishes. If he WAS working on the update (at any time) then I'm willing to wait until his webUI was updated rather than move elsewhere, as I prefer his work. :)

[Nexus77777]

With the latest GRisk GUI you can let Stable diffusion 2.0 run locally, (but its patreon)

[galexa]

it's not working for everyone though :(

[Nexus77777]

It works for me in GRisk, but to be honest, I prefer 1.4 or 1.5 over 2.0. Yes, the details are much better at 768x768; however, I tested it with my old prompts, and it does not produce nearly as good results in terms of "making sense," and the photo style produces blurry or grainy images quite often. Maybe it needs a different type of prompt.

[Nexus77777]

...and it still does not understand how a human hand should look like :-)

[officer-kd6dash3dot7]

That's not surprising, most humans can't draw hands well either.

[hypnotoadie]

Pretty sure Auto said it was midterms week for him on a thread. And if so, good luck m8. We can wait!

[choephix]

@AUTOMATIC1111 Go get em!

[jav12z]

This repo can work with the 512 and the 768 model.

https://github.com/uservar/stable-diffusion-webui

[officer-kd6dash3dot7]

What has been your experience with it, does it have any issues? Anything broken? TIA

[jimschweizer]

The 512-base-ema.ckpt is delivering images, but I wouldn't say the first batch was not much of an improvement over 1.5. The 768 model fails to load.

[Atomizer74]

I saw some SD2.0 stuff in the commits, but unless I am somehow on the wrong branch(dev2?), this doesn't load either 512 or 768 models, gives the standard size mismatch for model errors.
I confirmed the repository was working with another model and was generating images fine, and I didn't customize anything on my end, just straight up cloned into a fresh directory.

Edit: I take it back, did a fresh clone, this time only copying the SD2.0 models, it loaded the 512 model successfully, it might have had something to do with loading the old model first, and thus not loading the SD2.0 wrapper.

Edit2: Both the 512 base and 768 models appear to be working, though it appears to have a pickling issue with the midas model when loading the depth model.
`FileNotFoundError: [Errno 2] No such file or directory: 'midas_models/dpt_hybrid-midas-501f0c75.pt'

The file may be malicious, so the program is not going to read it.
You can skip this check with --disable-safe-unpickle commandline argument.`
I grabbed it from the official sources, so it is probably safe, but I think I will just wait before crossing that bridge.

[officer-kd6dash3dot7]

Wise. Thanks for your efforts.

[jav12z]

It has some issues when you change models, and sometimes I've needed to restart the app. I still haven't tried anything special just picture generation.

Xformers works, Styles works (sometimes), Variation Strength works. I thought I could use other homebrew models that are appearing but it's not the case.

Hypernetworks and embeddings dont work, and also can not be trainned :(

So far I didn't try anything more.

[ioma8]

FYI automatic made commit to master 37 minutes ago, so he is alright! I think its great news. Lets not push him though. Its ton work he did already.

[galexa]

copy paste it in a text file and save it?

Yes. (or just save the source of the web page)

[Koningsbruggen]

thank you

[Koningsbruggen]

Hmm I get an error. Did anyone else get it to work?

[Koningsbruggen]

[Koningsbruggen]

it give a size mismatch for model.diffusion_model. error

[enn-nafnlaus]

Is any of this supposed to actually work with Automatic1111?

Loading config from: /scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/models/Stable-diffusion/768-v-ema.yaml
LatentDiffusion: Running in v-prediction mode
Traceback (most recent call last):
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/venv/lib64/python3.10/site-packages/gradio/routes.py", line 284, in run_predict
output = await app.blocks.process_api(
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/venv/lib64/python3.10/site-packages/gradio/blocks.py", line 982, in process_api
result = await self.call_function(fn_index, inputs, iterator)
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/venv/lib64/python3.10/site-packages/gradio/blocks.py", line 824, in call_function
prediction = await anyio.to_thread.run_sync(
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/venv/lib64/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/venv/lib64/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/venv/lib64/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/modules/ui.py", line 427, in update_token_counter
tokens, token_count, max_length = max([model_hijack.tokenize(prompt) for prompt in prompts], key=lambda args: args[1])
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/modules/ui.py", line 427, in
tokens, token_count, max_length = max([model_hijack.tokenize(prompt) for prompt in prompts], key=lambda args: args[1])
File "/scratch/StableDiffusion/AUTOMATIC1111/stable-diffusion-webui/modules/sd_hijack.py", line 134, in tokenize
_, remade_batch_tokens, _, _, _, token_count = self.clip.process_text([text])
AttributeError: 'NoneType' object has no attribute 'process_text'
DiffusionWrapper has 865.91 M params.

[mykeehu]

This is what you get when you change the prompt at the model change. You don't need to bother with it.

[enn-nafnlaus]

And is anything other than 768 supposed to work at present? The 512s don't even seem to finish loading, they just yield "error" and I can't do anything until I restart. x4, same story.

[eypacha]

Hello! maybe an obvious question. I was reading that in this implementation it only works with the sd2 768 model, it means it doesn't work with the other models listed in this thread, but it still works all the functions and models of sd1.5, right?

[Kilvoctu]

Yes, only that SD2.0 768 model, for now.
All your models based on SD1.5 and prior should still work.

[ZeroCool22]

Hello! maybe an obvious question. I was reading that in this implementation it only works with the sd2 768 model, it means it doesn't work with the other models listed in this thread, but it still works all the functions and models of sd1.5, right?

It works with 512 too and even the inpainting!

You need to have the models like this...

💡IMPORTANT: You need the YAML for the 512 version, the one posted here only works with the 768.

[eypacha]

So I don't need to worry for my collection of 100gb of custom models hahah... As a precaution i was not gitpulling and testing 2.0 on another clone. Day will be a long night. Thanks

[ZeroCool22]

So I don't need to worry for my collection of 100gb of custom models hahah... As a precaution i was not gitpulling and testing 2.0 on another clone. Day will be a long night. Thanks

Yeah, but AUTO's did a great work keeping the compatibility with the old models too.

[ZeroCool22]

@eypacha

Here is the YAML for the 512 you need:

https://github.com/Stability-AI/stablediffusion/tree/main/configs/stable-diffusion

The one without "V" at the end.

[Atomizer74]

Seems to be working just fine to me

[specblades]

Seems to be working just fine to me

can u share yaml file?

u r running in eps mode. Its a joke or base model is different

[Atomizer74]

Seems to be working just fine to me

can u share yaml file?

u r running in eps mode. Its a joke or base model is different

All I did was go to this folder within the webui, copied the yamls to my SD2.0 folder and renamed accordingly

v2-inference.yaml -> 512-base-ema.yaml
v2-inference-v.yaml -> 768-v-ema.yaml
v2-inpainting-inference.yaml -> 512-inpainting-ema.yaml
v2-midas-inference.yaml -> 512-depth-ema.yaml
x4-upscaling.yaml -> x4-upscaler-ema.yaml

The only models I have used was the 512 base and 768, I was just trying to use the upscaler without success(No idea how that is meant to work, but it loaded without issue)

Edit: Oh, also wanted to add, the only models that seems to use the v-prediction is the 768 model, and I guess the upscaler? The 512 base model uses eps and runs fine.

Edit2: Final note, that is just what the webui pulls from the official stability ai repository, you can find the same configs here:
https://github.com/Stability-AI/stablediffusion/tree/main/configs/stable-diffusion

Edit3: Just noticed that is the exact same link ZeroCool22 posted above, the info was already here.

[enn-nafnlaus]

Seems to be working just fine to me

That is not what stock Automatic1111 looks like...

[Atomizer74]

Seems to be working just fine to me

That is not what stock Automatic1111 looks like...

Correct, I have some extensions and added settings in here to have them always at the top, but otherwise it is standard automatic1111.

Edit: forgot the setting shot

Even a commit or two behind, but you can see I haven't customized anything except replacing the Stable-diffusion model folder with a symlink(That is why the .txt file is missing), and webui-user.bat customization, I haven't done anything weird to get SD2.0 working, that is all automatic1111

[gwbyrd]

Anyone have a guess as to why any of these updates would break the randomize or prompt_travel scripts? I haven't changed anything except to do daily "git pull", but after it updated to v2.0, I am getting "error loading script: randomize.py" and the same thing for prompt_travel. The error is "Cannot import name 'FrozenCLIPEmbedderWithCustomWords' from module.sd_hijack." and "Cannot import name 'build_samplers_dict' from 'scripts.xy_grid'".

[gwbyrd]

Well, I just deleted and reinstalled the extensions and they seem to work now, but curious as to what broke them in the first place. Guess it doesn't matter, but surprised no one else seems to have posted about any similar problems.

[ifiwasakid]

does anyone know if the "NO MODULE XFORMERS" line is something that we should be concerned about? do we need to download that using PIP or somthing?

also has anyone else tried using symlinked files only to see the error that it's looking for the file using // double forward slash marks like this c://the//file//isnt//here//cus//theres//toomany//slashes
is that something findable and fixable in the code? so i don't have three versions of ckpt files now?

[enn-nafnlaus]

It's something you should be "concerned" about, in that you're missing out. ;) Missing it doesn't hurt anything, but having it makes generation faster and more memory efficient (a lot) - it's a nice upgrade! But it's also a PITA to set up, esp. on some OSes (like Fedora 36). "Oh no, that's too new! Oh no, that's too old!" Aargh.

Finally got it to work by installing Homebrew, downgrading my compiler to GCC11, manually installing half a million CUDA dependencies, and then following the instructions here:

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Xformers

A global install (root) doesn't seem to work. Nor does an install as the StableDiffusion user. Needs to be done within SD's activate environment. Note: I was initially confused by not finding a ".venv" directory. That is to say, I misread; it is, of course, "./venv"!

BTW: xformers is also the package that contains Dreambooth, so there is that.... ;)

[ifiwasakid]

okay well i guess it's NOT WORKING for EVERYONE yet, and i'm gonna have to go back to MrCheese, because this is what i got after doing everything, yaml files and all ...

C:\Users\nik\stable-diffusion1111-2>webui-user.bat
venv "C:\Users\nik\stable-diffusion1111-2\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 4b3c5bc24bffdf429c463a465763b3077fe55eb8
Installing requirements for Web UI
Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading weights [2c02b20a] from C:\Users\nik\stable-diffusion1111-2\models\Stable-diffusion\768-v-ema.ckpt
Traceback (most recent call last):
  File "C:\Users\nik\stable-diffusion1111-2\launch.py", line 294, in <module>
    start()
  File "C:\Users\nik\stable-diffusion1111-2\launch.py", line 289, in start
    webui.webui()
  File "C:\Users\nik\stable-diffusion1111-2\webui.py", line 131, in webui
    initialize()
  File "C:\Users\nik\stable-diffusion1111-2\webui.py", line 61, in initialize
    modules.sd_models.load_model()
  File "C:\Users\nik\stable-diffusion1111-2\modules\sd_models.py", line 261, in load_model
    load_model_weights(sd_model, checkpoint_info)
  File "C:\Users\nik\stable-diffusion1111-2\modules\sd_models.py", line 192, in load_model_weights
    model.load_state_dict(sd, strict=False)
  File "C:\Users\nik\stable-diffusion1111-2\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
        size mismatch for model.diffusion_model.input_blocks.1.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.1.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.input_blocks.1.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.input_blocks.1.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.2.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.2.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.input_blocks.2.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.input_blocks.2.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.4.1.proj_in.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.4.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.input_blocks.4.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.input_blocks.4.1.proj_out.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.5.1.proj_in.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.5.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.input_blocks.5.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.input_blocks.5.1.proj_out.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.7.1.proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.7.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.input_blocks.7.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.input_blocks.7.1.proj_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.8.1.proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.input_blocks.8.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.input_blocks.8.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.input_blocks.8.1.proj_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.middle_block.1.proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.middle_block.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.middle_block.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.middle_block.1.proj_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.3.1.proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.3.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.output_blocks.3.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.output_blocks.3.1.proj_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.4.1.proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.4.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.output_blocks.4.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.output_blocks.4.1.proj_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.5.1.proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.5.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.output_blocks.5.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([1280, 1024]) from checkpoint, the shape in current model is torch.Size([1280, 768]).
        size mismatch for model.diffusion_model.output_blocks.5.1.proj_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.6.1.proj_in.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.6.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.output_blocks.6.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.output_blocks.6.1.proj_out.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.7.1.proj_in.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.7.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.output_blocks.7.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.output_blocks.7.1.proj_out.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.8.1.proj_in.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.8.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.output_blocks.8.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([640, 1024]) from checkpoint, the shape in current model is torch.Size([640, 768]).
        size mismatch for model.diffusion_model.output_blocks.8.1.proj_out.weight: copying a param with shape torch.Size([640, 640]) from checkpoint, the shape in current model is torch.Size([640, 640, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.9.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.output_blocks.9.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.10.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.10.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.output_blocks.10.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.output_blocks.10.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.11.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
        size mismatch for model.diffusion_model.output_blocks.11.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.output_blocks.11.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
        size mismatch for model.diffusion_model.output_blocks.11.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
Press any key to continue . . .

C:\Users\nik\stable-diffusion1111-2>

[Atomizer74]

One thing I am not seeing:

Loading config from: D:\AI Generation\stable-diffusion-webui\models\Stable-diffusion\SD20\768-v-ema.yaml
LatentDiffusion: Running in v-prediction mode
DiffusionWrapper has 865.91 M params.
Loading weights [2c02b20a] from D:\AI Generation\stable-diffusion-webui\models\Stable-diffusion\SD20\768-v-ema.ckpt

You are not loading a yaml file at all, which means the yaml file is not there, and as a result it is running in eps-prediction mode instead of v-prediction mode.

[specblades]

Trying to load new 512x models and got this.
Any help?

File "D:\AI_WORKPLACE\AUTOMATIC1111\alwaysuptodate\stable-diffusion-webui\venv\lib\site-packages\yaml\scanner.py", line 129, in peek_token self.fetch_more_tokens() File "D:\AI_WORKPLACE\AUTOMATIC1111\alwaysuptodate\stable-diffusion-webui\venv\lib\site-packages\yaml\scanner.py", line 223, in fetch_more_tokens return self.fetch_value() File "D:\AI_WORKPLACE\AUTOMATIC1111\alwaysuptodate\stable-diffusion-webui\venv\lib\site-packages\yaml\scanner.py", line 577, in fetch_value raise ScannerError(None, None, yaml.scanner.ScannerError: mapping values are not allowed here in "D:\AI_WORKPLACE\AUTOMATIC1111\alwaysuptodate\stable-diffusion-webui\models\Stable-diffusion\512-base-ema.yaml", line 28, column 66

[swankwc]

Can both the original and 2.0 coexist within 1111?

[ZeroCool22]

Yes.

[Renaldas111]

Is x4 upscaler working for you?
I put ckpt of it inside a folder with all the models, downloaded yaml, renamed it to the same name as the upscaler, and getting:

Loading config from: I:\stable-diffusion-webui\models\Stable-diffusion\x4-upscaler-ema.yaml
LatentUpscaleDiffusion: Running in v-prediction mode
DiffusionWrapper has 473.40 M params.
Loading weights [5c5661de] from I:\stable-diffusion-webui\models\Stable-diffusion\x4-upscaler-ema.ckpt
Loading VAE weights from: I:\stable-diffusion-webui\models\Stable-diffusion\vae\vae-ft-mse-840000-ema-pruned.ckpt
Traceback (most recent call last):
File "I:\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 284, in run_predict
output = await app.blocks.process_api(
File "I:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 982, in process_api
result = await self.call_function(fn_index, inputs, iterator)
File "I:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 824, in call_function
prediction = await anyio.to_thread.run_sync(
File "I:\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "I:\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "I:\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run
result = context.run(func, *args)
File "I:\stable-diffusion-webui\modules\ui.py", line 1603, in
fn=lambda value, k=k: run_settings_single(value, key=k),
File "I:\stable-diffusion-webui\modules\ui.py", line 1444, in run_settings_single
if not opts.set(key, value):
File "I:\stable-diffusion-webui\modules\shared.py", line 477, in set
self.data_labels[key].onchange()
File "I:\stable-diffusion-webui\modules\call_queue.py", line 15, in f
res = func(*args, **kwargs)
File "I:\stable-diffusion-webui\webui.py", line 62, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights()))
File "I:\stable-diffusion-webui\modules\sd_models.py", line 292, in reload_model_weights
load_model(checkpoint_info)
File "I:\stable-diffusion-webui\modules\sd_models.py", line 261, in load_model
load_model_weights(sd_model, checkpoint_info)
File "I:\stable-diffusion-webui\modules\sd_models.py", line 227, in load_model_weights
sd_vae.load_vae(model, vae_file)
File "I:\stable-diffusion-webui\modules\sd_vae.py", line 153, in load_vae
load_vae_dict(model, vae_dict_1)
File "I:\stable-diffusion-webui\modules\sd_vae.py", line 180, in load_vae_dict
model.first_stage_model.load_state_dict(vae_dict_1)
File "I:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for AutoencoderKL:
Unexpected key(s) in state_dict: "encoder.down.3.block.0.norm1.weight", "encoder.down.3.block.0.norm1.bias", "encoder.down.3.block.0.conv1.weight", "encoder.down.3.block.0.conv1.bias", "encoder.down.3.block.0.norm2.weight", "encoder.down.3.block.0.norm2.bias", "encoder.down.3.block.0.conv2.weight", "encoder.down.3.block.0.conv2.bias", "encoder.down.3.block.1.norm1.weight", "encoder.down.3.block.1.norm1.bias", "encoder.down.3.block.1.conv1.weight", "encoder.down.3.block.1.conv1.bias", "encoder.down.3.block.1.norm2.weight", "encoder.down.3.block.1.norm2.bias", "encoder.down.3.block.1.conv2.weight", "encoder.down.3.block.1.conv2.bias", "encoder.down.2.downsample.conv.weight", "encoder.down.2.downsample.conv.bias", "decoder.up.3.block.0.norm1.weight", "decoder.up.3.block.0.norm1.bias", "decoder.up.3.block.0.conv1.weight", "decoder.up.3.block.0.conv1.bias", "decoder.up.3.block.0.norm2.weight", "decoder.up.3.block.0.norm2.bias", "decoder.up.3.block.0.conv2.weight", "decoder.up.3.block.0.conv2.bias", "decoder.up.3.block.1.norm1.weight", "decoder.up.3.block.1.norm1.bias", "decoder.up.3.block.1.conv1.weight", "decoder.up.3.block.1.conv1.bias", "decoder.up.3.block.1.norm2.weight", "decoder.up.3.block.1.norm2.bias", "decoder.up.3.block.1.conv2.weight", "decoder.up.3.block.1.conv2.bias", "decoder.up.3.block.2.norm1.weight", "decoder.up.3.block.2.norm1.bias", "decoder.up.3.block.2.conv1.weight", "decoder.up.3.block.2.conv1.bias", "decoder.up.3.block.2.norm2.weight", "decoder.up.3.block.2.norm2.bias", "decoder.up.3.block.2.conv2.weight", "decoder.up.3.block.2.conv2.bias", "decoder.up.3.upsample.conv.weight", "decoder.up.3.upsample.conv.bias".

[Atomizer74]

I haven't tried the upscaler yet, however one thing I did notice in your log is it is running in v-prediction mode, I could be mistaken, but I believe only the 768 model uses v-prediction, the others use eps-prediction(Unless I am wrong), if you copied the 768 yaml, you might be able to get away with opening it up and deleting the parameterization: "v" line(the whole line):

model:
  base_learning_rate: 1.0e-4
  target: ldm.models.diffusion.ddpm.LatentDiffusion
  params:
    parameterization: "v"
    linear_start: 0.00085

I had compared all the yamls, and that seemed to be the only difference I could find between the 512 one and the 768 one.

Edit: Thinking about it, I don't even know how to use the upscaler model, is it just img2img and setting the resolution higher with the upscaler model selected? The only upscaling I have done is with the SD upscale script, and the extras tab upscale, which doesn't use the SD model.

Edit again: My yaml also had the v prediction, and appeared to load fine, I don't have a vae selected though, maybe something to do with that? Going to do some testing if I can figure out how to use it.

Edit3: Yeah no idea how to use it, took a 512x512 image on img2img with the upscaler model selected, set resolution to 1024x1024 and got
RuntimeError: The size of tensor a (256) must match the size of tensor b (128) at non-singleton dimension 3
Set the resolution to 512x512 with no other changes and got
RuntimeError: The size of tensor a (128) must match the size of tensor b (64) at non-singleton dimension 3

So that is something I need to learn.

[rulare]

I'm probably making a dumb mistake somewhere. I have the 768 model (2c02b20a) and v2-inference.yaml (renamed to match) installed into a brand new pull, and it does start properly, but it will only generate extremely speckled and blurry images regardless of step count. Any ideas?

[rulare]

Yep, I downloaded the wrong yaml. That's exactly the kind of stupid mistake I figured it would be! Needed the one with the v. As long as it's named correctly, it still loaded with an incorrect yaml for me. Just not with no yaml.

Thanks!

[rulare]

I just noticed, how did you get the hypernetwork settings and such above your tabs there? I don't have that in my newest pull from this evening, but it seems super useful

[AugmentedRealityCat]

I just noticed, how did you get the hypernetwork settings and such above your tabs there? I don't have that in my newest pull from this evening, but it seems super useful

These are called quicksettings. Basically, you can list any parameter used by the program and have it exposed for control on the WebUI.
For example, I am using one to select the model, and a second one to set the masking strength in inpainting mode. Here is the code I've put under "Quicksettings" in the WebUI settings tab to achieve that:
sd_model_checkpoint, inpainting_mask_weight
But you can add as many as you want. Be aware that there is no clean formatting applied, so it can quickly get cluttered up there at the top of the page.

[Atomizer74]

I just noticed, how did you get the hypernetwork settings and such above your tabs there? I don't have that in my newest pull from this evening, but it seems super useful

In the settings, in the User interface section:

The specific ones I am using:
sd_model_checkpoint, sd_vae, sd_hypernetwork, sd_hypernetwork_strength, CLIP_stop_at_last_layers, inpainting_mask_weight, use_scale_latent_for_hires_fix

By default I believe it just has sd_model_checkpoint, but you could even remove that.

To find it for other settings, what I do is just right click on the setter(checkbox/dropdown/slider) and click inspect, which takes you to the source code, then find the <div id="setting_*" to find the name to put in there:

So setting_sd_hypernetwork becomes sd_hypernetwork in the Quicksettings list.

[githubUser01946]

I'm probably making a dumb mistake somewhere. I have the 768 model (2c02b20a) and v2-inference.yaml (renamed to match) installed into a brand new pull, and it does start properly, but it will only generate extremely speckled and blurry images regardless of step count. Any ideas?

You have to rename the .yaml file to match what your model is called before the .ckpt extension.

[SAC020]

I don't understand what I need to do to load the 2.0 768 ckpt (768-v-ema.ckpt, 5GB size)

• git pull reports up to date
• I have not "switched branches", I don't know how to do that (and if it is needed)
• I have the yaml files, and I definitely haven't put them there myself, they must have been autoinstalled by some process
• trying to load the ckpt returns the following

Loading weights [2c02b20a] from C:\ai\stable-diffusion-webui\models\Stable-diffusion\768-v-ema.ckpt
Traceback (most recent call last):
  File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 284, in run_predict
    output = await app.blocks.process_api(
  File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 982, in process_api
    result = await self.call_function(fn_index, inputs, iterator)
  File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 824, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\ai\stable-diffusion-webui\modules\ui.py", line 1603, in <lambda>
    fn=lambda value, k=k: run_settings_single(value, key=k),
  File "C:\ai\stable-diffusion-webui\modules\ui.py", line 1444, in run_settings_single
    if not opts.set(key, value):
  File "C:\ai\stable-diffusion-webui\modules\shared.py", line 477, in set
    self.data_labels[key].onchange()
  File "C:\ai\stable-diffusion-webui\modules\call_queue.py", line 15, in f
    res = func(*args, **kwargs)
  File "C:\ai\stable-diffusion-webui\webui.py", line 62, in <lambda>
    shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights()))
  File "C:\ai\stable-diffusion-webui\modules\sd_models.py", line 302, in reload_model_weights
    load_model_weights(sd_model, checkpoint_info)
  File "C:\ai\stable-diffusion-webui\modules\sd_models.py", line 192, in load_model_weights
    model.load_state_dict(sd, strict=False)
  File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
        size mismatch for model.diffusion_model.input_blocks.1.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).

(and it continues like that with "size mismatch" errors

[githubUser01946]

Someone has shared above what the yaml files should ne named as. Please go look.

[SAC020]

Renamed v2-inference-v.yaml to 768-v-ema.yaml and I am getting the same error

[Atomizer74]

You need to put the yaml files in the same folder as the ckpt, they won't do anything in that folder, like this:

[SAC020]

Tried that too, same problem. I don't have all the ckpts

Btw, I've used SD1.5 without yaml files (as far as I know), 2.0 is the first time I hear about yaml

[bbecausereasonss]

Any word on getting the upscaler working or another repo I can try it with?

[SAC020]

Is the depth img2img supposed to be working?

Getting this error when using it for img2img. Other ckpts work.

RuntimeError: Given groups=1, weight of size [320, 5, 3, 3], expected input[2, 9, 64, 64] to have 5 channels, but got 9 channels instead

[randomname1212]

https://github.com/Stability-AI/stablediffusion https://huggingface.co/stabilityai/stable-diffusion-2 https://huggingface.co/stabilityai/stable-diffusion-2-base https://huggingface.co/stabilityai/stable-diffusion-2-depth https://huggingface.co/stabilityai/stable-diffusion-2-inpainting https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler/tree/main

768 model download link on HuggingFace: https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/768-v-ema.ckpt 512 base model download link: https://huggingface.co/stabilityai/stable-diffusion-2-base/blob/main/512-base-ema.ckpt 512 depth model download link: https://huggingface.co/stabilityai/stable-diffusion-2-depth/blob/main/512-depth-ema.ckpt 512 inpainting model download link: https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/blob/main/512-inpainting-ema.ckpt new x4 upscaler download link: https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler/blob/main/x4-upscaler-ema.ckpt

These models are not compatible with the current release (2022-11-23) as far as I can tell - it might be a local problem but so far I haven't heard about any success. Please comment if you find a solution.

New stable diffusion model (Stable Diffusion 2.0-v) at 768x768 resolution. Same number of parameters in the U-Net as 1.5, but uses OpenCLIP-ViT/H as the text encoder and is trained from scratch. SD 2.0-v is a so-called v-prediction model.
The above model is finetuned from SD 2.0-base, which was trained as a standard noise-prediction model on 512x512 images and is also made available.
Added a x4 upscaling latent text-guided diffusion model.
New depth-guided stable diffusion model, finetuned from SD 2.0-base. The model is conditioned on monocular depth estimates inferred via MiDaS and can be used for structure-preserving img2img and shape-conditional synthesis.
d2i
A text-guided inpainting model, finetuned from SD 2.0-base.
We follow the original repository and provide basic inference scripts to sample from the models.

I watched this video on youtube and managed to get the base version of SD2.0 to work with Automatic1111 https://www.youtube.com/watch?v=Zx2zhbZLj9c&t=207s "Stable Diffusion 2.0 for Automatic 1111 - INSTALL GUIDE - Super Easy!"

[coronet1127]

I understand
Because the extension filename was hidden.
768-v-ema.yaml.yaml

Check the extension display from the Explorer file view.

[enn-nafnlaus]

Hooray for Windows' terrible design decisions... :Þ