New DreamBooth update is giving complete different results using same U_NET 2e-6 values as before

[meunumerotim333]

Please, I'm kind of desperate to have this solved, I used DreamBooth everyday to create images for my instagram, and today after I train Mo-Di model, it's giving me completely different results. I'm all day testing different methods but I'm frustrated.

I always trainining using u_net 2e-6 since last year, always used the value of 1000 using 10 pictures on 768 pixels using 1.5 model

The results was always like the person I was training was imported to the model style.. and not sharp and boring results like too much realistic.

I'm uploading an exemple on a image using the same pictures as I trained successfully before, even on the pictures I used the seed, the tests without the seed was terrible, very sharp and very realistic.. and isn't the intention.

I don't know how to upload image here, so I've uploaded on imgur.
image link:
https://i.imgur.com/YfSnopC.png

I also tried to train using 650 steps instead of 1000 but still is very sharp and isn't blending to the model style like I'm used too.

Please, I just would like to train the model again like I'm used to. I'm kind of desperate because without that my instagram will be probably dead. I don't know how to solve this without help.

detailed configuration:
model: 1.5 (modi external model)
u_net: 2e-6
text encoder: 350
steps: 1000
pictures: 10 768x

[TheLastBen]

Did you use the exact same settings and base model in both results in the image you uploaded ?

[meunumerotim333]

Looks like 4 days ago someone else complained about the same thing

That would be me I think. I'm still having the same problems. Today I spent hours on 10 images starting very low in training steps and then resuming training and increasing. I based the training on both default SD 1.5 and Analog Diffusion which I have had great results with before as a basis. Saving a couple of renders, I then resumed training and rolled the same images again based on PNG info from those saved images to give me the exact same image with the resume-trained model.

At too low training steps there is hardly any likeness. It looks photorealistic but it's a different person. Increasing a bit gives better and better likeness but before it ever reaches a great likeness, the face becomes fake looking. When a very good likeness is reached, there are distortions to the face. I also did a few attempts at training an object with the same results really. Eventually reaching a good shape understanding only when image is distorted.

Using 2e-6 as before for all of these tests and I realize that this is the default from when I had better results before. Perhaps I will do another test with 40+ images later, as I used to use a lot more images than I typically do nowadays.

I also want to express my gratitude to TheLastBen for all of this! I complain but all of his work has given me more happiness all in all, than frustration!

(Edited)

Have you tried this colab reverted by a member of community? Is an old colab notebook that work as before this problem start:
https://colab.research.google.com/github/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb

for me is working very well as I was used to

[THX1139b]

Have you tried this colab reverted by a member of community? Is an old colab notebook that work as before this problem start: https://colab.research.google.com/github/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb

for me is working very well as I was used to

Well, I finally tried this alternative notebook, speending a full workday on testing it. I tried a set of 28 very good quality photos of myself both at 512x512 and also at 768x768. 2e-6 with captions. The results look like Odo from Star Trek DS9, if that means anything. I resumed training and ended up training one of the batches for a full 2000 steps more but there was no discernible difference whatsoever from the first training run. It did look like me however, just more like a plastic prosthetic version of my face from a bad movie than a real human.

[meunumerotim333]

Have you tried this colab reverted by a member of community? Is an old colab notebook that work as before this problem start: https://colab.research.google.com/github/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb
for me is working very well as I was used to

Well, I finally tried this alternative notebook, speending a full workday on testing it. I tried a set of 28 very good quality photos of myself both at 512x512 and also at 768x768. 2e-6 with captions. The results look like Odo from Star Trek DS9, if that means anything. I resumed training and ended up training one of the batches for a full 2000 steps more but there was no discernible difference whatsoever from the first training run. It did look like me however, just more like a plastic prosthetic version of my face from a bad movie than a real human.

But you were used to use a configuration before? Because for me it's working just like I was used to. Maybe you need to use another U_Net Learning value to be more realistic. My style of training is to fit the style of the trained model style.. but isn't photorealistic. I think every model will need to test different values of training. The Text encoder is also important.

[THX1139b]

But you were used to use a configuration before? Because for me it's working just like I was used to. Maybe you need to use another U_Net Learning value to be more realistic. My style of training is to fit the style of the trained model style.. but isn't photorealistic. I think every model will need to test different values of training. The Text encoder is also important.

I’ve frankly lost track of exactly how everything was before. Mostly I used the default values with minor tweaks. I believe my best results were with a learning rate around 80 x or 100 x of the number of photos. I usually used 350 steps for the text encoder. And I used large image sets for training. Then I started mainly working with black and white photos of historical figures, which limited my choices to maximum 20 images that I could hunt down where the person had similar age, facial hair style and things like that. Also great results on most of my first models.

I tend to test on photorealism because if you can make a photorealistic believable character then that face will also work great for illustrations and whatnot. I’ve been making a lot of vintage poster art type images in the style of early 20th century advertising.

When I started getting good results with Dreambooth there was a new feature just added that let you use class images of faces of men, women, whatever. I used a set of 200 high quality photos that I put together but then that option quickly disappeared. Also there were no captions back then and a few other settings here and there that have changed. So my problem now may not be about u_net learning steps at all.

[meunumerotim333]

But you were used to use a configuration before? Because for me it's working just like I was used to. Maybe you need to use another U_Net Learning value to be more realistic. My style of training is to fit the style of the trained model style.. but isn't photorealistic. I think every model will need to test different values of training. The Text encoder is also important.

I’ve frankly lost track of exactly how everything was before. Mostly I used the default values with minor tweaks. I believe my best results were with a learning rate around 80 x or 100 x of the number of photos. I usually used 350 steps for the text encoder. And I used large image sets for training. Then I started mainly working with black and white photos of historical figures, which limited my choices to maximum 20 images that I could hunt down where the person had similar age, facial hair style and things like that. Also great results on most of my first models.

I tend to test on photorealism because if you can make a photorealistic believable character then that face will also work great for illustrations and whatnot. I’ve been making a lot of vintage poster art type images in the style of early 20th century advertising.

When I started getting good results with Dreambooth there was a new feature just added that let you use class images of faces of men, women, whatever. I used a set of 200 high quality photos that I put together but then that option quickly disappeared. Also there were no captions back then and a few other settings here and there that have changed. So my problem now may not be about u_net learning steps at all.

You can right click on the colab copy and revert old versions. Try to select an older one and see if revovers the one you were used to use.

[Omenizer]

Same here, can't get anything except the input images weirdly deformed?

[Bullseye-StableDiffusion]

Same here, can't get anything except the input images weirdly deformed?

Yes, because the latest update broke something.
The last known working commit by me is "e59c0fc - fix RAM problem for good, thanks to @Daviljoe193". Maybe Ben can revert anything to that commit to solve the problem. I tried for many times and yea, something from the last 2 days broke it. Same settings, same model.

[Bullseye-StableDiffusion]

Maybe it has to do something with that?

100%

[meunumerotim333]

how to upload images here? didn't find the option.

Oh, I've also trained a model now because I've seen the Ben did an update. But is still giving very bad results

[Omenizer]

Tried it today, no change. Empty prompts are giving me the instance images, after very short training.

[TheLastBen]

I'm working on a solution

[Omenizer]

Thanks! 🙏🏼🙏🏼🙏🏼

[Bullseye-StableDiffusion]

Tried it today, no change. Empty prompts are giving me the instance images, after very short training.

Try this notebook: https://github.com/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb
It seems that I fixed the problem by reverting everything as it was 2-3 days ago. Tested and working 100%.

[TheLastBen]

Fixed with the new default values, 100 steps textenc and 500 UNet

[Omenizer]

If you get over fitting, reduce the learning rate to 1e-6, and make sure when you resume the training to set the textenc to 0. Also use no more than 10 high quality pictures.

Hmm I'm in a weird place now, celebs and "person" look partly like my subject, but the subject doesn't look fully like itself. So both over- and underfitted at the same time 🥳

What are the respective effects of unet and text enoder training - can anyone explain?

[Omenizer]

I was already at 1e-6 for the text encoder

[Omenizer]

I was already at 1e-6 for the text encoder

Just try this collab from @Bullseye-StableDiffusion https://colab.research.google.com/github/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb and with settings you usually used

dude thanks but you already told me twice

[meunumerotim333]

I was already at 1e-6 for the text encoder

Just try this collab from @Bullseye-StableDiffusion https://colab.research.google.com/github/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb and with settings you usually used

thaaank you! My hero! Old method is just way better. Really important to still have this option. Many thanks

[Bullseye-StableDiffusion]

I was already at 1e-6 for the text encoder

Just try this collab from @Bullseye-StableDiffusion https://colab.research.google.com/github/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb and with settings you usually used

thaaank you! My hero! Old method is just way better. Really important to still have this option. Many thanks

Thanks for confirming that it works! And even better!

[Omenizer]

Anyone know that the Text Encoder influences, and what the effects of the Unet are? Or, what do I reduce/increase when?

[TheLastBen]

Train the text encoder only once for 80-100 steps, then in case you resume the training, set it to 0.

[Omenizer]

I'm at 35 steps now... What does the text encoder do exactly?

[TheLastBen]

it adds the new instance to the vocabulary (sort of)

[Omenizer]

Ok, I'm giving up for now. No idea what happened to the notebook - it was easy to get good results in 10 to 20 minutes, now I can't get anything usable at all :(

[meunumerotim333]

use the old colab notebook that our friend shared here above

this one https://colab.research.google.com/github/Bullseye-StableDiffusion/fixed/blob/main/fast_DreamBooth_fixed.ipynb