New update has poor results

[tpcdaz]

A few days ago I was using this with great results. Back when it wanted 3000 steps and about 20 photos. Since it updated I cannot get a single face looking good. It now claims to only want 650 steps on 10 photos but the results are terrible. Please go back to the previous state as it just worked flawlessly.

[TheLastBen]

there is a dropdown menu to select a lower learning rate, use it.

[nawnie]

i agree here, changing the default from 2e-6 was a dirty trick and a wasted model. 2e-6 isnt even on there as an option, all for progress but lets not just trash what we have built off of. came here to see why the lr is jacked all the way up to 2e-5 model came out ok, (way over cooked but i went for 3k steps, you are right it is wrong to spend 3k steps when the batch size can be put to 4 :D )

[tpcdaz]

3k steps may be wrong but it worked. Like you said 2e-6 is missing entirely so I couldn't go back to what worked even if I wanted to. I could leave it on default with the previous version and know 100% of the time my trained face would be perfect. Now it seems purely by luck or by chance that we get a good result. I've been trying for two hours to get training that even remotely looks like the person I'm trying to train, and still looks nothing like them. Seems like this was changed just for the sake of it. I live by the "if it aint broke dont fix it" approach to life. This for me now seems broken.

[nawnie]

3k steps may be wrong but it worked. Like you said 2e-6 is missing entirely so I couldn't go back to what worked even if I wanted to. I could leave it on default with the previous version and know 100% of the time my trained face would be perfect. Now it seems purely by luck or by chance that we get a good result. I've been trying for two hours to get training that even remotely looks like the person I'm trying to train, and still looks nothing like them. Seems like this was changed just for the sake of it. I live by the "if it aint broke dont fix it" approach to life. This for me now seems broken.

-- hey bud when you click to select rate. there is a pencil image to the right click that and on the menu that appears select allow input, hit ok you can now type your own,

Ben, i am excited to try the new rate now that i notied it, what led to the change, did i miss an ama somewhere?

[tpcdaz]

I had to use the edit and add 2e-6 back to the list, went back to 3000 and 350 like the previous and BETTER version which worked ok but still nowhere near as good as before. Such a shame that this was changed as before it was perfect. Now results are horrendous. Oh well.

[nawnie]

you ca go back to the previous commit, however if results are still not the same do note the text encoder rate has also changed.

[SomeKay]

Hey @tpcdaz, I've also noticed that the new settings are sadly not producing results which were awesome with just the default settings before. I've tried iterating with additional 150 steps a couple of times and all the same with different Text Encoding Steps, but can't get the same quality of results as before.

There is an easy fix for it in the way of using an older version.
I know that at the point of this commit from Dec 20th, the old settings are preserved and you can get to that point in time by adding the commit into the URL: https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/15e82997bbb9a6352c6ad82c42f351f075ca817e/fast-DreamBooth.ipynb - I use this version with the usual practice of adjusting the amount of steps to 200 * number of images.

Still, I would love to use the latest and greatest and get all the awesome fixes and features which @TheLastBen provides. Thank you very much for your hard work!
Does anyone have settings which they managed to use with 10 photos (or more or less) which produced the same results as before?

[TheLastBen]

I only changed the settings after extensive testing, 600-800 unet steps for 10 instance images is more than enough to get good results, just make sure you rename the images correctly.

[SomeKay]

I'm using the same input images from Google Drive as before (meaning that they are already named and haven't changed over time).
I still see my face in the results, but a really good resemblance (like before) happens very rarely and with the old version almost every produced image has great resemblance.

[TheLastBen]

To get a good balance between training time and good results, you need to do some tuning, start with 600 unet at 2e-5, if not enough, resume the training for 200 steps with a slightly lower lr 8e-6 until you get good results, the text encoder should not go over a total of 350 for 10 images during the whole training.

[Omenizer]

Same here, I had 2 sets of 11 images that I trained for 2200 steps, 475 text encoder with great results previously. Took 50min which was ok.

Now it takes 15 min but the results are not usable, and additional training doesn't help unfortunately.

What were the prev settings, and is there a way to configure them? Thanks!

[Quark999]

I think one of the problems is that different people train for different things, faces, whole people, styles, couples. And I think improvements in one area lead to problems in another...

[TheLastBen]

0eb0417

set both learning rates to 2e-6 and you're back to the previous settings.

[Omenizer]

Would you also recommend the previous step count, or will less work too? Never tried less than 200/image

[TheLastBen]

if you use the previous settings, stick to the same previous unet steps.

[SomeKay]

Thank you for this!

I can confirm that using:

• both learning rates: 2e-6
• unet steps: 200 * number of images (I've used 12 diverse images, so 2400 steps)
• text steps: 350

produces great results for training one person. 👍

[TheLastBen]

Nevertheless, try experimenting with 2e-5, I'm sure there is something there, I was able to retrain my previous models at 25% the time and get better styling.

[nawnie]

I made 8 models, off tiktok models, 2 came out identical, 6 came out close. 5 of those 6 where able to then be trained with extra steps, 1 well there's always 1. No one wants to hear it but, warm up steps can help, I suggest adding warm up to total steps. Training on just goku with 10 pictures, no specific poses or transformations, worked well. Even slapped a batch size of 4 on it and was done training in under 8 mins. I still believe your old code for 1.5 training unet and encoder at same time (with encoder at a very low lr) worked the best, may end up forking that and labeling it a legacy edition, cause it doesn't seem like 3.0 will be any more fun to fine tune.

…

On Tue, Dec 27, 2022, 5:33 AM Ben ***@***.***> wrote: Nevertheless, try experimenting with 2e-5, I'm sure there is something there, I was able to retrain my previous models at 25% the time and get better styling. — Reply to this email directly, view it on GitHub <#1162 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZPYLSAZADCKIO6QCWHRRW3WPLHW5ANCNFSM6AAAAAATJ4ITNM> . You are receiving this because you commented.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/1162/comments/4502283@ github.com>

[TheLastBen]

Training a unet on hidden states from a different text encoder would yield inconsistent results, when you freeze a text encoder, any data that comes from the text encoder after that is strange data and the unet is getting trained on that data, it's a wrong type of training, the ideal way is to finetune the text_encoder alone, then use frozen hidden states to train the unet which is the correct approach.

[SomeKay]

I made 8 models, off tiktok models, 2 came out identical, 6 came out close. 5 of those 6 where able to then be trained with extra steps, 1 well there's always 1. No one wants to hear it but, warm up steps can help, I suggest adding warm up to total steps. Training on just goku with 10 pictures, no specific poses or transformations, worked well. Even slapped a batch size of 4 on it and was done training in under 8 mins. I still believe your old code for 1.5 training unet and encoder at same time (with encoder at a very low lr) worked the best, may end up forking that and labeling it a legacy edition, cause it doesn't seem like 3.0 will be any more fun to fine tune.
…
On Tue, Dec 27, 2022, 5:33 AM Ben @.***> wrote: Nevertheless, try experimenting with 2e-5, I'm sure there is something there, I was able to retrain my previous models at 25% the time and get better styling. — Reply to this email directly, view it on GitHub <#1162 (reply in thread)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZPYLSAZADCKIO6QCWHRRW3WPLHW5ANCNFSM6AAAAAATJ4ITNM . You are receiving this because you commented.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/1162/comments/4502283@ github.com>

Out of curiosity, would you mind please pinpointing the commit at which unet and encoder were trained at the same time?
I would love to try it out. :)

[nawnie]

I'm reply to this via email not github or i'd link it, look for the commit before 2.0 was added. 2.0 likley requires too much vram to do both at once, which is why I feel an optimized 1.5 would be great. On Tue, Dec 27, 2022, 1:05 PM Kristian Poslek ***@***.***> wrote:

…

I made 8 models, off tiktok models, 2 came out identical, 6 came out close. 5 of those 6 where able to then be trained with extra steps, 1 well there's always 1. No one wants to hear it but, warm up steps can help, I suggest adding warm up to total steps. Training on just goku with 10 pictures, no specific poses or transformations, worked well. Even slapped a batch size of 4 on it and was done training in under 8 mins. I still believe your old code for 1.5 training unet and encoder at same time (with encoder at a very low lr) worked the best, may end up forking that and labeling it a legacy edition, cause it doesn't seem like 3.0 will be any more fun to fine tune. … <#m_6466903373772942331_> On Tue, Dec 27, 2022, 5:33 AM Ben *@*.***> wrote: Nevertheless, try experimenting with 2e-5, I'm sure there is something there, I was able to retrain my previous models at 25% the time and get better styling. — Reply to this email directly, view it on GitHub <#1162 (reply in thread) <#1162 (reply in thread)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZPYLSAZADCKIO6QCWHRRW3WPLHW5ANCNFSM6AAAAAATJ4ITNM . You are receiving this because you commented.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/1162/comments/4502283@ github.com> Out of curiosity, would you mind please pinpointing the commit at which unet and encoder were trained at the same time? I would love to try it out. :) — Reply to this email directly, view it on GitHub <#1162 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZPYLSFB2EXES7NNIEZRVNTWPM4WJANCNFSM6AAAAAATJ4ITNM> . You are receiving this because you commented.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/1162/comments/4504242@ github.com>