Training on multiple concepts

[muhammad-ahmed-ghani]

Hi @TheLastBen , I have a question related to optimal settings for training multiple concept means (on couple). I tried with 20 male and 20 female images and trained it on (text-2800, unet-4000) steps. I got bad results. Learning rate was 2e-6. Can you please help me with training on couple images ?
I also test with 10+10 images but results are pretty much the same.

Model: SD v1.5

Tried Settings:

• text-350, unet-4000 (learning rate 5e-6)
• text-900, unet-4000 (learning rate 2e-6)
• text-1400, unet-2000 (learning rate 2e-6)
• text-2800, unet-4000 (learning rate 2e-6)

[cdefghijkl]

What exactly the bad results is it? Not looking same as the instance images or kinda similar but skewed/distorted?

[TheLastBen]

40 images for one person is too much, keep it below 15 and the unet lr below 1e-5 for the v1.5 and 2e-5 for the v2

[cdefghijkl]

I know its too much but it works for me, I will try train with lesser image later.
The only thing I have a hard time to apply is how to not overfit and can apply many art style to the trained person, when I prompt "Painting of [filename] blablabla..." Its not showing painting but still a photo realistic.

[TheLastBen]

to solve that, limit the text encoder steps to maximum 400 steps, the unet steps won't affect styling

[cdefghijkl]

400 for one person?

[TheLastBen]

text encoder : 400 steps, 1e-6 for 15 images of one subject.