How to Train 2 Subject People with the New Method?

[MasterDenis]

Hello there wonderful community!
So lately I was testing the new method and was thinking of adding two named people into the training for parallel training in order to have them presented in the same composition later for the best results, for now I work with a colab notebook so I would like to ask which is the best way to go about it with the lowest ram available for colab,

I was wondering if by naming the subjects differently on their photos that would help or if there should be another approach.
i.e. Jimmy (1), Jimmy (2)... and Marie (1), Marie (2)...

[TheLastBen]

Simply by naming the pictures with unique identifiers will let you train on multiple subjects, keep the number of instance images as low as 15 per person.

And most importantly, you shouldn't use regular names for the instances, instead of jimmy, use a random short password-like name like : "jjjmmkttvbv" and make sure it doesn't contain known abbreviations. Otherwise the training won't be as efficient.

[giteeeeee]

Higher value puts more weight on the instance, meaning overfitting and harder to stylize

but why recommending 10%? If its effect is additive, isn't 10% almost like turning the encoder off?

[TheLastBen]

the text_encoder replaces the instance prompt, once it determines what type of subject you're training, it would be enough, if you keep training the text_encoder after that, it will make the instance harder to stylize, 10% is the minimum, you can go up to 50% and be able to stylize

[giteeeeee]

the text_encoder replaces the instance prompt, once it determines what type of subject you're training, it would be enough, if you keep training the text_encoder after that, it will make the instance harder to stylize, 10% is the minimum, you can go up to 50% and be able to stylize

Interesting. So the use of instance prompt + text encoder together, resulted in overfitting with the old method...
Thanks for the explanation, always great help

[MasterDenis]

the text_encoder replaces the instance prompt, once it determines what type of subject you're training, it would be enough, if you keep training the text_encoder after that, it will make the instance harder to stylize, 10% is the minimum, you can go up to 50% and be able to stylize

can you make on the end of the notebook a section where all those terms are explained in detail for best use? That would help to clear some confusion, for instance I did not know what the text encoder does prior to your explanation, if it is written down in the end of the notebook people can ready it without adding too much clutter on the executable code.

[TheLastBen]

Check the latest update, I explained the effect of the text_encoder on the training : https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast-DreamBooth.ipynb

[EricBintner]

This is great info. I also got a little confused on the new version. I don’t see a place to define a class. Last time I trained a model I specifically defined “dog” for the class, but now I’m not sure how to do that. I ran it again yesterday and got a successful set, but I never defined a class (in this case “person”) and SD runs fine without defining the class in the prompt.