Awful results since skip text encoder was removed.

[metzo007]

Hi,

since the option to remove the text encoder was removed, I am not able to get any good results.

Today I did a training with the min 350 Text encoder training steps and the model and I got no dreambooth at all. The results are like using the 1.5 model only,

Yesterday I did the same with 600 steps and the person fron the trainen pictures got 3 arms and the posture was all over the place.

I also did it with 100% UNet and it ignore all my prompts completely
but with 100% Text encoder it always a mix of the training images

I used the same dataset and training steps that I used before the change, and while the old model is very good, all the models I trained after the change are awful.
I just try to get the same results as before, but sadly the old versions don't work anymore.
Anyone can help?

[metzo007]

One more thing to add. We had an option to skip the text encode before and it worked fine, but if we now skip the text encoder by set it to zero, SD will ignore the prompts. So there was different method before. What was it?

[TheLastBen]

Even previously when you don't train the text encoder you will get vanilla results

[metzo007]

vanilla results = stock 1.5 model and not responsive to prompts? Is that what you are saying?

I alway skipped the text encoder and got a model that was very responsive to the prompt. Also, if i remebber correctly, there was a time when the text encoder was not even an option, and I got good results. This it what confuses me the most.

[TheLastBen]

you can't skip the text encoder and still get results, unless you caption the instance images

[metzo007]

caption the instance images = phtmejhn (1).jpg, phtmejhn (2).png.....

Yes, I did name them like this and skipped the text encoder. I am now confused on what the text encoder does? Is the encoder getting description for the images and train with the description.
If this is true, caption the instance images and using Unet training only should get the resulds I want. I will try later.

[nawnie]

A caption is giving the image a sentence/prompt instead of a single word name.

Phtmejhn is not your caption, that's your instance/subject word. "a photo of phtmejhn going to the store" is a caption. Now with no caption the text encoder would see the image and read it as, "phtmejhn-a person walking outside towards a building"

Do you see why both are not needed, at least in the context of this dreambooth.

[nawnie]

Hey ben, iirc there was a point where you had the text encoder train for a few steps even when at zero back when we were using percents a few months ago. This may be why this user is having the issues now.

…

On Sun, Dec 18, 2022, 6:59 AM metzo007 ***@***.***> wrote: caption the instance images = phtmejhn (1).jpg, phtmejhn (2).png..... Yes, I did name them like this and skipped the text encoder. I am now confused on what the text encoder does? Is the encoder getting description for the images and train with the description. If this is true, caption the instance images and using Unet training only should get the resulds I want. I will try later. — Reply to this email directly, view it on GitHub <#1066 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZPYLSBLTRNTXBGK5K2SWEDWN4DCXANCNFSM6AAAAAATCERHMY> . You are receiving this because you were mentioned.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/1066/comments/4441722@ github.com>

[TheLastBen]

Probably, but back then, when text_enc gets disabled, the training would run without the text_enc flag

[leonnn1]

I am getting some great results that I couldn't reach before, I am liking this model, it's just more complicated to get the grip of I think