Does having too many concept training steps negatively affect the results?

[ArsenalAlex108]

Hello, I am current training SD with an art style.

At first, I trained SD 1.5 with 20 (512x512) instance images and 103 (512x512) concept images, with 4000 training steps, 600 text encoder training steps and 600 text encoder concept training steps. While the resulting model still have some limitations, I consider it to be satisfactory.

In the second attempt, I trained SD 2.1-512px with the same parameters except that I used 154 concept images and increased the text encoder concept training steps to 1000. The resulting model, however, generate images that were worse than the previous model at best or looks nothing like the prompt at worst.

Images were generated using Euler a.
Prompts: a photo of a singular 2D SVG chess piece icon, (((a white medieval Catholic chancellor's helmet))), in black-and-white, white empty background, no reflections, with solid thin black outline:0.1, sharp, 35mm, simple, undetailed, concept art, digital art, clean, vector
Negative Prompts: extreme detail, weather, background, color

First Model:

Second Model:

I am unsure what the root of the problem is, whether it is because I used to many encoder concept training steps, or it is because of SD 2.

Thank you for spending time to respond to this discussion.

[TheLastBen]

Hi,
for a simple style, avoid too many steps and avoid concept training, start with 350 text_enc and 1500 steps, then slowly add more after testing the model.

[ArsenalAlex108]

Thank you, I was suspecting overfitting. One more thing, do I have to remember the unique identifier that I used or add it to the prompt?

[TheLastBen]

you need to use that identifier to call the trained style