Replies: 1 comment
-
If you're confused about the settings, always start with minimal training, set the training steps to 2000, text_enc 350, skip the concept and use only 10 instance images. The concept images help de-overfitt the text_encoder in case it gets overtrained, so start small, then resume the training slowly while testing the model. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
First of all: thanks for the great work Ben. It allowed me to thinker with art without having a good pc or the time to practice. Modern day hero stuff.
I have some doubts regarding the training of styles, i believe answering them would help many other who will be using SD as a helping tool in their job as concept artists and hobbyst:
When training a style, is it better to have a high number of instance images(around 30-50) and high number of concept images? As i understand, you train on the instance images to get the general style; you then train on the concept images to let the model know how different things, not present in the instance images, should be represented by the artstyle you are training. Is this assumption correct? I've finetuned a model using anything v3 as base since the style is anime-esque as a proof of concept on 39 images, 3900 steps,550 tex encoder,100 concept text encoder and 19 concept images. Now, the results i consider them to be good enough because it understood the style and it can recreate it. Unfortunately i trained it on the same kind of images (fullbody drawings on white backgrounds) so it's kinda shitty when drawing faces since in 512x512 you really cant see them;since i havent used enough concept images i suppose that to be the reason it only draws the same kind of attire (the images where for a fantasy game, so armor, military suits, etc...). Are my assumptions correct? If i lowered the number of instance images and added more concept images, would it still reproduce the style as the model with 39 images while being more varied in what it can do? Should i add faces in the instance images too or just add them to the concept ones?
Another question now, more focused on the concept images and embeddings/hypernetwork. Lets suppose i have a few drawings made by the same artist as above in the composition as a reference/character sheet [NSFW link? it's danbooru so...]. If i wanted to be sure to have the model be able to make similiar compositions, where should i add these kind of images? Instance?Concept?Both? Can i train an embedding on them (around 20 images) so i'm sure it can make them? If i train an embedding this way, should the images be present in the instance, concept or in both or in none?
Lets say i now want the model to be able to draw chibi and i have a few images made by the same artist, do i proceed as above or is it better to make a new model?
And now the million dollar question, what are the best values for the encoders in your opinion when finetuning a model that isn't the base ones from stabilityAI? As i said, with the values i used above it really catched the style, is it because Av3 was similiar tot the style? Or should we just yolo and experiment?
Thanks in advance to whoever finds the time to answer me, I tried looking around and i found the threads opened before to have very sparse info and sometimes outdated, hopefully we'll have a mini bible regarding style training in this thread.
Beta Was this translation helpful? Give feedback.
All reactions