How do fine-tune the captions for each image trained?

[wktra]

When I upload my images to train, I think this auto generates what is happening the picture, right? How do I check and fine tune what each image is labelled before I train? @TheLastBen

[TheLastBen]

you can manually caption each image, by adding words to the name, for example if the instance name is "ejhruhfee", you can caption each image like this : ejhruhfee_word1_word2_word3_wordn.jpg, the prompt for this image would become ; ejhruhfee word1 word2 word3 wordn,
but remember, if you add captions to the images, the model could become rapidly over-trained and non-retrainable.

[nawnie]

d8ahazard/sd_dreambooth_extension#72

If this indeed how it works thats why_doing_this causes blending cause those are clases whie before the _ is the instance prompt, people have specifically mentioned this colab in the comments ill do a clone wars style 😉 to check it out in the next few days (cfgi) and to confirm it ill use sd 1.5 instead of Nov or waifu

[nawnie]

Looking at train dreambooth.py a _ is seen a s a " "

[nawnie]

Looking at train dreambooth.py a _ is seen a s a " "

However it's being displayed as a no space for me, that may be due to a batch size of 8

[TheLastBen]

_ is being replaced by a space, - is removed and a digit is removed and () are removed

[nawnie]

using very long prompts as the file name seems to yield great results, on a set of 6 people, 4 poses and 3 images per pose, about 200 images in total for 4k steps 40% text encoder im planing for many more steps but was intending to only have a few poses at first and as i lower the rate for my person add in more of the poses... but i may only need 3, think the Key here is to NEVER use the exact same prompt for two images, and be very detailed, also using !subjectname the one person i put the class id before the subject is very far behind in training however, so im wandering since set prior reservation rate is still set to 1 is that left over from the contains faces or is that actually referring to the instance prompt we write and not a "Class" identifier. tried putting images into folders... Fast DB said NO lol. this seems to be working well.

example being A close up photo of the woman !ZZX with glasses on, smiling(11)

this worked really well except smiling and glasses help call her better than woman.. but i should of known that starting with joes dreambooth,

PLEASE DO NOT FEEL THE NEED TO ANSWER THIS ANYTIME SOON do the V2 thing :D

[nawnie]

tokenizer shows 2 tokens for !+name hoping that helps the call once further training is done, the pose images are labeled to even more detail. in pose ive gone ahead and used !poses in !armsbehindhead style as an ending, in hopes that with enough examples i can run !poses as a wild card having it on multiple, though most likley it'll blend or give me a portrait style photo filter

[nawnie]

so reporting back, after about 10k steps, all subjects began to blend... said what ever was afraid using ! would do that.. just woke up and oaded it up after running another 6 hours and i have some of the most stylized models i have made yet, i chose 5 subjects that have.. unique features, white Cherokee mix, history of drugs, big nose... etc and there was 0 blending. 1 person 'my uncle ive mentioned earlier, is my control,,, and well i through out his personal model this one works so much better.

there was 1 issue, real name was Whitney I've used her real name before and its worked, but with !whitney as the term i cant not get whitney Houston with !Whitney's hair.. its hilarious. ive got a bleach anime model on git hub im going to use the same images and label them, before they where labeled character names style name or ... ability name.

[nawnie]

• when using a space it gets read back during training without a space but when generating images it does indeed separate the terms. so again i think the key is not to tag but to write a prompt you would use to generate the image your labeling

[nawnie]

• last note on training i started with a batch of 8 but backed down to a batch of 1 overnight

[nawnie]

two more test ran, spaces are the issue, when using spaces you get no spaces when it shows the name under training in dreambooth after replacing spaces with _ any name already known by SD or not works. im not sure where the space gets lost but it definitely gives better results this way

spaces: acloseupphotoofwhitney
underscore: a close up of whitney

[nawnie]

results came out to not being a Whitney Huston with blonde hair and blue eyes but a lil blonde nerdy woman :P

[nawnie]

They blended eventually I had huge progress very early but before hitting perfection it blew up. Using both the text encoder and captions probably caused the early issue. And ya it was super quick to happen thats why I was surprised. Their where multiple instances where pictures contained more then one subject which could if caused both blending and early similarities

…

On Mon, Nov 28, 2022, 12:50 AM luci9t ***@***.***> wrote: @nawnie <https://github.com/nawnie> How do you add captions and not have it overfit? You mentioned you used 200 images and 4k steps that seems so underbaked when I think about what the recommendation is in the notebook ie. 200 steps per image, is there something here that I'm missing? Is it more dynamic than just that rule? — Reply to this email directly, view it on GitHub <#427 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZPYLSGEHM5WNRP5Z26RHMDWKRIZ7ANCNFSM6AAAAAARZQE5TY> . You are receiving this because you were mentioned.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/427/comments/4250634@ github.com>

[nawnie]

Also I'm playing with the learning rate, using consistent instead of polynomial. So it stays at 2e-6

…

On Mon, Nov 28, 2022, 4:22 PM shawn ohagan ***@***.***> wrote: They blended eventually I had huge progress very early but before hitting perfection it blew up. Using both the text encoder and captions probably caused the early issue. And ya it was super quick to happen thats why I was surprised. Their where multiple instances where pictures contained more then one subject which could if caused both blending and early similarities On Mon, Nov 28, 2022, 12:50 AM luci9t ***@***.***> wrote: > @nawnie <https://github.com/nawnie> How do you add captions and not have > it overfit? > You mentioned you used 200 images and 4k steps that seems so underbaked > when I think about what the recommendation is in the notebook ie. 200 steps > per image, is there something here that I'm missing? Is it more dynamic > than just that rule? > > — > Reply to this email directly, view it on GitHub > <#427 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AZPYLSGEHM5WNRP5Z26RHMDWKRIZ7ANCNFSM6AAAAAARZQE5TY> > . > You are receiving this because you were mentioned.Message ID: > <TheLastBen/fast-stable-diffusion/repo-discussions/427/comments/4250634@ > github.com> >