a few questions about the text encoder

[nawnie]

biggest one, does the encoder save to the .ckpt OR is it saved in the session info, if i resume training from an old ckpt but as a new session am i missing out on previous text encoder training?

#278 (reply in thread)
Here you mention the text encoder replacing the the instance prompt, does that mean if the encoder is off we can use the file names as instance prompts.

i mentioned the other day i was working on a guide and i came up with some interesting results,
Using Full prompts as file names gave me images of all my subjects SUPER quick but then the inevitable happened. as soon a they really looked familiar they started to blend.

i also meant to ask a question before about ziping images after we place them in the upload folder. is this an important function to dream booth or can i just edit the two cells to remove the zipping.

Currently im playing around with the repo
https://colab.research.google.com/github/victorchall/EveryDream-trainer/blob/main/Train-Runpod.ipynb?authuser=2#scrollTo=503322f5

i found it branched off of one i had mentioned earlier that used the filename as the subject. Is there any way to implement this into yours, maybe a toggle use text encoder or provide your own captions?

[TheLastBen]

The textenc is saved into the ckpt, so if you load a ckpt, the textenc will come with it, no need to retrain it.

finetunning the text encoder will use clip to get the textual embeddings, no need for a manual instance prompt which is generally short and causes overfitting.

The inst images are zipped in case you load a session from different gdrive because gdown can't download more than 50 files per folder.

You can caption the images by separating the words with an underscore _ use sentences instead of keywords.

[nawnie]

thats awesome. so does it add text to the image name so each time it reads that image it gets retains the information? i know you have said repeatedly to stop at 1k steps, but what if there is more then 1k images in the set, ive realized generally dreambooth isnt well suited for this much but honestly yours handles it really well. im specificly trying to get one with well over 200k images to work and well thats probably on me,

[TheLastBen]

if it's that many instance images, you scale the total number of textenc steps accordingly, 20 pics is 1000 steps, 2000 pics is around 5000 steps, 200k pics is around 20k steps

[nawnie]

Ok, defenitly have had some issues, with the encoder being super high, my current trial had it sent for 100% for 1 epoch. Now I keep it 1% each run doing 4 epoch a set, this has yielded some great results so far. Can't believe the a100 is pychibg batches of 14 on here and 16 on another.. oddly theirs uses more ram to start but can do more batches idk.. Ty again.

…

On Sun, Nov 27, 2022, 12:30 PM Ben ***@***.***> wrote: if it's that many instance images, you scale the total number of textenc steps accordingly, 20 pics is 1000 steps, 2000 pics is around 5000 steps, 200k pics is around 20k steps — Reply to this email directly, view it on GitHub <#660 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZPYLSDM6HAXY27MJFAVMHDWKOSDRANCNFSM6AAAAAASMK4HBE> . You are receiving this because you authored the thread.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/660/comments/4247851@ github.com>

[nawnie]

So, the text encoder coder bleeding may result from it labeling everything in the text files man or woman, it may be worth while to use the faces option or create a woman man labeled files to regulate when using multiple subjects to reduce this blend. Or to add code to refer to the gender as the file name.. that would make a massive difference I believe.

I'm going to try this out later, but make sure to get a wide variety of people. Working on an animation pack tonight so may be a day or two before I try this,