Face restore dilemma: blurry or looks like the training person

[magickaito]

Hi I trained 30 images of myself with 6000 steps. When I generate images, I noticed:

1. If face restore is off, the output looks more like myself, but the face is quite always blurry and out of focus
2. If face restore is on, the output is sharp, but it looks less like myself

Anyone has similar experience?

[cerega66]

What did you train on? What is the photo quality? Are there extra items in the photos? Have you tried changing the face restoration method? What model did you train on?

[magickaito]

Hi I am still learning so I try my best to answer your questions:

1. I train my models using this Joepenna's dreambooth notebook
2. I use the pre-generated regularization images without changing, "person_ddim"
3. I tried one more night to re-train with higher quality images, but the outcome it about the same, I am forced to turn off the face restore and accept generated images that are less sharp
4. I used 30 training images, maybe 1 or 2 have extra objects but generally most images are clean.
5. I tried changing between CodeFormer and GFPGAN, I still find that turning off completely looks closest to my original face.
6. I modified the script on the notebook to train on SD 1.5 instead of 1.4

[CCpt5]

There's a discord channel for Dreambooth w/ lots of discussions specific to Joepenna's repo. Have you visited there? Honestly sounds like you just need to train a better model. 30 images is quite a lot and it really seems like "less is more" - you can start to confuse the training w too many images.

Mid-range faces always seem to be a bit problematic, but there's lots of discussion (and people w/ experience) on discord about what might work best: https://discord.com/channels/1023277529424986162/1023293330601287711

Personally I wouldn't use the face restore for something where likeness is important.......I'd use only it to keep random faces from looking distrored/smushed/etc and destroying an otherwise nice image. But I could be wrong on its intention - I just train lots of models/run lots of prompts vs. using that.

[magickaito]

Thanks! never came into my mind that less could be more. thanks for the tips, will check the discussions to see if there's any clues there.