Can I train the sd 1.5 model with a mix of 512x512 and 756x756 images?

[gilroff]

is training on 756x756 even possible on sd 1.5?

[TheLastBen]

yes it is possible to train the v1.5 on 768px images, but at inference time, you can not go below 768px or you will get bad results.
here is an example : https://huggingface.co/TheLastBen/hrrzg-style-768px

[ProgGrog]

ah, that makes sense.

I've been training in multiple actors, so I'd thought to try different image sizes for foreground/background, to give it more options for layout. But it sounds like that isn't the right method. Thanks so much!

[TheLastBen]

if you're training on known actors, there is a shortcut for that, when you train on them, use the exact name as the instance name, for example for Brad Pitt, you would use brad_pitt (1).jpg ...etc, and entirely skip the text encoder start with a high (default) learning rate for 650 steps, then reduce it to 6e-8 and resume the training for 300 steps at a time.

[ProgGrog]

Alright, I'll try it. I was studying your variable training advice in other threads earlier. By multiple actors, I was training say, (male ranger) (on_horseback) (wearing_leather armor) (next to female wizard), (dragon in background); or three teenage {boy|girl} (tattered clothes)(holding grappling hooks)(swinging between skyscrapers)(ruins of a city in the background).
So more wide shots instead of portraits, and trying to train actions as concepts. I got one 1.5 model working well in November (via luck) with almost no encoder, then fudged it.
(also, gotcha a coffee, bud. Can't understate how great this is for you to make this accessible.)

[TheLastBen]

Thanks for the coffee, try experimenting with the V2 versions, they are surprisingly good, the high lr works best with them.
The previous settings for the V1.5 are 3000 steps unet and 350 text encoder and 2e-6 lr for both.

[ProgGrog]

Hey, just wanted to say you were right :D

You were right about v21 liking the faster training rate. 1e-5.

Also, I tried the mixed 512 and 768 on a v21 768 model and you were right, it wasn't good. It did have the mix of sizes that I had hoped for, but there was no reasoning behind the sizes (ie there's no depth inference, which is an obvious result after you try it).

I got good results from v21 512, for quite the bank of images on 10 concepts. Though I've only trained it at high rate so far. So far,
350 steps of Text enc at 1e-6, then 650 steps Unet at 1e-5, repeating Unet at 1e-5 until it got close, then I was anticipating slower speeds, but they haven't been super necessary yet.

I'm returning to v21 768, but I was going to mention to you that adding a "keep the file name the same" for the folder to folder batch upscaler would be a godsend for migrating from 512 to 768.

Thanks for the improvements you keep making!