Add base settings' target_resolution to the image cache key.

[maedtb]

This makes changes to a settings target_resolution field force the use of different cache keys for image latents. This fixes changing the base settings (not the per-concept one) target resolution field not regenerating cache when latent caching is enabled, effectively ignoring changes to the resolution setting until cache is regenerated for a different reason. Changing the per-concept target resolution override already worked as expected for regenerating the cache as is unaffected by this change.

This change will also change all image cache keys, which will force all users to automatically regenerate their image caches. Previously generated caches will need to be cleaned manually, if the clear_cache_before_training setting is not enabled.

[dxqb]

I didn't realize we had any useful feature that automatically detects if changes were made that require re-caching. I assumed you manually have to delete the cache when you do that. Am I wrong?
Are you sure variations_group_in_name names has this effect and is intended for that? This sounds more like it's involved in generating variations, so variations are not created across all samples but only across groups where these values are the same.
How does resolution fit in there?

[Nerogar]

variations_group_in_name is the correct parameter. Re-caching if augmentations change is currently enabled by passing the concept.image key to this parameter. The cache directory name is a hash of all the values passed in there, which automatically triggers the caching if anything changes.

If I remember correctly, the name was derived from the grouping of training samples into variations. If all the variations_group_in_name values are the same for two samples, they are put into the same variation group.

[maedtb]

I didn't realize we had any useful feature that automatically detects if changes were made that require re-caching. I assumed you manually have to delete the cache when you do that. Am I wrong?

Currently if you change any parameter in a concept's image tab, it will effectively generate a different cache key for that concept (stably so, so if you swap back to the previous parameter set, it will go back to using the previous cache key). Most other base configuration parameters do not affect image cache generation, other than the target resolution (IF, and only IF, the concept's image resolution override parameter isn't enabled and set to a different value).

Are you sure variations_group_in_name names has this effect and is intended for that? This sounds more like it's involved in generating variations, so variations are not created across all samples but only across groups where these values are the same. How does resolution fit in there?

The cache key is related to samples, in that two variations of the same source image will go under the same cache key, and the settings which control the cache key are mostly settings which generate different variations, but the enable-target-resolution-override and target-resolution-override settings are exceptions to this that currently cause a different cache key (and importantly different results) to be generated if set to different values.

Whether this current behavior of changing the cache key based on image parameters should extend to the model's base target-resolution is an open question. I propose that it should, as it is likely what users will want and is what is currently the case for almost every configuration open--change a setting and get a different result, instead of invisibly(!) use the existing cache (if enabled) and get a stale result (if clear-cache is not enabled). I believe currently the only other case where the cache can go stale/bad relies on users modifying/adding/removing the images in the path of a concept. I am not aware of any other setting which would affect the result of an image latent that is not on the list (path, whether to navigate to subdirs, image variation and resolution settings, and the random seed) already, other than the config's base resolution, which this PR adds. Let me know if you are aware of anything else I am missing, though.

[maedtb]

I want to highlight an unfortunate part of this change however, is that we don't actually care about the value of settings.target_resolution in all cases, we actually the computed end result of the three settings in this evaluation: settings.target_resolution if not concept.image.enable_resolution_override else concept.image.resolution_override, but I don't believe there is any trivial way to adjust this code to be smarter about this, without making a larger change [in both MGDS and OneTrainer].

The effect of this is the following table:

target_resolution	enable_override	resolution_override	cache_key
1024	0	512	ABCDEF0...
1024	0	1024	BDF2468...
1024	1	512	CE01357...
1024	1	1024	D01ABCD...

What we actually want is this table instead, where cache is only changed when the values of target_resolution and resolution_override differ AND enable_override is set:

target_resolution	enable_override	resolution_override	cache_key
1024	0	512	ABCDEF0...
1024	0	1024	ABCDEF0...
1024	1	512	CE01357...
1024	1	1024	ABCDEF0...

With that said, this change reduces potential incorrect results, even if it potentially forces cache to be regenerated in cases where it is not necessary.

[dxqb]

Besides the resolution, I think the following settings should invalidate your cache, but don't:

• Text encoder dropout probability
• Clip skip
• Text encoder sequence length
• Train text encoder
• Train embedding
• Changing train dtype
• Aspect ratio bucketing

This list might be incomplete and I haven't checked each one of them, just to illustrate my point:
Your idea to invalidate the cache when you change relevant settings sounds very useful. Doing that only for 1 setting out of many seems incomplete though.

[dxqb]

I'm looking at your feedback, re: all of the things you'd want to regenerate an image/text cache for, and I'm happy to add all of those config values into the cache-key hash if that is what you want. The only inputs in the settings namespace that are currently exposed to MGDS are resolution and frames (as target_resolution and target_frames), but I can add in the entire settings dict in the spot that currently hard-codes res/frames.

Future ref from Maed