Why is no one talking about --sub-quad-q-chunk-size --sub-quad-kv-chunk-size and --sub-quad-chunk-threshold optimisations

[Justanetizen]

I'm running on windows with a radeon RX 6600 and tried adding --sub-quad-q-chunk-size 512 --sub-quad-kv-chunk-size 512 --sub-quad-chunk-threshold 80 to webui-user as I saw 1 person using deep in another thread.

Don't know what these settings are really doing but I can now gen up to 5 images per batch with a very large speed increase without running out of memory and also getting a speed increase on low batch sizes. Previously could only do 1 or 2 images per batch.

Can also gen much higher resolutions, previous highest was around 640 x 640 now largest I've tried is 896 x 896 and no memory error.

There also appears to be no drop in quality so what's the downside and are there better paramaters to use for these settings?

[cibernicola]

In general, little is said about optimizations or parameters to optimize yields in a scattered and unorganized manner. I don't know if it's because everybody has it very clear or because of the opposite :S

[vladmandic]

majority of users are running with nvidia hardware and in that case, xformers is by far the best cross-attention optimization method (after that far second is fallback to doggettx), so other methods did not receive a lot of benchmarking.

and params you've mention apply when using sub-quadratic cross-attention which may be best when runing on amd gpu hardware.
i think a separate discussion on what are best params for amd gpu would be welcome by amd users.

[cibernicola]

I'm an Nvidia user and I'm still seeing a bit of a mess when it comes to configurations. I've been watching, observing and testing since the beginning, as well as commenting and trying to contribute and no, there is nothing that is orderly and clear and if there is, I haven't seen it :/

[elen07zz]

i agree, im an amd user, i dont have idea what's the best option here.

[yamfun]

I have a rx6600 too, and tried a bunch of different combinations those params from many different posts and still I always get 1it/s on Windows. can you share your lines in webui-user.bat please?

also what does it say inside System Info tab? mine are just
Torch
2.0.0+cpu full nohalf (I use --precision full --no-half)
GPU
(blank)
Memory optimization
medvram
Cross-attention
(blank)

Thanks