List of things that make SD go faster but not yet in Forge

[wywywywy]

I guess some of you may have read this blog on SD optimisation https://www.felixsanz.dev/articles/ultimate-guide-to-optimizing-stable-diffusion-xl It lists out quite a lot of things that can speed up inference by quite a bit, but many of these are not in Forge yet.

I'm listing a few interesting ones out below.

1. Disabling CFG

You know disabling CFG completely saves a huge amount of time, though it's not exactly useful. But do you know that you can disable CFG after ~75% of the sampling steps are done without losing quality?

I don't see an extension or a built-in way to do this in Forge / A1111, but I'm guessing this shouldn't be too hard to implement.

2. DeepCache

A huge increase in performance with small drop in quality. There's an extension for A1111 but it doesn't work with ControlNet nor Forge https://github.com/aria1th/sd-webui-deepcache-standalone And also the upstream DeepCache package has not been updated since Dec 23.

3. OneDiff

A much more flexible & fast torch.compile alternative that works with Loras etc. Good performance gains with no loss in quality. There's an extension for A1111 but it doesn't work with ControlNet nor Forge https://github.com/siliconflow/onediff

[minsky91]

All great ideas (and great optimization blog)! Sadly, Forge's leading programmer seems to have abandoned the project. How about giving some quantification of projected speed gains? Like, publishing a table with generation runtimes of a couple of test images side by side, Forge's times against Automatic's with these extensions activated. That just might help to turn back lllyasviel's attention.

[wywywywy]

Like, publishing a table with generation runtimes of a couple of test images side by side

The author did an excellent job in the blog with lots of graphs, tables, and image comparisons. It answers all the questions and is better than me repeating everything here.

But to summarise, the author used OneDiff + Disabling CFG @ 75% + TinyVAE, and the generation went from ~14s to ~4s, which is about 70% performance improvement.

[wywywywy]

Just did some testing on my side.

RTX3090, SDXL, 1024x1024 with 5 LORAs, batch of 8, 30 steps DPM++ 2M Karras.

Method	Time
Automatic1111	1m45s
Forge	1m10s
ComfyUI	1m07s
ComfyUI + OneDiff	0m30s!
ComfyUI + OneDiff + DeepCache	0m14s!!!

That's a HUGE improvement!

Obviously the first time OneDiff is used it takes time to compile (few mins), but the compiled cache is saved on disk so it doesn't need to recompile after.

[pinea00]

You can try DPM SDE turbo sampler in forge(step8 cfg2)

[minsky91]

Thanks, impressive-looking gains! I installed DeepCache under Automatic and got about 25% speedup when generating a batch of images, no visible degradation of the quality. Installed and activated it in Forge as well but it didn't make any difference in runtimes at best, or caused an error when enabled fully. This is what you meant under "doesn't work with ControlNet nor Forge", I guess.

I tried installing OneDiff in Automatic, but its script doesn't show up in the UI after installing and applying the changes. Probably needs an additional setup action, no time to pursue this further.

ComfyUI I don't use (being antagonistic to the workflow interface), but I really envy its speed.