x-stable-diffusion port for massive speed up? (TensorRT, nvFuser etc.)

[aliencaocao]

I came across this implementation of SD that uses many optimizations, and most of them I think can be ported and (optionally) enabled by the user. The speed up shown is very significant.

https://github.com/stochasticai/x-stable-diffusion

Anyone familiar with this repo have any idea how easy/hard will it be to port over these optimizations? I can work on them in Dec if it is not excessively hard.

[ghost]

The question is, can it be combined with xformers? If it is so much faster, we might ditch xformers entirely.

There are also open questions about image quality and animation capabilities

[aliencaocao]

The repo already use xformers, namely their FlashAttention. For accuracy, it should be the same level as existing FP16+xformers, since the rest of the optimizations are more on the hardware-software bridging level, not quantization.

What I feel is that users who want absolute accuracy should not be using the existing optimizations in the first place, like xformers or FP16, while those with limited VRAM and hardware will want the fastest possible optimizations at cost of accuracy.

[BunnyViking]

agree, im all for options. I myself don't use xformers because I want consistency but im all for having the option to do stuff faster if i'm just messing about or super rough drafting.

[Somasonic]

IME xformers negatively impacts IQ, so if there’s a more accurate speed optimisation I think it’s worth investigating.

[grexzen]

The TensorRT kernels are the main speed booster, I have seen them in the GPT models, that is awesome they can be used here.

https://colab.research.google.com/drive/1mT9CzFgZCCrakC0N-GkmvDgSkIbE2cWq#scrollTo=SaJmSjD0VXgu

They transform the model to onnx, then to TensorRT.

https://www.photoroom.com/tech/stable-diffusion-25-percent-faster-and-save-seconds/

[Centurion-Rome]

Maybe someone of devs will take a look on it :)

[78Alpha]

I suppose this is relevant
https://github.com/VoltaML/voltaML-fast-stable-diffusion