nvFuser and TensorRT for huge performance gains, implementation when?

[NikkMann]

As described in this reddit thread, apparently using pytorch's new nvFuser makes SD 2.5 times faster on full precision:
https://www.reddit.com/r/MachineLearning/comments/xa75km/p_pytorchs_newest_nvfuser_on_stable_diffusion_to/

And as in this reddit thread, using TensorRT will have an AT LEAST 25% performance improvement:
https://www.reddit.com/r/StableDiffusion/comments/xdxika/making_stable_diffusion_25_faster_using_tensorrt/

Honestly surprised Automatic111 hasn't hopped on these, at least to talk about them.

[ClashSAN]

There's Xformers, already implemented, that doesn't require diffusers version of sd.
Not all optimizations stack to provide a speed increase, and I think those example would be comparing against an older hugginface diffusers version that was slower

[NikkMann]

Wouldn't it be worth a shot though? xformers is around 25 to 50% faster depending on the system. With nvFuser and TensorRT together I'm sure we could get at least another 25%

[NikkMann]

The reddit post for nvFuser has them using SD 1.4 looks like

[78Alpha]

There is a github with it for testing

https://github.com/VoltaML/voltaML-fast-stable-diffusion