Core ML for Stable Diffusion in macOS 13.1 and iOS 16.2

[StableInquest]

Apple announced this today: https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon

How would one go about integrating this version (https://github.com/apple/ml-stable-diffusion) into the 1111 webui?

[ClashSAN]

That architecture needs diffusers, webui uses the original stable-diffusion architecture.
I don't think it diffusers will be integrated, as trying to do so may break many existing features and functions for the program users.

[StableInquest]

https://huggingface.co/blog/diffusers-coreml

[autumnmotor]

The current pytorch implementation is (slightly) faster than coreml. At least for now.
Check out my post at the URL below.
#5309

Only if there is a clear benefit, such as a significant speed improvement, should you consider integrating it into the webui.

My guess is that Apple announced support for SD in the coreml format, but I think they wanted to appeal that SD can be executed on mobile terminals (iPhone, iPad).
This announcement got me a little excited.
However, as a result of benchmarking the sample code, I thought that the coreml format would not bring good news for webui.
At least for now.

[StableInquest]

that's unfortunate to hear. I see you tested on the latest beta 4 as well in your benchmarks. I was excited initially with the announcement. Hopefully as more develops it can be incorporated and provide gains, especially for training.

[calben99]

I just tested coreml using Mochi Diffusion. It is 3x faster than my automatic1111 setup, which has been fully optimized with torch 2.0 etc...