Can Apple Silicon machines use the new ane_transformers with pytorch2.x ?

[theesfeld]

Mac M1 Pro here, installing the PyTorch 2.x nightly made a huge difference for me in generation times.

Today, I read this news: https://github.com/apple/ml-ane-transformers

Is this something that we can use in place of the xformers module that doesn't load on M1 Macs?

Thanks for the help

[system1system2]

Great question. Hopefully, @brkirch will provide guidance at some point.

[theesfeld]

Agreed. I hope we can drop this in place of xformers module. Currently, M1 Pro cannot load xformers due to lack of gpu, even though we have a neural core array onboard!