Model training using M1 metal gpu.

[Anand195]

Hello, guys

As stated in #11436, we finally get hands on with model training using M1 metal gpu.

I was trying to train the NER model on my m1 Mac metal gpu using custom dataset, it takes too long to train.
Do we have any fix or solution for slow training on metal gpu?
I know it's experimental feature. it's great to train transformer model on m1 gpu.

Any help or suggestions are appreciable.

[rmitsch]

Hi @Anand195, what does "too long" mean exactly? How long does the training take on a different GPU?

[Anand195]

It take around 5-7 hours on M1 metal GPU to train the model for just 300 file. specifically 120 in test/dev and 280 in train, and model training takes around 1-2 hours on colab GPU with the same dataset.

[danieldk]

Training models on a regular M1 Mac is going to be many times slower than a reasonably modern CUDA-capable GPU. Even training on an M1 Ultra with 64 GPU cores is going to be much slower. The compute power of an M1 Ultra is 21 TFLOPS, the compute power of the Tensor Cores on an RTX 2060 Super, a budget card from 2019, is 57 TFLOPS. Besides that, the PyTorch CUDA kernels benefit from years of optimization compared to Metal kernels, and some some PyTorch ops are not yet implemented in Metal and fall back to CPU kernels.

It is currently not cost-effective to train larger models like transformers on Apple Silicon GPUs. In my opinion it's currently primarily useful for small test runs during development. (And of course for inference on Macs.)

[Anand195]

Hi @danieldk ,
Thanks for the quick reply and now I got the clear picture about slow model training on Apple silicon.
As of now will continue our model training using CUDA only!