Is it possible to train with multiple GPUs?

[elozano98]

Train with multiple GPUs

I am training my transformer pipeline using my GPU:

$ python -m spacy train ./config/config.cfg --output ./models --gpu-id 0

Since I have a lot of data, the training process is not fast enough, so I would like to know if there is an option to enable more than one GPU when training.

Thank you!

[MightyGoldenOctopus]

Take a look to distributed training capabilities through Ray with SpaCy: https://spacy.io/usage/training#parallel-training

[polm]

Multi-GPU support is still very much a work in progress. Take a look at #8093 for a recent thread on the subject.

[elozano98]

I've read something about using spacy-ray to configure workers so that each one is associated with one particular GPU. Is it possible?

[polm]

That's the approach they took in that thread, which I think had some issues. That's everything I know about the subject, sorry.