How to load LLM with multiple GPUs

[yileitu]

I want to use Llama-2-13b-hf to do NER task like code examples in https://spacy.io/usage/large-language-models

[components.llm.task]
@llm_tasks = "spacy.NER.v3"

[components.llm.model]
@llm_models = "spacy.Llama2.v1"
name = "Llama-2-13b-hf"

But I only got several RTX3090 cards. How to configure to perform inference on multiple GPUs in this spaCy library?

[svlandeg]

Duplicate of explosion/spacy-llm#424 - we'll go ahead and close this one.