Set specific GPU

[Onyoursix]

Is there a way to specify the GPU I want to use? I have a machine with multiple GPUs and I would like to load the model into a specific GPU instead of it defaulting to the first one.

[jongwook]

Setting the environment CUDA_VISIBLE_DEVICES will work, like:

CUDA_VISIBLE_DEVICES=3 whisper audio.wav

--device cuda:3 may also work, but I haven't tested.

[Aaryan369]

Tried with --device cuda:3, it is successfully able to load the model into the chosen GPU.

[nathan-skynet]

And in python ?

[glangford]

@nathan-skynet Try

model = whisper.load_model('large-v2', device='cuda:3')

See

whisper/whisper/__init__.py

Line 99 in ba3f3cd

def load_model(

[nathan-skynet]

Thanks

[psp631]

How to make specific modifications on line 99, thank you for your help.