I can load the model on a specific GPU, but it still performs on the other one.

[eric781ctf]

I have acquired two RTX 4090 GPUs to run Whisper on numerous videos.

As part of the setup, I'm attempting to load two models on different GPUs using Python threading.

I used device='cuda:{}'.format(cuda) to assign each model to a specific GPU. Upon checking the task manager, I can confirm that both models have been successfully loaded onto their respective GPUs.

However, when I initiate the model.transcribe(file), the GPU usage on cuda:0, which is my first GPU, increases usage while no activity is observed on cuda:1.

I reference this method from the comment in #815, I would like to know how can i solve this problem.

Thanks!

Update

After using pynvml to record the GPUs usage, I realize that there might be some bug in the task manager to observe the usage of GPU 1.