How to get whisper to use GPU on Vultr A100 instance?

[Robert-K-S]

I spun up a 12288.00 MB NVIDIA A100 server on Vultr in the hopes that I would be able to get whisper to use the A100 GPU, but whenever I run whisper, it's still only using CPU. It seems that torch.cuda.is_available() is returning False.

Any tips or suggestions for how I can get whisper to run using GPU capability on an A100?

EDIT: I hadn't installed CUDA. Found instructions here. Going to give that a shot. I guess I just assumed that it would come with the default install.

EDIT2: My initial assumption was correct ("There's no need to install any drivers on a Vultr Cloud GPU. Licensed NVIDIA drivers and the CUDA Toolkit are preinstalled by cloud-init when you deploy our standard images."). Trying to reinstall CUDA made things worse. I found this page that helped me get back to the correct install.

[phineas-pta]

if cuda already installed but pytorch doesn't detect it, meaning your pytorch is cpu only, you have to reinstall pytorch with cuda support

[WildBirdTG]

is it possible that the issue was caused by pytorch version is not compatible with the installed CUDA version? Also some instances have different CUDA version results from nvcc and nvidia-smi.