running on cuda

[zootato]

Hi All,

Am able to run on cpu on ipynb with this code.

model = whisper.load_model("medium", 'cpu')
result = model.transcribe("TEST.mp3")
result

However when I try to run it with cuda, I get this error:
ValueError: Expected parameter logits (Tensor of shape (1, 51865)) of distribution Categorical(logits: torch.Size([1, 51865])) to satisfy the constraint IndependentConstraint(Real(), 1), but found invalid values:
tensor([[nan, nan, nan, ..., nan, nan, nan]], device='cuda:0')

Any help? Thank you.

[rspreet92]

Same problem here. I've tried all versions of cuda but no help. Currently using it with CPU only.

[underlines]

I had a similar problem on Windows 11 using WSL2 ubuntu on a RTX3080 10GB:

pip install git+https://github.com/openai/whisper.git
...
Successfully installed charset-normalizer-2.1.1 ffmpeg-python-0.2.0 filelock-3.8.0 future-0.18.2 huggingface-hub-0.10.1 idna-3.4 more-itertools-9.0.0 numpy-1.23.4 packaging-21.3 pyparsing-3.0.9 pyyaml-6.0 regex-2022.9.13 requests-2.28.1 tokenizers-0.13.1 torch-1.12.1 tqdm-4.64.1 transformers-4.23.1 typing-extensions-4.4.0 urllib3-1.26.12 whisper-1.0
...
sudo apt update && sudo apt install ffmpeg
...
(whisper) underlines@DESKTOP-Q2DTD09:~/github/whisper$ whisper cup\ noodle.mp3 --model large
100%|█████████████████████████████████████| 2.87G/2.87G [07:58<00:00, 6.45MiB/s]
/home/underlines/anaconda3/envs/whisper/lib/python3.10/site-packages/torch/cuda/__init__.py:146: UserWarning:
NVIDIA GeForce RTX 3080 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3080 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Traceback (most recent call last):
  File "/home/underlines/anaconda3/envs/whisper/bin/whisper", line 8, in <module>
    sys.exit(cli())

But following the error message's suggestion, I installed a newer version of PyTorch from here by selecting Stable > Windows > Pip > Python > CUDA 11.6 and running pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116 in my whisper venv

After that whisper was running on the GPU just fine.

[rspreet92]

Hey try this.
fp16=False in Python and --fp16 False in the command line.

This will use fp32 model instead of the default fp16 which I read that is not that good with GTX series.

[Jacky-MYQ]

i tried using with "cuda" that i got the same problem you facing with. I added "fp16=False" in this code model.transcribe("...",fp16=False), i found it worked and i got an useful ".srt" file.
I don't konw why!!! hahahahaha!!
Hoping this can help your guys!