Does Whisper only Support Nvidia GPU's?

[RaymondTheBrave0]

GPU support in Whisper.

Does Whisper only support Nvidia GPU’s?

I have an AMD Radeon RX 570 Graphics card which has 8GB GDDR5 Ram which would be great for processing the transcription.

If Whisper only supports Nvidia, is support being planned for AMD Graphics cards any time soon. I am using Linux Mint.

[jongwook]

It should work if you install pytorch packages specially built for ROCm, although it's not the most straightforward process and I haven't tried; more discussions in:

• ROCm support #55

[RaymondTheBrave0]

Thanks to everyone for their input. It all helped.

I downloaded from https://www.openssl.org/source/ the latest version which was 3.0.12 at the time and installed it via the make commands etc . I had 3.0.2 on my linux mint system but this did not seem to be up to date enough.
I then decided to use anaconda to create a separate environment for AI Whisper as Anaconda I found out is good at making sure pre-reqs etc are met and it also list which packages etc are installed in the env.
I am currently running the command to install the Roc software for my AMD graphics card with no errors about SSL at all. I also found out that there is no support for AMD graphics cards for Whisper in Windows only on Linux.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.6

[ignacio82]

@RaymondTheBrave0 any chance that you would consider making a docker container available so people can run whisper with an AMD GPU?

[RaymondTheBrave0]

I have not gotten the AMD GPU option to work at all. I tried everything and then gave up. There were no real clear instructions on how to debug the issue. I am not even sure that my GPU is supported which is Raedon RX 570.

It is all very opaque when it comes to finding out what is supported and how.

Thanks

[viebrix]

@RaymondTheBrave0 Yesterday I managed to get Whisper (or Whisper-WEBUI) start and running with GPU (GFX803) RX 580 (8GB). Should be similar to your RX 570
Here is the process description and also my compiled whl files. But be careful installing those on your own risk. Because hardware differs a little bit.
https://github.com/viebrix/pytorch-gfx803-for-Whisper

[VTrngNghia]

I have a 6700XT. Should that work?

[viebrix]

I have no 6700XT but I have read that the gfx1031 is treated as gfx1030 so you can try to use:

export HSA_OVERRIDE_GFX_VERSION=10.3.0 instead of HSA_OVERRIDE_GFX_VERSION=8.0.3
if this not work try:
export HSA_OVERRIDE_GFX_VERSION=10.3.1 instead of HSA_OVERRIDE_GFX_VERSION=8.0.3

and

export PYTORCH_ROCM_ARCH=gfx1030 instead of export PYTORCH_ROCM_ARCH=gfx803
if this not work try
export PYTORCH_ROCM_ARCH=gfx1031 instead of export PYTORCH_ROCM_ARCH=gfx803

see also:
https://www.reddit.com/r/ROCm/comments/uax358/build_for_unofficial_supported_gpu_6700xt_gfx1031/

https://www.computerbase.de/forum/threads/amd-rocm-5-6-fuer-rdna3-und-radeon-pro-gpus.2149817/

Disclaimer: I do not have this hardware and I can not say how it will react. Can be that it work, there is also a little chance that your hardware gets corrupt/defect...