Unsure if using GPU or not during transcription

[vitthal-bhandari]

I have written a very simple barebones code to transcribe audio segments using whisper. The code can be accessed here.

I am running the code on my jupyter notebook and my laptop has a GPU. (for sharing I have changed it to a colab file)
I am unsure if the underlying whisper model utilizes my GPU or not. Transcribing a 16-minute audio took a couple of minutes and I also see the warning message below -

D:\Softwares\Anaconda\lib\site-packages\whisper\transcribe.py:114: UserWarning: FP16 is not supported on CPU; using FP32 instead
warnings.warn("FP16 is not supported on CPU; using FP32 instead")

running the !nvidia-smi command gives the below output -

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 512.72       Driver Version: 512.72       CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ... WDDM  | 00000000:01:00.0 Off |                  N/A |
| N/A   49C    P8    N/A /  N/A |      0MiB /  4096MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

If the model is not using my GPU, how can I ensure that it does so?

[phineas-pta]

try whisper.load_model("tiny", device="cuda:0")