GPU works only first time

[Atefeh197]

Hello

I use the whisper model for English audio files, the first time I used the whisper with GPU, it worked properly, but after that, it had a memory error.

"Cuda is out of memory"!

This is my code:

model = whisper.load_model('large-v2','cpu')
result = model.transcribe("1.wav")

How I can fix the error?

[phineas-pta]

you can try to free up gpu memory: torch.cuda.empty_cache()

[Atefeh197]

Thank you.
I used it, but it does not help me. I also get "cuda out of memory"!

[phineas-pta]

seems like you don't have lots of VRAM, i think you have to reload the model after each usage, or use the medium model

[Atefeh197]

My memory is 64GiB.

I reload the model after each usage.

it is weird because the first time it worked with the large model!

I do not understand what is the problem.

[Atefeh197]

This is my error:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 7.92 GiB total capacity; 6.40 GiB already allocated; 44.00 MiB free; 6.74 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[phineas-pta]

you only have 8 GB VRAM and that's not enough for large model

[jongwook]

Looks like the your VRAM (8GB) is not big enough to keep running the large-v2 model using the implementation in this repo, which is not very well optimized for memory. You may have a better chance with other implementations like whisper-jax or faster-whisper; the latter seems to require only ~5GB memory for running large-v2.