FP16 model size same as FP32?

[jzju]

I have a 6GB gpu that barely fit the medium model and get the same out of memory error when using FP16 flag. This project running whisper with FP16 CPU https://github.com/ggerganov/whisper.cpp#memory-usage seem to only requre 1.7GB RAM. Is there a FP16 model that only requires 2GB VRAM on medium?

[NanaOrangejuice]

Check https://github.com/openai/whisper/blob/main/whisper/__init__.py, *.pt in dict _MODELS are all FP16 models.

[phineas-pta]

yes the fp16 option is misleading, it doesn't load the model as float16

i asked a question #1175 but no answer

whisper.cpp doesn't use GPU for now

you can check out https://github.com/guillaumekln/faster-whisper for lower VRAM