Whisper base model: Transcript generated from Audio file indicating slow response time.

[purusothaman123]

We have used the below configuration in AWS server to generate the text from audio using whisper base module. For testing we used 5 mb audio file which is taking 10 to 15 minutes to generate the transcript. Similarly it takes more time for the audio files having more than 5mb.Can you provide the details on what type of configuration should be used to get the quick response in whisper base module.

AWS configuration details
EC2 details
instance type : t2.xlarge
vCPUs : 4
Memory : 16 GB RAM

[phineas-pta]

use gpu

[montesclarosglennbenedict]

Like @phineas-pta said, use a GPU. T4 GPUs appear to be the standard here which anything above T4 having only diminishing returns. You can also run multiple instances of the model per GPU to get work done faster. There's also batch interference to consider.

If you're dead set using a CPU for whatever reason, you can use model quantisation but again, GPUs have been the golden standard for doing this stuff.