How to run whisper on Amazon EC2 Inf1 Instances

[toanhuynhnguyen]

How to run whisper on Amazon EC2 Inf1 Instances?
Thank you.

[ryanheise]

Take a look at this tutorial.

[toanhuynhnguyen]

@ryanheise Do you think that tutorial so complicated, and have you tried it.
I just need to run whisper on EC2 Inf1 Instances like normal EC2 instance or like we run it in our local PC.

[ryanheise]

I haven't used Inf1 instances, although it is clear from the documentation that it requires the use of a specialised SDK to prepare the model for inference on that hardware. If you're not prepared to do that, then I would not recommend an Inf1 instance because that is what you would have to do if you wanted to use that hardware. If you'd like to run it just like on a local PC, then I would suggest an EC2 instance with more standard hardware like the g4dn.xlarge instance type which has an NVIDIA T4 GPU. (NVIDIA also provides an AMI for it with the NVIDIA drivers preinstalled.)

But since this question is about running on AWS Inferentia chips, the tutorial I linked to hopefully answers that.