What file formats are supported for input?

[rumpleniltkins]

Just wondering what file types are supported by the model. Are ogg vorbis files acceptable input?

[glangford]

ffmpeg is used to load audio (see the link to the code below), so the input type must be supported by ffmpeg. Note that you can also use a container such as .mp4 of audio+video as input.

whisper/whisper/audio.py

Line 22 in 28769fc

def load_audio(file: str, sr: int = SAMPLE_RATE):

[glangford]

See ffmpeg -formats; on my platform there are 395 entries, I don't know if they all work :)

[gwpl]

Can we ask for support by Whisper API for OGG/OGA and FLAC?
(sorry if I didn't find right place to file such request).

As I read on https://platform.openai.com/docs/api-reference/audio/create#audio/create-file , official API does not support currently two very nice opensource formats:

• ogg/oga https://en.wikipedia.org/wiki/Vorbis
• FLAC https://en.wikipedia.org/wiki/FLAC

I have tons of recordings and it as default on a lot of software and devices.

As we see from example snippet OGG/OGA is supported:

for i in *.ogg; do
  ffmpeg -acodec libvorbis -i "$i" -acodec pcm_s16le "${i%ogg}wav"
done

( from https://stackoverflow.com/a/62267248/ )

ffmpeg -i inputfile.flac output.wav

from https://stackoverflow.com/a/23380032/544721

Can we please add support in Whipser API for OGG/OGA and FLAC formats?
Looking at codebase

whisper/whisper/audio.py

Line 46 in c09a7ae

ffmpeg.input(file, threads=0)

it looks that it's not a problem on whisper python level but more on Whisper API side.

Otherwise, anyway I would be thankful for making sure it works well with github.com/openai/whisper for local deployments.

[Pachocastillosr]

+1 for OGG

[jongwook]

FLAC and OGG are now supported in the Whisper API!

[gwpl]

Thank you! That's a great news!

Shouldn't documentation get updated to celebrate this! https://platform.openai.com/docs/api-reference/audio/create#audio/create-file ?

[shyamal-anadkat]

we should get the docs updated!

[rattrayalex]

Proposed adding this to the OpenAPI spec here: openai/openai-openapi#64

[Hobbymaker-de]

.opus and .amr supported, too. (used by messengers)

[SwimSwag10]

Could you provide the documentational proof that .amr and .opus are supported? Last I checked they were not (at least .amr was not).

[Hobbymaker-de]

I wrote the comment, because there is no documentation. But it is my daily business to transcript .opus and .amr files from WhatsApp-Chats and SKY-Messenger - and it works well; I transcribed thousands.....
I think it depends on the ffmpeg.