How do whisper classify language?

[BaffledCoder]

Does whisper directly classify language from audio, or does it first converts to text and then classify language?

[glangford]

If the language is not specified, whisper classifies the language using the first 30s of audio. The function to do that is here:

whisper/whisper/decoding.py

Line 19 in e8622f9

def detect_language(

[BaffledCoder]

Thanks for sharing, but I meant that whisper detect_langugae is based on classifying directly from the audio that is mel-features
Or does it requires text to predict language of audio

[glangford]

Directly from the first 30s of audio.