sentence level language detection

[josephwong14wkh]

Did anyone has idea on sentence-level language detection? I am currently trying to finetune whisper and hope that it can detect multiple languages given an audio. For example, given a 10s audio with two speakers , if one speaker speaks english in first 5s and another speaker speaks Mandarin in the next 5s, i want whisper to detect "en" for the first 5s and "zh" for the next 5s.

[glangford]

See the ongoing discussion and example code here

• Multi-Language Audio and Transcription Inconsistencies #2009

[gyllila]

I’m afraid you’ll have to split the audio in sentences yourself, save in separate files, then use Whisper for each. It’s not difficult to write a small script and do this routinely. For splitting in sentences, you can use the "Analyze/Stille-Finder" function in Audacity, or the python library pydub.