Suggestion: Add optional language parameter to Whisper (override automatic language detection)

[mark-reijerkerk]

The Whisper model card shows that the decoding pipeline already includes a LANGUAGE TAG stage before transcription. This means Whisper is already internally capable of transcribing in a specific language when the tag is known.

Right now, the model always performs automatic language identification, but this can be unreliable for multilingual speakers, mixed-accent audio, or short/noisy recordings.

Proposal:
Add an optional user-provided language parameter that overrides Whisper’s automatic language detection.
When this parameter is not provided, Whisper continues to detect the language automatically as it does today.

This would give developers more control, improve transcription accuracy, and avoid unintended translation or language switching — without requiring any change to the underlying Whisper architecture.

[MarktHart]

Try the --language arg

[mark-reijerkerk]

Thanks — I’m aware of the --language arg. The issue is that it doesn’t function as a strict constraint; Whisper still switches languages under uncertainty. My proposal is aimed at giving the decoder an actual hard language lock, not a soft preference.

[MarktHart]

Could you share an audio file that actually goes wrong?

Here the language arg is given to the tokenizer, here it's added to the tokenizer's sot_seqeunce, which is then used via here and here.

Retrying the decode (with fallback) doesn't look like it's able to overwrite the language token again, but instead retries the whole sequence again with another temperature.

In short: the language token does seem to be locked in when the language flag is used. You might be seeing the model outputting a different language as hallucination, or the model outputting english when translating, which are both not solvable with code.