How do I choose whether the subtitles are spoken language or written language?

[DayC22]

I am a Cantonese speaker
When using WHISPER a few days ago, the subtitles produced were in written language
But when I use it today, I found that it was changed to spoken language

Is there a way to choose whether the subtitles are spoken or written?

[jongwook]

We only had a single label zh for Chinese, and during training the model has seen both Cantonese and Mandarin labels for Cantonese speech. So it has been observed that sometimes the model would "translate" spoken Cantonese to the standard Mandarin form.

It might be possible to steer the model in either way by prompting with some phrase specific to Cantonese/Mandarin, in a similar manner to #277 (comment) where it controlled simplified/traditional writing by just one different character.

[cuichungman]

FYI - that Chinese is not a single language, but it is a family of several languages. So using "zh" is not an appropriate way for a single language, neither Cantonese nor Mandarin.

For the sake of long-term practical application, please consider a comprehensive solution to this issue. It is advised to label Cantonese as "yue" and Mandarin as "cmn," in accordance with their respective language codes. Continually using "zh" may result in the same problems that the author has highlighted.

[BrightXiaoHan]

Is it possible for whisper to detect whether the audio is Cantonese or Mandarin out of the box?