Tatar language (--language tt or --language Tatar) shows result with wrong alphabet

[beatthem]

Hello!
I use whisper (openai-whisper==20230314) for transcribing tatar language audio [https://www.youtube.com/watch?v=w_m_q34Q-qA](from youtube video) - first changed to mp3 with ffmpeg videoplayback.mp4 tatar.mp3, then used command

whisper tatar.mp3 --model large --language Tatar

also tried

whisper tatar.mp3 --model large --language tt

result is the same - have this transcription

[00:00.000 --> 00:08.540]  Қышы табиғаты білән ялғыш жән, мүмин қышы қаталанса, ялғышса, аңа тавбә істіқфар ітарға ашығырға керег.
[00:08.540 --> 00:14.540]  Дұрүсліктә, Аллақ аның құлы тавбә ітгәңгә сұйыны қәдістәк ілә,
[00:14.540 --> 00:20.680]  бір құз шүлді, бір өзі қолып, ұғалтған диясын тавып сұйын ғәңге қорағанда,
[00:20.680 --> 00:25.440]  Аллақ шынлықта өзінің құлы тавбә ітгәңгә көбір әк сұйына.

but there are no i, 'ұ', 'қ' letters in tatar alphabet.
Tatar alphabet:
https://en.wikipedia.org/wiki/Tatar_alphabet
(look accepted spelling, it is now used in Tatarstan (the motherland for tatars)

[phineas-pta]

whisper doesn't perform well on languages with low resources (the paper said there're only 14h of tatar audio, no error rate given)

to make it ready-to-use you have to fine tune whisper

[phineas-pta]

that is certainly about quality, please take a look at the transcript: those characters i ұ қ appear randomly but very often, meaning the model may not yet attain convergence for tatar language and spill out random characters

you have to fine tune for tatar language

[phineas-pta]

or maybe it's a problem with the tokenizer, take a look at the default openai tokenizer to see if tatar is well supported

[Tex2002ans]

whisper doesn't perform well on languages with low resources (the paper said there're only 14h of tatar audio

I'm suspecting most (all?) of that 14 hours was from Mozilla's:

• Common Voice

As of today, it looks like there's 31 hours of validated Tatar audio, but still, incredibly low.

@beatthem, if you want more accurate Tatar Speech-to-Text, it may be good to focus on getting:

• More/better Tatar sentences
• More Tatar speakers

in Common Voice too! :)

Then, like phineas-pta said, that data can then be used to train Whisper better for your specific language.

---

Side Note: One of the datasets Whisper was trained on was:

• Common Voice v9.0 (April 2022).

Today, Common Voice v13.0 (April 2023) is out.

---

Side Note 2: You said this:

but there are no i, 'ұ', 'қ' letters in tatar alphabet.
[...]
Tatar alphabet: https://en.wikipedia.org/wiki/Tatar_alphabet

Although, on the Wikipedia page, it says қ is a newer accepted letter since 1989:

In 1972, prof. Nikolai Baskakov suggested three new letters to be added to the Tatar Cyrillic alphabet: Қ, Ғ and Ў for the sounds [q], [ʁ] and [w], to make the Tatar spelling phonetic. On 18 May 1989, the Orthographic Commission formed by the KFAN published the new alphabet, which included Baskakov's three new letters, and the new spelling rules.[11] The new alphabetic order was as follows, with the new letters shown in brackets:

А Ә Б В [Ў] Г [Ғ] Д Е (Ё) Ж Җ З И Й К [Қ] Л М Н Ң О Ө П Р С Т У Ү Ф Х Һ Ц Ч Ш Щ Ъ Ы Ь Э Ю Я

қ is towards the middle of the list, in the brackets!

Side Note 2.1: I also did a search in Common Voice's Tatar sentences (~18000 sentences), and I don't see i, ұ, or қ anywhere... so unsure exactly where those 3 characters are coming from.

[phineas-pta]

openai didn't use commonvoice in training but in evaluating and comparing whisper to other models

[Tex2002ans]

openai didn't use commonvoice in training but in evaluating and comparing whisper to other models

Ahh, okay. Thanks for the clarification.