Language detected in 30 seconds, transcript returned in another language's character set.

[mequanent]

I am a beginner to ASR and saw 'Amharic' language in the Whisper's list of languages. I tried it with a 39 seconds audio and detected the language correctly but returned the transcript in Hebrew. Both languages are Semitic languages.
Some expected starting words , to show how the Amharic characters look like: "በኦሮሚያ ክልል ግጭት በተቀሰቀሰባቸው አካባቢዎች ..."
The sample audio I used can be found on this link: https://github.com/mequanent/Audio-Processing/blob/main/sample_amharic_audio.wav
I will appreciate if I can get any comment on this case. Thank you.

20 hrs of Amharic Speech Dataset: https://github.com/getalp/ALFFA_PUBLIC/tree/master/ASR/AMHARIC. [In case it can be considered in future versions]

[jongwook]

Unfortunately Whisper is really bad at Amharic, and has seen little to no Amharic script. In Appendix E of the paper it shows that it has only seen a small amount of Amharic audio -> English caption pairs, but the speech translation evals in D.3.1 shows that it's far from being useful.

[mequanent]

Thank you for your explanation, appreciated! I understand that Whisper has seen no Amharic script.
In case Whisper will be updated and you want to consider Amharic, the above-mentioned 20 hrs of data may help. Because it has the corresponding Amharic script for each audio. In addition, the Data and Knowledge Engineering Group also presented a larger Amharic speech dataset, with Amharic script included. It can be found here: https://www.findke.ovgu.de/findke/en/Research/Data+Sets/Amharic+Speech+Corpus.html (I attached the screenshot of the website to make sure that the link is legit and I downloaded from it safely.)
I can contribute in its dataset-specific pre-processing steps. Thank you!

[bemnet4u]

Common voice also has some data https://commonvoice.mozilla.org/en/datasets. I am interested to have Amharic ASR. I am native speaker. If there are things I can help with, I am interested to know.

[mequanent]

Thank you for your update on the Common Voice. Amharic was not included last year and I remember that I filled an application form for it to be included, keeping in mind that others too may also did the same.

[vbachani]

Unfortunately, Amharic ASR is pretty much non-usable even with Whisper v3. As @bemnet4u pointed out, Common Voice has some Amharic data—the latest version (16.1, dated 4 January 2024) has 1,223 Amharic samples, which is not bad.

This post explains how to fine tune Whisper with Common Voice data (and even has a Google Colab notebook for doing so), but it's quite involved and not easy to follow for me. If anyone else has any luck, please let me know or post back here! Then we might add to this list with a working Amharic ASR model to save future Amharic transcribers the trouble of retracing our steps.