text-to-text translation

[BaGRoS]

Hi
Please add an option so that I can use Whisper to translate text-to-text, from different languages ​​into English.
BaGRoS

[zanjabil2502]

Input of whisper is a audio, you can use text to speech first and the audio as input of whisper

[zanjabil2502]

Maybe you can use NLLB from META, this model have more 200 language translation

[MeinDeutschkurs]

Maybe you can use NLLB from META, this model have more 200 language translation

May I ask you for a link? I googled lots, but I'm not sure if I found the correct resource. Thx, Johannes

[zanjabil2502]

Maybe you can use NLLB from META, this model have more 200 language translation

May I ask you for a link? I googled lots, but I'm not sure if I found the correct resource. Thx, Johannes

Use can choose link below:

Official Meta:
https://github.com/facebookresearch/fairseq/tree/nllb

Official Huggingface:
https://huggingface.co/facebook/nllb-200-distilled-600M

Official Ctranslate2:
https://opennmt.net/CTranslate2/guides/transformers.html

You can choose some models from NLLB:

• Distilled 600M
• 600 M
• Distilled 1.3 B
• 1.3 B
• 3.3 B

[zanjabil2502]

From my experience, if you want have the best result, first you want transcribe the audio from original source, example the audio using indonesian language, so you must transcribe it to indonesian language.

After that, if you want translate it to other language, as example japanese, first you must translate the indonesian lang to english lang, after it you translate from english to japanese

This process have the best result but it will have processing time longer

[MeinDeutschkurs]

Thank you so much for the links and for sharing your experiences. I‘ll have a look into it. Can‘t wait to see the results.

[jongwook]

Whisper can't do this out of the box; duplicate of #36, #201

[EliasPereirah]

That would be great.

[pr0mila]

I’ve created a code-switched language dataset for fine-tuning Whisper, including audio data along with CSV and Parquet files, which I’ve stored on Hugging Face. After preparing the dataset, I fine-tuned the model for translation. You can explore the entire end-to-end project in my repo. Here’s the link to check it out: https://github.com/pr0mila/MediBeng-Whisper-Tiny