The concept of transcribed bilingualism.

[Ye83]

When recording in real time, for example, I specified the output voice to be English, and the audio played was a mixture of Japanese and English, sometimes translating Japanese into English and sometimes not. Can this be set to fixed not translate or translate?

[makaveli10]

Hello @Ye83 Can you share the commands used to run whisper server and client?

[Ye83]

@makaveli10 Thank you for your answer,
This is the server code：

This is the client code：
The client connects directly with the webSocket and sends this json string when the connection is successful:
{
"uid": "7b6d5a7c-c878-42fa-8493-43025b1e34ee",
"language": "en",
"task": "transcribe",
"model": "small",
"use_vad": true
}
And Can I add your personal contact information? I have a lot of questions to ask you.

[makaveli10]

Thanks for sharing the details. One thing you could try is set the language as japanese and the task as translate.

[Ye83]

This is the result of watching youtube bilingual videos using the Google plugin. The task is translate and the language is en, but the output is still in Chinese. You can try the Google plugin and watch this video for a few minutes, and the same should happen.
youtube video link:https://www.youtube.com/watch?v=J9M-Xgt5qzw
Thank you for your reply. I need your help

[makaveli10]

@Ye83 after testing this with large-v3 it seems like an issue when there are two languages, it works well in translating chinese to english when there is only chinese present in the audio.

And Can I add your personal contact information? I have a lot of questions to ask you.

The contact details are in the readme. Happy to answer all questions. Thanks!

[Ye83]

Thank you for conducting the tests and providing the answers. Is there a possibility to optimize this issue? It would be greatly appreciated. @makaveli10

[jsichi]

I added a start on an implementation in #200 for a semi-related issue, where I wanted the transcription to be able to preserve multiple input languages, and also wanted to restrict the set of languages to listen for (since sometimes I was seeing Chinese where it should have been Russian, for example).