Local-first, real time translation of livestreams using a custom tuned Whisper model

[danirukun]

https://github.com/cyberofficial/Synthalingua
(I am just a contributor, not the original author)

This implementation is a standalone wrapper of Whisper with a heavily tuned model for conversational english, japanese, indonesian etc., and should work especially well for things like live streams on Youtube or Twitch, with an optional Discord webhook to send translations to Discord.

[dgoryeo]

Hi @danirukun , this looks quite interesting. Is there any way to input audio from other sources than microphone, for example from a Zoom conference, or a VLC media player?

[danirukun]

Hi, yes, it accepts any audio input device and can be configured in the arguments to the script.

[ZQ-Dev8]

Very cool! Looking forward to trying the reverse translation once it's implemented!

[cyberofficial]

:0 Just found your comment, it has been implemented for little bit now.

cyberofficial/Synthalingua#38

Though, the script was tiny bit updated to ver since release though Release 1.0.997

Uses a custom fine tuned model (in formal talk, so business stuff and news casting stuff won't be as strong) but still works fine majority of time.

There's also a portable version 1.0.996 + Portable (1.0.997 is just a setup fix.)

Give it a whirl and submit any issues or feedback you may have 💖