Transcribing WhatsApp voice notes using Whisper

[bjnortier]

Hi, I wrote a transcriber for WhatsApp voice notes. Blog post is here:
https://www.bjnortier.com/2022/10/07/I-Hate-Voicenotes.html

Demo video is here:
https://www.bjnortier.com/assets/videos/ihatevoicenotes-demo.mp4

It works quite well but privacy is a concern.

[imsaksham-c]

Looks cool, can you provide some details on how you integrated this?

[bjnortier]

Such as? There are some details on the blog. I've discarded this project and I'm now focusing on the iPhone app that uses Whisper
https://apps.apple.com/us/app/hello-transcribe/id6443919768

[jasalt]

Sounds useful, nice demo. This could benefit from using Matrix.org as with it the same code would work also with Telegram and Signal etc, plus give better security options as personal WhatsApp account could be used and scripted.

Also, I checked the Hello Transcribe app but did not quite get it's utility yet over default iOS 16 live dictation and Just Press Record app which has decent transcription but is otherwise pretty great in usability terms with the Watch App and iCloud sync, been user for 5 years. Cool that you got it running and all the way to App Store though, must have been plenty of work. EDIT: noted you had a thread for it here #443, but leaving this comment here for now.

[bjnortier]

Several users have told me Whisper does much better than iOS live dictation or Just Press Record for their accents, but I get it won't offer utility over these for some users. Happy to serve those users for whom Whisper does a better job :)

[Sisekelo]

hi @bjnortier are you willing to share the ihatevoicenotes project?

trying to do something similar but having a few issues:

1. downloading the audio from the whatsapp url
2. converting the .ogg file to mp3 so openai can use it.

any advice?

[altbert]

Here is a similar project that i've created Whatsapp Speech To Text
With some simple changes you certainly can do the same thing that ihatevoicenotes does.
Also in the branch main_openai_api i do convert the ogg to mp3 using ffmpeg python library.

[bjnortier]

Made the repos public :) There's a NodeJS app for the web hooks. It adds the received message to a queue. The python worker reads it from the queue. Can't really remember but looking at the code the python Whisper package does automatic conversion using ffmpeg

https://github.com/bjnortier/ihatevoicenotes-whatsapp-webhooks
https://github.com/bjnortier/ihatevoicenotes-whatsapp-worker

[Sisekelo]

thank you very much :)