Whisper Model Testing

[jpvajda]

For capturing feedback on testing Deepgram's support of Open AI Whisper Models.

[notAPanda]

Hey Everyone.

Whisper gives me great results in Polish language but is lacking paragraphs and punctuation through Deepgrams api. Are paragraphs going to work anytime soon?

[jpvajda]

@notAPanda 👋 Thanks for asking this question! I'm checking with one of our engineering teams to confirm if Paragraphs will be supported when we launch our Whisper Models on Production. You are currently testing a very early beta of Whisper so it doesn't have all the Deepgram capabilities just yet.

[notAPanda]

To be precise. Punctuation works with Whisper out of the box.
Only problem I have is that there are no paragraphs when using Whisper model.

[notAPanda]

There seems to be another problem with whisper. Start and End properties of a word is set to 0.
I am able to workaround the Paragraphs problem with a simple function. But I need Start and End properties for producing SRT and VTT files.

Screen:

[notAPanda]

I know Whisper does not provide start and end data on a word level. It provides start end end on the segment level. Hoping to find a solution on word level or paragraph level.

[notAPanda]

Today I have made some additional tests... and it is very inconsistent...
I am transcribing the same audio file for testing purposes... and it keeps getting worse.
I have multiple repeated word sequences that do not apear inside the audio file.

"O, od komieniu. O, od komieniu. O, od komieniu. O, od komieniu. O, od komieniu. O, od komieniu. O, od komieniu."

Nonsense... where audio file has some preety decent dialog going on.
It does not happen when using Whisper on my local machine.

[jpvajda]

@notAPanda thanks for all the feedback here. You are currently testing a very early version of our support of the Whisper models, so they lack a lot of the core capabilities of Deepgram. We are currently working on releasing improved Whisper models on production which will be coming in the next few weeks with support for many of our Deepgram features.

Let me know if you'd like to test out these new models on our beta endpoint. It's a invite only beta, but I can provision a project for you to test, Just send a request to test Whisper on Beta to product@deepgram.com and I can get you set up.

[exbald]

I have a rather long audio ~ 4 hours in Spanish. It has been pending for a day. Is there a way to cancel it or actually transcribe it?

Thanks!
IDs:
cc095616-b0cb-4e1a-988c-22d501caf293
cc095616-b0cb-4e1a-988c-22d501caf293
cc095616-b0cb-4e1a-988c-22d501caf293

[jpvajda]

@exbald Deepgram has a 10 minute time out on all transcription requests, assuming this request time out you should have received a 504 error in the Console Logs.

You can definitely try requesting again, and I'd recommend trying a smaller size whisper model unless you really need the large one. We support: tiny, base , small and medium which should run more quickly.

Here is helpful guide we made for using Whisper Cloud:

https://developers.deepgram.com/documentation/guides/deepgram-whisper-cloud/

If you are still seeing these requests as pending in your Console logs just let me know and I can have someone look into it.