Bug: some models are stopping transcribing the video before the end. Why? Anything that can be done about it?

[mikkovedru]

Some models are stopping transcribing the video before the end.

I downloaded 1080p version of this 30min video using yt-dlp. Then I ran whisper on it, using the base.en model. About 45s from the end were not transcribed. It was not feasible for me to test all the rest of the models using the full file.

I cut the small version of the video (the last 41s) and ran models: tiny, tiny.en, base, base.en, small, small.en, medium, large. Of those, all the models worked fine except the large. It did not transcribe everything, even though it stopped in different place than the base.en model with the full file.

This is an .mkv file renamed to .mp4 (because github would not allow uploading Matroska files; just download and run it locally):
https://user-images.githubusercontent.com/2521942/228718376-2a243cc3-819f-4dc7-ad2d-7ef9ddec9b8a.mp4

Can you confirm the existence of bug?

Is there anything that can be done about it? Any volunteers? Sadly I don't have enough expertise to fix the issue.

[ururk]

I ran the large model using whisper.cpp against a 1.25 hour lecture, and about midway the instructor says:

Let's take a break. Five minutes.

The remainder of the transcript was the text [ Break ] 😄 The other models transcribed it OK using both standard whisper and whisper.cpp.

You might want to try using the parameter:

--condition_on_previous_text False

[ururk]

FYI, I tried running your sample, mac, M1, CPU, large model, standard whisper - transcribes fine.

[mikkovedru]

I am also using CPU (Intel Core i7-4790K CPU @ 4.00GHz × 4). Linux Mint 21.1 Cinnamon. Standard whisper.

I tested some more (with the small 41s sample) and I was able to repeat the bug using the whisper --model large --language en --task translate command. Notice, that it is --task translate and not --task transcribe. @ururk , could you please test it once more.