Missing sequences in transcription after Whisper update.

[WojciechTyczynski]

After updating Whisper from the release 20230124 to 20230314, I noticed that the small.en and large models have issues with missing segments in transcriptions, mostly at the end or close to the end of the audio (for both transcriptions with and without word-level timestamps). As can be seen below transcription for the newer version is missing a segment between 03:00.000 and 03:18.100. What can cause it and is there any possible solution for this?

Transcriptions example:

• Whisper 20230314 - small.en:

[00:01.440 --> 00:05.980] A superhero of the Avengers fearlessly taking on any fight. [00:06.520 --> 00:10.180] An ordinary man with a strength that never gives up. [00:10.580 --> 00:11.380] Do you remember the pain? [00:11.980 --> 00:14.300] All of it. Yeah, I was awake through every moment. [00:15.140 --> 00:18.320] Who risked his own life to try to save his nephew. [00:18.780 --> 00:22.120] I'd just perfectly see him in a pool of blood coming from his head. [00:22.720 --> 00:25.220] When I ran up to him, you know, I didn't think he was alive. [00:26.080 --> 00:27.600] I'd do it again. [00:28.560 --> 00:29.160] You'd do it again? [00:29.160 --> 00:30.020] Yeah, I'd do it again. [00:30.600 --> 00:32.040] Because it's going right at my nephew. [00:35.600 --> 00:38.160] You have a video which is haunting. [00:38.940 --> 00:41.260] It's January 1st at 8.42. [00:41.900 --> 00:44.220] And you are 13 minutes away. [00:46.200 --> 00:48.400] Someone's been run over by a snow cat. Hurry. [00:48.780 --> 00:49.380] You stay cross. [00:49.720 --> 00:50.820] There's a lot of blood over here. [00:51.040 --> 00:51.880] He is in rough shape. [00:52.540 --> 00:54.200] Keep breathing, man. Keep fighting. [00:54.760 --> 00:55.600] Hang in there, brother. [00:59.160 --> 01:02.000] This is the sound of someone that was dying. [01:03.180 --> 01:06.120] Seven tons of machinery bearing down. [01:06.780 --> 01:10.540] And one man's unwilling to fight and survive. [01:12.320 --> 01:14.400] Eight ribs broken in 14 places. [01:14.940 --> 01:17.100] Right knee, right ankle broken. [01:17.400 --> 01:18.680] Left leg, tibia broken. [01:19.100 --> 01:20.260] The left ankle broken. [01:20.520 --> 01:21.540] Right clavicle broken. [01:21.740 --> 01:22.820] Right shoulder broken. [01:23.140 --> 01:23.960] Face-eyed socket. [01:24.180 --> 01:25.600] The jaw, the mandible broken. [01:26.220 --> 01:27.260] Lung collapsed. [01:27.760 --> 01:30.340] Pierced from the rib bone, your liver. [01:31.780 --> 01:32.860] Which sounds terrifying. [01:33.300 --> 01:33.380] Yeah. [01:34.420 --> 01:36.140] And they're like, what's my body look like? [01:36.260 --> 01:38.760] Am I just going to be like a spine and a brain, [01:38.880 --> 01:39.760] like a science experiment? [01:40.500 --> 01:42.540] His extraordinary fight to live. [01:42.960 --> 01:45.140] And his family's heartache and pain. [01:45.500 --> 01:48.360] Right there, by his side, through it all. [01:48.920 --> 01:51.000] I heard that you had, in sign language, [01:51.000 --> 01:52.900] you said to your family, I'm sorry. [01:58.100 --> 01:58.700] Yeah. [01:58.600 --> 01:59.500] I'm sorry. [02:00.700 --> 02:02.820] A story of terror, survival. [02:03.080 --> 02:04.100] I chose to survive. [02:04.720 --> 02:05.520] You think that killed me? [02:06.200 --> 02:06.500] No way. [02:07.020 --> 02:07.620] And triumph. [02:08.920 --> 02:10.820] Jeremy Renner, Diane Sawyer. [02:11.140 --> 02:12.900] Do you dream of doing those stunts again? [02:14.360 --> 02:17.460] I've lost a lot of flesh and bone in this experience, [02:17.460 --> 02:20.840] but I've been refuelled and refilled with love and titanium. [02:22.580 --> 02:23.540] The exclusive interview. [02:23.920 --> 02:26.960] You look in the mirror and do you see the same face? [02:27.280 --> 02:29.320] No, I see a lucky man. [02:30.600 --> 02:33.700] Thursday, April 6th at 10, 9 central on ABC. [02:36.360 --> 02:39.960] Wow. [02:40.960 --> 02:43.000] Jeremy Renner telling his story for the first time, [02:43.160 --> 02:45.520] Diane Sawyer, since that unbelievable accident. [02:45.840 --> 02:48.100] And it looks like it is going to be such an emotional [02:48.100 --> 02:50.840] and courageous interview and you can see it next Thursday, [02:51.080 --> 02:53.660] April 6th, 10, 9 central on ABC. [02:54.180 --> 02:54.320] Robin? [02:54.480 --> 02:56.020] We'll certainly be tuning in. [02:56.120 --> 02:58.880] His recovery, simply remarkable. [02:59.040 --> 02:59.380] Remarkable. [03:18.100 --> 03:19.380] We'll see you next week on GMA.

• Whisper 20230124 - small.en:

[00:00.000 --> 00:06.000] A superhero of the Avengers fearlessly taking on any fight. [00:06.000 --> 00:10.000] An ordinary man with a strength that never gives up. [00:10.000 --> 00:12.000] Do you remember the pain? [00:12.000 --> 00:15.000] All of it. Yeah, I was awake through every moment. [00:15.000 --> 00:19.000] Who risked his own life to try to save his nephew. [00:19.000 --> 00:22.000] I'd just perfectly see him in a pool of blood coming from his head. [00:22.000 --> 00:26.000] When I ran up to him, you know, I didn't think he was alive. [00:26.000 --> 00:28.000] I'd do it again. [00:28.000 --> 00:29.000] You'd do it again? [00:29.000 --> 00:30.000] Yeah, I'd do it again. [00:30.000 --> 00:32.000] Because it's going right at my nephew. [00:35.000 --> 00:38.000] You have a video which is haunting. [00:38.000 --> 00:41.000] It's January 1st at 8.42. [00:41.000 --> 00:44.000] And you are 13 minutes away. [00:46.000 --> 00:48.000] Someone's been run over by a snow cat. Hurry. [00:48.000 --> 00:49.000] You stay cross. [00:49.000 --> 00:51.000] There's a lot of blood over here. [00:51.000 --> 00:52.000] He is in rough shape. [00:52.000 --> 00:53.000] Oh, oh, oh. [00:53.000 --> 00:54.000] Keep breathing, man. Keep fighting. [00:54.000 --> 00:56.000] Hang in there, brother. [00:56.000 --> 00:58.000] Oh, oh, oh. [00:58.000 --> 01:01.000] This was the sound of someone that was dying. [01:01.000 --> 01:05.000] Seven tons of machinery bearing down. [01:05.000 --> 01:10.000] And one man's unwilling to fight and survive. [01:11.000 --> 01:13.000] Eight ribs broken in 14 places. [01:13.000 --> 01:14.000] Yeah. [01:14.000 --> 01:16.000] Right knee, right ankle broken. [01:16.000 --> 01:18.000] Left leg, tibia broken. [01:18.000 --> 01:19.000] The left ankle broken. [01:19.000 --> 01:21.000] Right clavicle broken. [01:21.000 --> 01:22.000] Right shoulder broken. [01:22.000 --> 01:23.000] Face-eyed socket. [01:23.000 --> 01:25.000] The jaw, the mandible broken. [01:25.000 --> 01:27.000] Lung collapsed. [01:27.000 --> 01:30.000] Pierced from the rib bone, your liver. [01:30.000 --> 01:33.000] Which sounds terrifying. [01:33.000 --> 01:34.000] Yeah. [01:34.000 --> 01:36.000] And they're like, what's my body look like? [01:36.000 --> 01:40.000] Am I just going to be like a spine and a brain, like a science experiment? [01:40.000 --> 01:43.000] His extraordinary fight to live. [01:43.000 --> 01:45.000] And his family's heartache and pain. [01:45.000 --> 01:48.000] Right there, by his side, through it all. [01:48.000 --> 01:53.000] I heard that you had, in sign language, you said to your family, I'm sorry. [01:53.000 --> 01:54.000] Yeah. [01:54.000 --> 01:55.000] Sorry. [01:55.000 --> 01:58.000] A story of terror, survival. [01:58.000 --> 01:59.000] I chose to survive. [01:59.000 --> 02:00.000] You think that killed me? [02:00.000 --> 02:01.000] No way. [02:01.000 --> 02:02.000] And triumph. [02:02.000 --> 02:05.000] Jeremy Renner, Diane Sawyer. [02:05.000 --> 02:08.000] Do you dream of doing those stunts again? [02:08.000 --> 02:13.000] I've lost a lot of flesh and bone in this experience, but I've been refueled and refilled [02:13.000 --> 02:15.000] with love and titanium. [02:15.000 --> 02:22.000] The exclusive of the story of the story of the story of the story of the story of the [02:22.000 --> 02:23.000] exclusive interview. [02:23.000 --> 02:27.000] You look in the mirror and do you see the same face? [02:27.000 --> 02:30.000] No, I see a lucky man. [02:30.000 --> 02:36.000] Thursday, April 6th at 10, 9 central on ABC. [02:36.000 --> 02:38.000] Wow. [02:38.000 --> 02:45.000] Jeremy Renner telling his story for the first time to Diane Sawyer since that unbelievable [02:45.000 --> 02:46.000] accident. [02:46.000 --> 02:49.000] And it looks like it is going to be such an emotional and courageous interview and you [02:49.000 --> 02:54.000] can see it next Thursday, April 6th, 10, 9 central on ABC. [02:54.000 --> 02:55.000] Robin. [02:55.000 --> 02:56.000] We'll certainly be tuning in. [02:56.000 --> 02:59.000] His recovery, simply remarkable. [02:59.000 --> 03:00.000] Remarkable. [03:00.000 --> 03:01.000] Well, hey there, GMA fans. [03:01.000 --> 03:02.000] Robin Roberts here. [03:02.000 --> 03:05.000] Thanks for checking out our YouTube channel. [03:05.000 --> 03:07.000] Lots of great stuff here. [03:07.000 --> 03:12.000] So go on, click the subscribe button right over here to get more of awesome videos and [03:12.000 --> 03:15.000] content from GMA every day, anytime. [03:15.000 --> 03:19.000] We thank you for watching and we'll see you in the morning on GMA.

Source audio: Jeremy Renner to open up in exclusive interview with Diane Sawyer l GMA

[ryanheise]

I have encountered something similar before. The cause was a delayed timestamp for the end of the previous window which caused it to start the next window too late and miss words.

There may be different types of scenarios that can cause it, but one such scenario is addressed in #1114 when word timestamps are enabled. You can try it to see if it catches your case.

[WojciechTyczynski]

I ran Whisper with the changes from your PR; unfortunately, it doesn't catch my case, and parts of the transcript are still missing.

[Harleyzheng]

Bumping this one up. I frequently encounter this issue as well.

[openSourcerer9000]

Makes an otherwise perfect transcriber pretty useless : (. medium.en on local here