RuntimeError: Requested next frame while there are no more frames left to decode.

[lowdewijk]

🐛 Describe the bug

We have some code that uses a GPU VideoDecoder:

        if self.video_reader is None:
            self.video_reader = VideoDecoder(
                str(self.video),
                device=self.decoding_device,
                num_ffmpeg_threads=1,
                seek_mode="exact",
            )
            assert self.video_reader is not None, "Failed to load video reader"

        for frame_idx in range(0, len(self.video_reader), self.num_frames):
            end_frame = frame_idx + self.num_frames
            if end_frame > len(self.video_reader):
                break

            stacked_frames = self.video_reader.get_frames_in_range(
                frame_idx, end_frame, self.temporal_reduction_factor
            ).data      

This code runs on very large video datasets and until now without problems. However, we added some new data and after 22 hours of running this error was thrown:

Original Traceback (most recent call last):
  File "/home/lodewijk/ml2/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/lodewijk/ml2/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 33, in fetch
    data.append(next(self.dataset_iter))
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lodewijk/ml2/validation/iterable_videos_dataset.py", line 84, in __iter__
    yield from (item for item, _ in self.wrapped_dataset)
  File "/home/lodewijk/ml2/validation/iterable_videos_dataset.py", line 84, in <genexpr>
    yield from (item for item, _ in self.wrapped_dataset)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lodewijk/ml2/validation/iterable_videos_dataset.py", line 139, in __iter__
    for item in dataset:
  File "/home/lodewijk/ml2/validation/iterable_videos_dataset.py", line 279, in __iter__
    stacked_frames = self.video_reader.get_frames_in_range(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lodewijk/ml2/.venv/lib/python3.12/site-packages/torchcodec/decoders/_video_decoder.py", line 245, in get_frames_in_range
    frames = core.get_frames_in_range(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lodewijk/ml2/.venv/lib/python3.12/site-packages/torch/_ops.py", line 723, in __call__
    return self._op(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Requested next frame while there are no more frames left to decode.

In the first place I am surprised that this can happen, as the frame range should be in range of the video.

I looked at the CPP code that throws this error and it seems I can safely catch this error and simply break the for loop, so I will now continue with:

            try:
                frames = self.video_reader.get_frames_in_range(start, end)
            except RuntimeError as e:
                if "no more frames left to decode" in str(e):
                    # EOF for this video
                    break
                raise 

I don't really like this workaround though. Perhaps someone can tell me if I am doing something wrong or how I should handle this.,

Versions

Collecting environment information...
PyTorch version: 2.6.0+cu126
Is debug build: False
CUDA used to build PyTorch: 12.6 ROCM used to build PyTorch: N/A OS: Ubuntu 24.04.2 LTS (x86_64) GCC version: (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 Clang version: 18.1.3 (1ubuntu1)
CMake version: Could not collect
Libc version: glibc-2.39

Python version: 3.12.3 (main, Aug 14 2025, 17:47:21) [GCC 13.3.0] (64-bit runtime)
Python platform: Linux-6.8.0-64-generic-x86_64-with-glibc2.39
Is CUDA available: True
CUDA runtime version: 12.8.93
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090 Ti
Nvidia driver version: 570.158.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_cnn.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_engines_runtime_compiled.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_graph.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_heuristic.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops.so.9.8.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.7
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 24
On-line CPU(s) list: 0-23
Vendor ID: AuthenticAMD
Model name: AMD Ryzen 9 7900X 12-Core Processor
CPU family: 25
Model: 97
Thread(s) per core: 2
Core(s) per socket: 12
Stepping: 2
CPU(s) scaling MHz: 40%
CPU max MHz: 5651.0000
CPU min MHz: 400.0000
BogoMIPS: 9400.62
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local user_shstk avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd cppc amd_ibpb_ret arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic vgif x2avic v_spec_ctrl vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid overflow_recov succor smca fsrm flush_l1d
Virtualization: AMD-V
L1d cache: 384 KiB (12 instances)
L1i cache: 384 KiB (12 instances)
L2 cache: 12 MiB (12 instances)
L3 cache: 64 MiB (2 instances)
NUMA node(s): 1
NUMA node0 CPU(s): 0-23
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Mitigation; Safe RET
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] numpy==2.1.3
[pip3] nvidia-cublas-cu12==12.6.4.1
[pip3] nvidia-cuda-cupti-cu12==12.6.80
[pip3] nvidia-cuda-nvrtc-cu12==12.6.77
[pip3] nvidia-cuda-runtime-cu12==12.6.77
[pip3] nvidia-cudnn-cu12==9.5.1.17
[pip3] nvidia-cufft-cu12==11.3.0.4
[pip3] nvidia-curand-cu12==10.3.7.77
[pip3] nvidia-cusolver-cu12==11.7.1.2
[pip3] nvidia-cusparse-cu12==12.5.4.2
[pip3] nvidia-cusparselt-cu12==0.6.3
[pip3] nvidia-nccl-cu12==2.21.5
[pip3] nvidia-nvjitlink-cu12==12.6.85
[pip3] nvidia-nvtx-cu12==12.6.77
[pip3] torch==2.6.0+cu126
[pip3] torch_tensorrt==2.6.0+cu126
[pip3] torchaudio==2.6.0+cu126
[pip3] torchinfo==1.8.0
[pip3] torchvision==0.21.0+cu126
[pip3] triton==3.2.0
[conda] Could not collect

[scotts]

@lowdewijk, this generally means one of two things:

1. There's a bug in TorchCodec.
2. The video is corrupted in some way - it has invalid metadata, or somehow the data itself has been corrupted.

Can you point to a video which exhibits this behavior? We can try to figure out what's going on. Even in the case of a corrupted video, we do want to try to fail gracefully - although in some instances, a clean runtime error is as good as we can do. (As opposed to a segfault!)

[augustwester]

I'm seeing this issue too. For a specific video file (that I cannot share, unfortunately), I can do the following:

>>> decoder = VideoDecoder("video.mp4", seek_mode="exact")
>>> len(decoder)
187757
>>> frame = decoder.get_frame_at(187756)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/dist-packages/torchcodec/decoders/_video_decoder.py", line 211, in get_frame_at
    data, pts_seconds, duration_seconds = core.get_frame_at_index(
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 829, in __call__
    return self._op(*args, **kwargs)
RuntimeError: Requested next frame while there are no more frames left to decode.

This happens not just for the last frame but for every frame down to (and including) index 187586.

I can decode every frame of the video with ffmpeg using the command ffmpeg -i video.mp4 -vf showinfo -f null -. When I do this, the last line shows:

[Parsed_showinfo_0 @ 0xb74c62f40] n:187756 pts:18775600 pts_time:6258.533333 duration: 100 duration_time:0.0333333 fmt:yuv444p cl:left sar:1/1 s:1024x512 i:P iskey:0 type:P checksum:5683FFC6 plane_checksum:[4770E5D6 E5A7D899 16724148] mean:[130 157 104] stdev:[39.7 19.5 15.5]

Note the n:187756 part which shows the zero-indexed frame numbers matching with the reported length by torch codec.

[scotts]

@augustwester, can you show the output of decoder.metadata?

[augustwester]

@augustwester, can you show the output of decoder.metadata?

VideoStreamMetadata:
  duration_seconds_from_header: 6258.566667
  begin_stream_seconds_from_header: 0.0
  bit_rate: 1941392.0
  codec: h264
  stream_index: 0
  begin_stream_seconds_from_content: 0.0
  end_stream_seconds_from_content: 6258.566667
  width: 1024
  height: 512
  num_frames_from_header: 187757
  num_frames_from_content: 187757
  average_fps_from_header: 30.0
  pixel_aspect_ratio: 1
  duration_seconds: 6258.566667
  begin_stream_seconds: 0.0
  end_stream_seconds: 6258.566667
  num_frames: 187757
  average_fps: 29.99999999840219