Support FFmpeg 8

[scotts]

🚀 The feature

FFmpeg 8 is out today! Scroll down to the releases section: https://www.ffmpeg.org/download.html

We should add support. We will need to:

1. See if we can compile our existing C++ code against FFmpeg 8. If we can't, we'll need to adapt our code to any changed APIs.
2. Create new binaries to compile against in CI.
3. Add 8 to our CI.
4. Make sure all tests pass.

[pearu]

Just FYI: torchcodec builds fine with ffmpeg8 after applying

diff --git a/src/torchcodec/_core/CMakeLists.txt b/src/torchcodec/_core/CMakeLists.txt
index 1f09159..360799a 100644
--- a/src/torchcodec/_core/CMakeLists.txt
+++ b/src/torchcodec/_core/CMakeLists.txt
@@ -271,7 +271,9 @@ else()
     elseif (${libavcodec_major_version} STREQUAL "60")
         set(ffmpeg_major_version "6")
     elseif (${libavcodec_major_version} STREQUAL "61")
-        set(ffmpeg_major_version "7")
+      set(ffmpeg_major_version "7")
+    elseif (${libavcodec_major_version} STREQUAL "62")
+        set(ffmpeg_major_version "8")
     else()
         message(
             FATAL_ERROR
diff --git a/src/torchcodec/_core/ops.py b/src/torchcodec/_core/ops.py
index 807ed26..148054b 100644
--- a/src/torchcodec/_core/ops.py
+++ b/src/torchcodec/_core/ops.py
@@ -41,7 +41,7 @@ def load_torchcodec_shared_libraries():
     #      libraries do not meet those conditions.
 
     exceptions = []
-    for ffmpeg_major_version in (7, 6, 5, 4):
+    for ffmpeg_major_version in (8, 7, 6, 5, 4):
         pybind_ops_module_name = _get_pybind_ops_module_name(ffmpeg_major_version)
         decoder_library_name = f"libtorchcodec_core{ffmpeg_major_version}"
         custom_ops_library_name = f"libtorchcodec_custom_ops{ffmpeg_major_version}"

and using -Wno-error=deprecated-declarations.

There are new failures in testing:

========================================================================================= short test summary info ==========================================================================================
FAILED test/test_decoders.py::TestVideoDecoder::test_get_frame_at_av1[cuda] - RuntimeError: Could not push packet to decoder: Function not implemented
FAILED test/test_decoders.py::TestVideoDecoder::test_10bit_gpu_fallsback_to_cpu - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_decoders.py::TestVideoDecoder::test_10bit_videos_cpu[asset0] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_decoders.py::TestVideoDecoder::test_10bit_videos_cpu[asset1] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_decoders.py::TestAudioDecoder::test_metadata[asset1] - assert 13.056 == 13.248 ± 1.3e-05
FAILED test/test_metadata.py::test_get_metadata_audio_file[get_container_metadata_from_header] - assert 13.056 == 13.248
FAILED test/test_metadata.py::test_get_metadata_audio_file[metadata_getter1] - assert 13.056 == 13.248
FAILED test/test_metadata.py::test_repr - AssertionError: assert 'AudioStreamM...ormat: fltp\n' == 'AudioStreamM...ormat: fltp\n'
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library[filtergraph] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_scaling[input_video0-1.31-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_scaling[input_video0-0.71-0.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_scaling[input_video0-1.31-0.7] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_scaling[input_video0-0.71-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_scaling[input_video0-1.0-1.0] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_dimension_order[filtergraph-NHWC] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_dimension_order[filtergraph-NCHW] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-30-1.31-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-30-0.71-0.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-30-1.31-0.7] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-30-0.71-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-30-1.0-1.0] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-32-1.31-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-32-0.71-0.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-32-1.31-0.7] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-32-0.71-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-32-1.0-1.0] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-300-1.31-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-300-0.71-0.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-300-1.31-0.7] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-300-0.71-1.5] - RuntimeError: Failed to set output pixel formats: Invalid argument
FAILED test/test_ops.py::TestVideoDecoderOps::test_color_conversion_library_with_generated_videos[128-300-1.0-1.0] - RuntimeError: Failed to set output pixel formats: Invalid argument
================================================================== 31 failed, 1027 passed, 20 skipped, 1 deselected in 152.12s (0:02:32) ===================================================================

Compare this to when using ffmpeg 7:

========================================================================================= short test summary info ==========================================================================================
FAILED test/test_decoders.py::TestVideoDecoder::test_get_frame_at_av1[cuda] - RuntimeError: Could not push packet to decoder: Function not implemented
=================================================================== 1 failed, 1057 passed, 20 skipped, 1 deselected in 143.96s (0:02:23) ===================================================================

[Ryan5453]

This would be great, Homebrew is now installing FFmpeg v8 by default on macOS

[lfos]

This is now also one of the last two remaining packages blocking the Arch Linux ffmpeg 8 rebuild [1]. Would be great to have this fixed.

Looking at the test failures, it looks like a few additional tweaks to the code might be required to get things compatible.

[1] https://archlinux.org/todo/ffmpeg-80-and-friends/

[lfos]

@scotts - Any chance somebody can take a look at the two issues with ffmpeg 8 (invalid pixel format and duration mismatch as can be seed in the test failures added by @pearu above)?

[NicolasHug]

@lfos we'll be working on enabling FFmpeg8 support in the next few weeks. These tests are safe to skip and ignore if they're blocking any downstream release.

[Dan-Flores]

This is completed in #935, #947. Support for FFmpeg8 on Windows will be tracked in #951, so closing this issue.