Expose cpu_fallback in VideoDecoder API #1093

mollyxu · 2025-12-03T20:41:21Z

Expose cpu_fallback in VideoDecoder API

Expose the cpu_fallback property in the public API to allow users to check whether the video decoder fell back to CPU decoding.

The FallbackInfo class provides a clean interface for users to check decoder fallback status:

bool(fallback_info) returns True if any fallback occurred
str(fallback_info) provides a human-readable explanation of why fallback happened
Tracks two types of fallback: NVcuvid not found and unsupported video formats

The cpu_fallback property on VideoDecoder returns a FallbackInfo instance that can be queried after decoding at least one frame. This helps users understand why GPU decoding may not be used and debug performance issues.

mollyxu · 2025-12-04T04:47:47Z

src/torchcodec/decoders/_video_decoder.py

        )

+        self._fallback_info = FallbackInfo()
+        self._has_decoded_frame = False


Instead of tracking whether a frame has been decoded, we could also just call _update_cpu_fallback inside every method that would decode a frame. The problem with that approach would be that we would be returning non tensors in the compiled methods. As such, we would have to do something like inside

if torch.compiler.is_compiling(): return

I think we should implement this _has_decoded_frame logic in C++ for 2 reasons:

it's simpler and can be localized to a single place (see suggested implementation below)

it's only relevant for the default interface, not for the beta interface.

So maybe all we need is to update the default interface's returned string, and let it indicate whether its status is known or unknown yet. Here:

torchcodec/src/torchcodec/_core/CudaDeviceInterface.cpp

Lines 357 to 363 in 38fa96c

std::string CudaDeviceInterface::getDetails() {

// Note: for this interface specifically the fallback is only known after a

// frame has been decoded, not before: that's when FFmpeg decides to fallback,

// so we can't know earlier.

return std::string("FFmpeg CUDA Device Interface. Using ") +

(usingCPUFallback_ ? "CPU fallback." : "NVDEC.");

}

To know whether a frame has been decoded yet, I think we can simply set a boolean field to true when

torchcodec/src/torchcodec/_core/CudaDeviceInterface.cpp

Line 238 in 38fa96c

void CudaDeviceInterface::convertAVFrameToFrameOutput(

is called. This would be a new private boolean attribute on the CudaDeviceInterface class.

NicolasHug

Thanks for working on this @mollyxu ! Left a few comments and suggestions regarding the implementation, let me know if I can clarify anything.

NicolasHug · 2025-12-04T10:49:36Z

src/torchcodec/decoders/_video_decoder.py

+
+
+@dataclass
+class FallbackInfo:


We could keep this class private for now, but I was thinking, maybe it could be public and named CpuFallbackStatus? I'm hoping that it might resolve both @scotts preference and mine:

The CpuFallbackStatus class will be public and visible from the docs, and it will be clear from the docstring that the dec.cpu_fallback attribute is an instance of this class. The attribute name cpu_fallback is fairly generic and allows us to extend the functionality in the future, while the class name itself communicates that it it's not just a simple bool, or a simple string.

NicolasHug · 2025-12-04T10:53:15Z

src/torchcodec/decoders/_video_decoder.py

+        self.__nvcuvid_unavailable = False
+        self.__video_not_supported = False


Using double underscores works, it's the more "hardcore" version to indicate that something is private. (ref). We don't use double underscores in TC, we've mostly just used single underscore up to now. For consistency, I think it's best to stick to our current practice of using single undercores.

EDIT: see my other comment below, the name mangling is a bit surprising, so let's definitely use single underscores

src/torchcodec/decoders/_video_decoder.py

NicolasHug · 2025-12-04T10:58:37Z

src/torchcodec/decoders/_video_decoder.py

+        self._update_cpu_fallback()
+        return self._fallback_info


2 nits:

No need to define a separate _update_cpu_fallback() function, it can just be inlined here

The convention (I think it's a convention??) is to use the same name for the @property and for the underlying cached object. That is, I think self._fallback_info should just be self._cpu_fallback. It makes it more obvious that it relates to the @cpu_fallback property.

NicolasHug · 2025-12-04T11:00:33Z

src/torchcodec/decoders/_video_decoder.py

+
+            if "CPU fallback" in backend_details:
+                if "NVCUVID not available" in backend_details:
+                    self._fallback_info._FallbackInfo__nvcuvid_unavailable = True


Ah, so this _FallbackInfo__nvcuvid_unavailable is the name-mangling consequence of using double leading underscore. Let's definitely use single underscores :)

NicolasHug · 2025-12-04T11:07:18Z

src/torchcodec/decoders/_video_decoder.py

        )

+        self._fallback_info = FallbackInfo()
+        self._has_decoded_frame = False


I think we should implement this _has_decoded_frame logic in C++ for 2 reasons:

it's simpler and can be localized to a single place (see suggested implementation below)

it's only relevant for the default interface, not for the beta interface.

So maybe all we need is to update the default interface's returned string, and let it indicate whether its status is known or unknown yet. Here:

torchcodec/src/torchcodec/_core/CudaDeviceInterface.cpp

Lines 357 to 363 in 38fa96c

std::string CudaDeviceInterface::getDetails() {

// Note: for this interface specifically the fallback is only known after a

// frame has been decoded, not before: that's when FFmpeg decides to fallback,

// so we can't know earlier.

return std::string("FFmpeg CUDA Device Interface. Using ") +

(usingCPUFallback_ ? "CPU fallback." : "NVDEC.");

}

To know whether a frame has been decoded yet, I think we can simply set a boolean field to true when

torchcodec/src/torchcodec/_core/CudaDeviceInterface.cpp

Line 238 in 38fa96c

void CudaDeviceInterface::convertAVFrameToFrameOutput(

is called. This would be a new private boolean attribute on the CudaDeviceInterface class.

NicolasHug · 2025-12-04T11:11:52Z

test/test_decoders.py

+
+        _ = decoder.get_frame_at(0)
+
+        assert decoder.cpu_fallback.status_known


For the beta interface, we should be able to assert that status_known is true before we even decode any frame. I think you might need some slight modification to the implementation above in order to achieve that.

ping on this, we seem to not have tests for the beta interface now?

test/test_decoders.py

NicolasHug · 2025-12-05T10:15:19Z

src/torchcodec/decoders/_video_decoder.py

+        - Use ``bool(cpu_fallback_status)`` to check if any fallback occurred
+
+    Attributes:
+        status_known (bool): Whether the fallback status has been determined.


I think it's OK to expose publicly. Let's just document that:

for the Beta CUDA backend this is always known

for the ffmpeg one, it's known after decoding the first frame.

We can link to this concept of CUDA backend by linking to https://meta-pytorch.org/torchcodec/stable/generated/torchcodec.decoders.set_cuda_backend.html#torchcodec.decoders.set_cuda_backend

Also, we'll probably want to document this class publicly in docs/source/api_ref_decoders.rst. We should indicate that users should never instantiate this class directly, and only accessed via the VideoDecoder.cpu_fallback attribute.

NicolasHug · 2025-12-05T10:28:46Z

test/test_decoders.py

+
+        assert "FFmpeg CUDA" in str(ref_dec.cpu_fallback)
+        assert ref_dec.cpu_fallback.status_known
+        assert bool(ref_dec.cpu_fallback)


Here and everywhere else, we don't need to explicitly call bool(). The Pythonic way is to rely on the fact that bool() is essentially called automatically in contexts where it's needed (it's not necessarily how it's implemented in CPython but the idea is the same). This includes:

assert cond

if cond

etc.

For example, to check whether a list is empty, the Pythonic way is to just check if l: .... We don't do if bool(l):

NicolasHug · 2025-12-05T10:30:50Z

test/test_decoders.py

+        _ = dec.get_frame_at(0)
+        assert "FFmpeg CUDA" in str(dec.cpu_fallback)


2 comments:

Syntax: we don't need to use _ = .... Just call a raw dec.get_frame_at(0)

We shouldn't need to decode a frame here, because we should already be able to tell that it's the FFmpeg backend. We don't know whether there's a fallback, but that's not needed info here.

test/test_decoders.py

NicolasHug · 2025-12-05T10:32:12Z

test/test_decoders.py

+
+        _ = decoder[0]
+
+        assert decoder.cpu_fallback.status_known


In this scenario, we should be able to assert that the status is known before we decode any frame?

NicolasHug · 2025-12-05T10:33:29Z

test/test_decoders.py

+
+        _ = decoder.get_frame_at(0)
+
+        assert decoder.cpu_fallback.status_known


ping on this, we seem to not have tests for the beta interface now?

NicolasHug · 2025-12-05T10:34:11Z

test/test_decoders.py

+        assert "Fallback status: Falling back due to:" in str(decoder.cpu_fallback)
+
+    @needs_cuda
+    def test_cpu_fallback_no_fallback_on_supported_video(self):


let's parametrize this over both interfaces - the beta and ffpmeg ones

NicolasHug · 2025-12-05T10:35:38Z

test/test_decoders.py

+        assert not bool(decoder.cpu_fallback)
+        assert "No fallback required" in str(decoder.cpu_fallback)
+
+    def test_cpu_fallback_status_cached(self):


IIUC this test mostly tests that the output value doesn't change, not that it's cached. I think testing the cache behavior is potentially really difficult, and perhaps not needed after all. I'd suggest to remove it?

NicolasHug · 2025-12-05T10:38:37Z

test/test_decoders.py

+
+        assert first_status == second_status
+
+    def test_cpu_fallback_multiple_access_methods(self):


I think this test is technically a subset of the previous one: if we 'cache' the cpu_fallback result, then a consequence is that calling different methods isn't going to change it.

Maybe want you wanted to check is that the status becomes "known" and that it works with multiple decoding method? If that's the case I think we'd need to re-create the VideoDecoder object in-between method calls. But TBH, I'm not sure it's a critical test to have, so I might suggest to remove it too.

NicolasHug · 2025-12-05T10:47:26Z

src/torchcodec/decoders/_video_decoder.py

+        # We can only determine whether fallback to CPU is happening when this
+        # property is accessed and requires that at least one frame has been decoded.


I think this comment is slightly misleading because it's only really true for the ffmpeg interface. Can I suggest the following - and also please check me on my understanding here:

We only query the CPU fallback info if status is unknown. That happens either when:

this @Property has never been called before

no frame has been decoded yet on the FFmpeg interface.

Note that for the beta interface, we're able to know the fallback status right when the VideoDecoder
is instantiated, but the status_known attribute is initialized to False.

expose cpu_fallback

b32e6f3

meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Dec 3, 2025

Molly Xu added 2 commits December 3, 2025 12:53

modify comments

cf5b718

modify comments

6e69c8c

mollyxu commented Dec 4, 2025

View reviewed changes

mollyxu marked this pull request as ready for review December 4, 2025 04:49

mollyxu changed the title ~~expose cpu_fallback~~ Expose cpu_fallback Dec 4, 2025

mollyxu changed the title ~~Expose cpu_fallback~~ Expose cpu_fallback property in VideoDecoder API Dec 4, 2025

mollyxu changed the title ~~Expose cpu_fallback property in VideoDecoder API~~ Expose cpu_fallback in VideoDecoder API Dec 4, 2025

NicolasHug reviewed Dec 4, 2025

View reviewed changes

Molly Xu and others added 3 commits December 4, 2025 11:03

address feedback:

5ac8321

switch _.code._get_backend_details() to new api

e97490e

Merge branch 'meta-pytorch:main' into cpu-fallback

f353758

NicolasHug reviewed Dec 5, 2025

View reviewed changes

	std::string CudaDeviceInterface::getDetails() {
	// Note: for this interface specifically the fallback is only known after a
	// frame has been decoded, not before: that's when FFmpeg decides to fallback,
	// so we can't know earlier.
	return std::string("FFmpeg CUDA Device Interface. Using ") +
	(usingCPUFallback_ ? "CPU fallback." : "NVDEC.");
	}

		self.__nvcuvid_unavailable = False
		self.__video_not_supported = False


		_ = decoder.get_frame_at(0)

		assert decoder.cpu_fallback.status_known

		_ = dec.get_frame_at(0)
		assert "FFmpeg CUDA" in str(dec.cpu_fallback)


		assert first_status == second_status

		def test_cpu_fallback_multiple_access_methods(self):

		# We can only determine whether fallback to CPU is happening when this
		# property is accessed and requires that at least one frame has been decoded.

Expose cpu_fallback in VideoDecoder API #1093

Are you sure you want to change the base?

Expose cpu_fallback in VideoDecoder API #1093

Conversation

mollyxu commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!