NVIDIA Video Codec SDK unreliability with variable bitrate (reconfigure encoder)

[yuv418]

When trying to update the encoder and change the bitrate based on congestion information, we get an "invalid version." (NV_ENC_ERR_INVALID_VERSION).

This used to not happen when we used Video Codec SDK 12.0. On 13.0 and 12.1.x, this persists.

Analysis:

• Call NvEncReconfigureEncoder
• NvEncReconfigureEncoder checks versioning info for NV_ENC_RECONFIGURE_PARAMS
• A function is called from here, which checks NV_ENC_INTIALIZE_PARAMS's version.
• A function is called from here, which checks NV_ENC_CONFIG's version. This version is wrong sometimes, leading to a failure.

The issue seems to be NV_ENC_CONFIG, which seems to have a different version number on every run. Very weird; this is almost certainly the issue.

[yuv418]

It looks like the NV_ENC_CONFIG data is somehow overwritten or modified during the course of execution, potentially from another CUDA-related or encoder-related thread... benefits of using unsafe in Rust... Nonetheless, it's not necessarily clear from the documentation that internals modify NV_ENC_CONFIG.

We could/should figure out what and why it's being modified/.

[yuv418]

This is somewhat solved now by incessantly cloning buffers. However, there's a potential chance this is an upstream bug (or at least a documentation issue), since it's not clear NVENC goes and modifies the NV_ENC_INITIALIZE_PARAMS or NV_ENC_CONFIG or whatever if we pass a pointer to it into the library. Perhaps this is related to using a preset config.

Maybe worth writing an MVP in C++ and submitting a bug report if it persists.