test: add cppapi test to give TorchCodecConfig.cmake a try

[dvrogozh]

That's a test for the TorchCodecConfig.cmake we've introduced in dc87228. Test is integrated into TorchCodec pytest suite. It assumes that ffmpeg is installed and can be used for both runtime and compilation (i.e. .pc and include files are available). Test builds an executable which is linked to TorchCodec core library. There are 2 kind of tests executed:

1. Building with ffmpeg being found via pkg-config
2. Building with ffmpeg being pointed to by TORCHCODEC_FFMPEGn_INSTALL_PREFIX environment variable

Test which relies on environment variable assumes that the ffmpeg install layout will match expectations hardcoded in the test and in the TorchCodecConfig.cmake. It potentially might fail - we need to check in the CI.

@NicolasHug, please, help to review paying attention on the above. I hope this will allow us to have test naturally integrated into TorchCodec test suite. If not, let's discuss what we need to modify.

CC: @NicolasHug

[dvrogozh]

Thank you for triggering CI. Ok, we have the whole bunch of errors which are circling around the pattern that we need not only runtime, but development environment as well, i.e. we need pkg-config, CUDA dev. files and ffmpeg dev. files available. Depending on the workflow we get somewhat different errors:

1. linux_cuda_wheel.yaml - we are missing CUDA which causes failure to import torch target in cmake:

CMake Error at /__w/_temp/conda_environment_19876943476/lib/python3.10/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:90 (message):
  Your installed Caffe2 version uses CUDA but I cannot find the CUDA

2. linux_wheel.yaml and macos_wheel.yaml - we are missing ffmpeg dev. files (are they available with conda installed ffmpeg?)

CMake Error at /usr/local/share/cmake-3.31/Modules/FindPkgConfig.cmake:645 (message):
  The following required packages were not found:
   - libavdevice

3. windows_wheel.yaml - we are missing pkg-config

CMake Warning at C:/Miniconda3/envs/test/Lib/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.

Across these errors the most worrisome seems to be missing CUDA. @NicolasHug, is that reasonable to request having CUDA fully installed on the runner environment for these tests or we better change a strategy? For example, we can consider to avoid integrating this test into regular torchcodec pytest suite and run it separately within cpp_tests.yaml which is less demanding in terms of pre-installed environment?

[dvrogozh]

is that reasonable to request having CUDA fully installed on the runner environment

Actually, it's installed, but was not detected. I guess that due to OS differences in cmake behavior. On ubuntu it get's detected from /usr/local, on AlmaLinux it seems not:

$ docker run -it --rm pytorch/manylinux2_28-builder:cuda12.6 ls /usr/local/cuda/include
bootstrap_device_host        cudaTypedefs.h                 cusparseLt.h                              nv_decode.h
builtin_types.h              cudaVDPAU.h                    cusparse_v2.h                             nvFatbin.h

[NicolasHug]

Thanks for the PR @dvrogozh ! I have a few questions / comments:

• Do we want all this to work when users are building torchcodec from source, or do we want it to work when they install a pre-built version of torchcodec (e.g. via pip install torchcodec ...) ? I assume we want the latter?
• Would it be acceptable to only support Linux for now? I feel like Windows and MacOS support might be lower priority for the feature, and focusing on linux could make it easier.

we need not only runtime, but development environment as well, i.e. we need pkg-config, CUDA dev. files and ffmpeg dev. files available.

Right now, the newly added test are ran under this install-and-test job:

torchcodec/.github/workflows/linux_wheel.yaml

Line 60 in 38fa96c

install-and-test:

. The point of this job is to have a minimal environment that looks like what an end-user would have (typically without development dependencies!). If we need dev stuff, it's best to run the new test in a separate job. That is, a new job, similar in nature to install-and-test, that still depends on the build job above, but where we can install the required dev dependencies safely. This new job doesn't even need to rely on pytest, we could have some inline bash commands directly calling into cmake, which might be simpler too? Does that make sense?