Packaging Regression in v2.9.0: torchaudio.load fails on non-shared Python builds

[Manamama]

Summary

A packaging change in torch==2.9.0 and torchaudio==2.9.0 has introduced a regression that causes torchaudio.load() to fail on Linux systems where Python was not built with the --enable-shared flag. This is a very common configuration for developers who install Python from sources like the deadsnakes PPA on Ubuntu.

The failure occurs because the new torchcodec backend, now a default dependency, is linked against libpythonX.Y.so. When this shared library is not found in the systems standard paths, the program crashes with a RuntimeError. This breaks years of expected behavior where torchaudio worked reliably on these systems.

This violates the portability principles of the manylinux wheel standard.

Bug Description and Reproduction

Working Configuration:

• torch==2.8.0+cpu
• torchaudio==2.8.0+cpu
• torchcodec is NOT installed.
• torchaudio.load() works perfectly, correctly using the sox backend.

Failing Configuration:

• torch==2.9.0+cpu
• torchaudio==2.9.0+cpu
• torchcodec==0.8.0 (installed as a dependency)
• torchaudio.load() fails.

Error Log:

RuntimeError: Could not load libtorchcodec. ...

Diagnosis: The Root Cause

The error message is misleading. The true cause was diagnosed by running ldd on the .so files:

ldd /home/user/.local/lib/python3.10/site-packages/torchcodec/libtorchcodec_custom_ops7.so

This reveals the critical missing dependency:

libpython3.10.so.1.0 => not found

The torchcodec binary wheel for v0.8.0 is built in an environment that links it against libpython.so. This breaks compatibility with any Python installation that does not provide this shared library, which is common in non-conda, PPA-based, or manually compiled Python environments on Linux.

The older torchaudio==2.8.0 works because its compiled libraries do not link against libpython.so, respecting the manylinux standard and ensuring portability.

Steps to Reproduce

1. Environment: Use Ubuntu 20.04 (or a derivative like Zorin OS 16.3) where the system Python is 3.8.
2. Install Python 3.10 from the deadsnakes PPA: sudo add-apt-repository ppa:deadsnakes/ppa && sudo apt-get update && sudo apt-get install python3.10.
3. Install the failing versions: pip install torch==2.9.0 torchaudio==2.9.0 torchcodec --index-url https://download.pytorch.org/whl/cpu.
4. Execute a simple script: import torchaudio; torchaudio.load("some_audio.wav").
5. Observe the RuntimeError.
6. To confirm the fix, uninstall the packages and install the working versions: pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cpu. (Ensure torchcodec is uninstalled).
7. Run the same script and observe that it succeeds.

Proposed Solution

This is a packaging regression that breaks a significant user workflow. The ideal solution is for the official torchcodec wheels to be built in a way that does not link against libpython.so, restoring the portability that is standard for the manylinux ecosystem.

Alternatively, if this is an intentional design choice, the failure should be more graceful, with a clear ImportError explaining that a Python installation with shared library support is now a requirement.

Ref:

  File "/home/usern/.local/lib/python3.10/site-packages/torchaudio/__init__.py", line 86, in load
    return load_with_torchcodec(
  File "/home/usern/.local/lib/python3.10/site-packages/torchaudio/_torchcodec.py", line 82, in load_with_torchcodec
    from torchcodec.decoders import AudioDecoder
  File "/home/usern/.local/lib/python3.10/site-packages/torchcodec/__init__.py", line 10, in <module>
    from . import decoders, samplers  # noqa
  File "/home/usern/.local/lib/python3.10/site-packages/torchcodec/decoders/__init__.py", line 7, in <module>
    from .._core import AudioStreamMetadata, VideoStreamMetadata
  File "/home/usern/.local/lib/python3.10/site-packages/torchcodec/_core/__init__.py", line 8, in <module>
    from ._metadata import (
  File "/home/usern/.local/lib/python3.10/site-packages/torchcodec/_core/_metadata.py", line 16, in <module>
    from torchcodec._core.ops import (
  File "/home/usern/.local/lib/python3.10/site-packages/torchcodec/_core/ops.py", line 84, in <module>
    load_torchcodec_shared_libraries()
  File "/home/usern/.local/lib/python3.10/site-packages/torchcodec/_core/ops.py", line 69, in load_torchcodec_shared_libraries
    raise RuntimeError(
RuntimeError: Could not load libtorchcodec. Likely causes:
          1. FFmpeg is not properly installed in your environment. We support
             versions 4, 5, 6 and 7.
          2. The PyTorch version (2.9.0+cpu) is not compatible with
             this version of TorchCodec. Refer to the version compatibility
             table:
             https://github.com/pytorch/torchcodec?tab=readme-ov-file#installing-torchcodec.
          3. Another runtime dependency; see exceptions below.
        The following exceptions were raised as we tried to load libtorchcodec:
        
[start of libtorchcodec loading traceback]
FFmpeg version 8: Could not load this library: /home/usern/.local/lib/python3.10/site-packages/torchcodec/libtorchcodec_core8.so
FFmpeg version 7: Could not load this library: /home/usern/.local/lib/python3.10/site-packages/torchcodec/libtorchcodec_custom_ops7.so
FFmpeg version 6: Could not load this library: /home/usern/.local/lib/python3.10/site-packages/torchcodec/libtorchcodec_core6.so
FFmpeg version 5: Could not load this library: /home/usern/.local/lib/python3.10/site-packages/torchcodec/libtorchcodec_core5.so
FFmpeg version 4: Could not load this library: /home/usern/.local/lib/python3.10/site-packages/torchcodec/libtorchcodec_custom_ops4.so
[end of libtorchcodec loading traceback].
Command exited with non-zero status 1

only if:

pip show torch torchaudio
Name: torch
Version: 2.9.0+cpu

works on:

pip show torch torchaudio
Name: torch
Version: 2.8.0+cpu

and older ones. Force reinstalling torch* modules and even apt reinstalling some -dev stuff does not help with the missing .so files.

[Manamama]

Of course, it is also due to: #3958 (comment) , torchcodec being used now

[Manamama]

Ok, the mystery (only partially) solved:

 ldd /home/user/.local/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so
	linux-vdso.so.1 (0x00007fffbe65d000)
	libtorch.so => not found
	libtorch_cpu.so => not found
	libc10.so => not found
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fa13d208000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa13cf9a000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa13ce4b000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa13ce24000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa13cc32000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa13d32d000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa13cc2c000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa13cc09000)
user@user-HP-Pavilion-x360-Convertible:~/Downloads/GitHub/Ubuntu_Scripts_1$ locate /libtorch.so
/home/user/.local/lib/python3.10/site-packages/torch/lib/libtorch.so
user@user-HP-Pavilion-x360-Convertible:~/Downloads/GitHub/Ubuntu_Scripts_1$ locate /libtorch_cpu.so
/home/user/.local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
user@user-HP-Pavilion-x360-Convertible:~/Downloads/GitHub/Ubuntu_Scripts_1$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.local/lib/python3.10/site-packages/torch/lib/
user@user-HP-Pavilion-x360-Convertible:~/Downloads/GitHub/Ubuntu_Scripts_1$ ldd /home/user/.local/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so
	linux-vdso.so.1 (0x00007ffd42392000)
	libtorch.so => /home/user/.local/lib/python3.10/site-packages/torch/lib/libtorch.so (0x00007f9eda3c1000)
	libtorch_cpu.so => /home/user/.local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so (0x00007f9ec5ed2000)
	libc10.so => /home/user/.local/lib/python3.10/site-packages/torch/lib/libc10.so (0x00007f9ec5dc3000)
	libgomp.so.1 => /home/user/.local/lib/python3.10/site-packages/torch/lib/libgomp.so.1 (0x00007f9ec5b87000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9ec58f1000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9ec57a0000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9ec577b000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9ec5589000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9eda470000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9ec557f000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9ec5579000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9ec5556000)

so one must export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.local/lib/python3.10/site-packages/torch/lib/ , then some pip install -U multiprocess, then pip install -U xformers (sic) and hope for the best with e.g. whisperx ...

[Manamama]

Note to self mostly, the fastest and maybe the most elegant is to indeed recompile with:

wget https://www.python.org/ftp/python/3.10.14/Python-3.10.14.tar.xz
tar -xf Python-3.10.14.tar.xz
cd Python-3.10.14
./configure --enable-shared --prefix=/usr/local
make -j$(nproc)
sudo make install

as it solves most or all of the missing shared .so files problem...

[pearu]

The failure occurs because the new torchcodec backend, now a default dependency, is linked against libpythonX.Y.so.

Can you raise this issue in torchcodec?