RPATH Fix for portable_lib Python Extension

[GregoryComer]

Note: This is an attempt to cherry-pick Mergen's RPATH fix from #13254 onto main. Fixes #14421. Original description below.

Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from
the CI build environment.

Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib
Referenced from:
.../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried:
'/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file)

Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for
runtime library resolution.

Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669):

• macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location
• Linux: Uses $ORIGIN/../../../torch/lib for the same purpose
• Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency

Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue
when using ExecutorTorch wheels built on CI.

Note: The same fix may be needed for _training_lib if it experiences similar issues.

Test Plan:

  # Build the wheel locally
  python setup.py bdist_wheel

  # create fresh conda env
  conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11

  # install
  pip install ./dist/executorch-*.whl

  # Verify
  python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')"

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14422

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Unrelated Failure

As of commit 446d855 with merge base 0f22062 ():

NEW FAILURES - The following jobs have failed:

• pull / test-samsung-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 716bdad425ec555c786ecbd46b649d853ca785b99ea28d6667520be2c611eff3 /exec failed with exit code 1
• trunk / unittest-release / macos / macos-job (gh)
  backends/xnnpack/test/ops/test_conv1d.py::TestConv1d::test_qs8_conv1d_batchnorm_seq

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• trunk / unittest-release / linux / linux-job (gh) (trunk failure)
  [ FAILED ] LoggingTest.Utf8Truncation

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[abhinaykukkadapu]

@GregoryComer Do you think this will work for windows?

[GregoryComer]

That's a good question. I was planning to test the windows wheel on a local machine. I'll see if we run into any issues.

[GregoryComer]

Fixing lint. Other than that, CI looks good. I took a look at the previous cherry-pick attempt to main (#13290) and it looks like the build failures were due to CMake export changes. That's resolved now.

[mergennachin]

Thank you!

[GregoryComer]

I verified that the wheel built from this job, installed into a clean env, can load pybindings on macos.

I also tested on Windows on a clean env, and I can't load the _portable_lib dll, though it's also broken on the release branch. I'll file as release blocking task.

[GregoryComer]

@pytorchbot cherry-pick --onto release/1.0 -c critical

[pytorchbot]

Cherry picking #14422

The cherry pick PR is at #14442 and it is recommended to link a critical cherry pick PR with an issue. The following tracker issues are updated:

• [v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job