Skip to content

Conversation

@larryliu0820
Copy link
Contributor

@larryliu0820 larryliu0820 commented Nov 26, 2024

Summary:
Fixes #7086
Based on this wiki:

https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling

Takeaway of reading this doc:

  • Set CMAKE_BUILD_WITH_INSTALL_RPATH to False so that we don't add install path (pip-out, cmake-out) into RPATH.
  • Set CMAKE_INSTALL_RPATH to be able to refer to libraries in the same install directory if any.
  • Set CMAKE_INSTALL_RPATH_USE_LINK_PATH to True so that we include other so files outside of the build tree can be added to RPATH.

Test Plan:

Tested on Colab:

!mkdir et; cd et; git clone https://github.com/pytorch/executorch.git
!cd et/executorch; git submodule update --init

# need to modify Codegen.cmake to make the following working
!cd et/executorch; bash install_requirements.sh
!cd et/executorch; CMAKE_INSTALL_PREFIX=/usr/local/lib/python3.10/dist-packages python setup.py bdist_wheel

Reviewers:

Subscribers:

Tasks:

Tags:

Summary

[PLEASE REMOVE] See CONTRIBUTING.md's Pull Requests for ExecuTorch PR guidelines.

[PLEASE REMOVE] If this PR closes an issue, please add a Fixes #<issue-id> line.

[PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: " label. For a list of available release notes labels, check out CONTRIBUTING.md's Pull Requests.

Test plan

[PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable.

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 26, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7096

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Cancelled Job

As of commit 8b55993 with merge base 2a292c3 (image):

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 26, 2024
@larryliu0820 larryliu0820 added topic: not user facing ciflow/binaries/all Release PRs with this label will build wheels for all python versions labels Nov 26, 2024
@larryliu0820 larryliu0820 changed the title Test Fix shared library rpath once for all Nov 26, 2024
Copy link
Contributor

@kirklandsign kirklandsign left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@kirklandsign
Copy link
Contributor

That's much cleaner!

@larryliu0820 larryliu0820 merged commit dedf77b into main Nov 27, 2024
110 of 121 checks passed
@larryliu0820 larryliu0820 deleted the test-rpath branch November 27, 2024 00:43
larryliu0820 added a commit that referenced this pull request Dec 3, 2024
Summary: This is a followup to #7096. That PR was aiming to rely on
`CMAKE_INSTALL_RPATH_USE_LINK_PATH` to set RPATH automatically. However
due to the caveat that we are not installing `_portable_lib.so` into the
correct path in CMake stage (we move the .so in setup.py after it's
installed), we are not able to add the correct RPATH to
libcustom_ops_aot_lib.so.

This PR still set the INSTALL_RPATH manually and adds TODOs to fix it
later.

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
larryliu0820 added a commit that referenced this pull request Dec 3, 2024
* [pybind] Fix rpath for custom ops

Summary: This is a followup to #7096. That PR was aiming to rely on
`CMAKE_INSTALL_RPATH_USE_LINK_PATH` to set RPATH automatically. However
due to the caveat that we are not installing `_portable_lib.so` into the
correct path in CMake stage (we move the .so in setup.py after it's
installed), we are not able to add the correct RPATH to
libcustom_ops_aot_lib.so.

This PR still set the INSTALL_RPATH manually and adds TODOs to fix it
later.

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Remove typo

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/binaries/all Release PRs with this label will build wheels for all python versions ciflow/binaries ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: not user facing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pip install executorch doesn't install llama custom ops

4 participants