Add workflow to build Python wheels#19456
Conversation
Test Results 21 files 21 suites 3d 6h 11m 57s ⏱️ Results for commit 7d6cd62. ♻️ This comment has been updated with latest results. |
e376949 to
58efb5c
Compare
02e1ae6 to
aaffbd8
Compare
guitargeek
left a comment
There was a problem hiding this comment.
Thank you so much for this! I'm done with the review.
|
Thank you @guitargeek ! I addressed all the review comments, in particular:
|
570f735 to
870fbb0
Compare
guitargeek
left a comment
There was a problem hiding this comment.
Thanks for addressing the comments so quickly, it looks very good now!
Can we just do one CI run with the "clean build" flag please, since you changed "SearchInstalledSoftware.cmake"? And I guess you will also squash the commits a bit 😄
870fbb0 to
e38b8c0
Compare
|
Squashed all changes into a single commit. First commit currently is taken as-is from #19465 |
e38b8c0 to
3bfd16f
Compare
guitargeek
left a comment
There was a problem hiding this comment.
Thanks a lot for the PR! Works great, also locally for me with pip install root_src/!
This commit introduces the infrastructure required to build Python wheels from the ROOT project. In turn, these can be used to effectively install the package via `pip install ROOT`. For more details, see https://indico.cern.ch/event/1338689/contributions/6010410/. A Python wheel is a binary archive with the compiled artifacts of a package build. When installing the ROOT package via pip, the package manager will simply decompress the archive and all the ROOT libraries will be ready for use in the user's Python environment. In order to enable building a Python wheel of ROOT, the following changes have been made. First, it is required that all the components of the software do not explicitly link against the Python shared library (e.g. 'libpython.so'). The requirement comes from PEP503 which establishes a portable way to build Python wheels across different platforms, see https://peps.python.org/pep-0513/#libpythonx-y-so-1. Second, the directory structure of the destination user Python environment follows a semi-defined logic, where all artifacts of a package are unpacked at a root directory e.g. `site-packages` inside a sub-directory with the package name, e.g. `site-packages/ROOT`. In order to respect this structure, proper RPATH management of the ROOT libraries and, most notably, the CPython extensions (libROOTPythonizations and cppyy) is required. The `-Drpath=ON` build option is used in the build of the wheel. The CPython extensions are also modified with the right RPATH location. This is handled by building ROOT with the CMake variables CMAKE_INSTALL_BINDIR, CMAKE_INSTALL_INCLUDEDIR, CMAKE_INSTALL_LIBDIR and CMAKE_INSTALL_PYTHONDIR following the `site-packages/ROOT` structure during the build process and then moving the entire directory created this way directly to the target user Python environment directory. On top of being able to use ROOT as a Python package, the project provides a few CLI tools such as `root`. This first commit demonstrates how to also expose these tools in the Python packaging way, i.e. declaring a Python script to the build backend that invokes the correct command. This is defined in the [project.scripts] section of the newly created pyproject.toml file. At the moment, only the `root` command is going to be available from the Python wheel, other commands can be added in the future in a similar way. The wheel of ROOT is currently built via setuptools and using the `manylinux` platform tag to create portable wheels that can work across any Linux x86_64 distribution. One modification particular to the setuptools build is the declaration of a dummy CPython extension module in the setuptools `setup` function. This extensions has no code and takes no part in the build process, but it is enough to signal to the build backend that this package needs to be built for a specific Python version (required at the moment to ensure compatibility of the CPython extensions with the target system). Another issue is pypa/distutils#284, at this moment it looks like more recent versions of setuptools break this CMake extension compatibility. This should be addressed in the future. A new github workflow is written to automatically create Python wheels for every Python version supported by ROOT: 3.8, 3.9, 3.10, 3.11, 3.12 and 3.13 at the time of writing. The workflow is set to run nightly and can be later adjusted to run on specific triggers, e.g. during a PR build with a special label.
3bfd16f to
7d6cd62
Compare
guitargeek
left a comment
There was a problem hiding this comment.
Thanks for addressing the comments!
Now that the workflow is fully functional, see e.g. https://github.com/root-project/root/actions/runs/16493485486, I am reopening this PR.
The wheels can be installed on any Linux x86_64 node via: