Skip to content

Add workflow to build Python wheels#19456

Merged
vepadulano merged 1 commit intomasterfrom
experimental-pip-install-root
Aug 1, 2025
Merged

Add workflow to build Python wheels#19456
vepadulano merged 1 commit intomasterfrom
experimental-pip-install-root

Conversation

@vepadulano
Copy link
Member

Now that the workflow is fully functional, see e.g. https://github.com/root-project/root/actions/runs/16493485486, I am reopening this PR.

The wheels can be installed on any Linux x86_64 node via:

$: pip install ROOT --index-url=https://root-experimental-python-wheels.web.cern.ch

@vepadulano vepadulano requested a review from guitargeek July 24, 2025 12:07
@vepadulano vepadulano self-assigned this Jul 24, 2025
@vepadulano vepadulano requested a review from bellenot as a code owner July 24, 2025 12:07
@vepadulano vepadulano requested a review from dpiparo as a code owner July 24, 2025 12:07
@github-actions
Copy link

github-actions bot commented Jul 24, 2025

Test Results

    21 files      21 suites   3d 6h 11m 57s ⏱️
 3 220 tests  3 220 ✅ 0 💤 0 ❌
65 897 runs  65 897 ✅ 0 💤 0 ❌

Results for commit 7d6cd62.

♻️ This comment has been updated with latest results.

@vepadulano vepadulano force-pushed the experimental-pip-install-root branch from e376949 to 58efb5c Compare July 25, 2025 10:02
@vepadulano vepadulano mentioned this pull request Jul 25, 2025
@vepadulano vepadulano force-pushed the experimental-pip-install-root branch from 02e1ae6 to aaffbd8 Compare July 29, 2025 08:19
Copy link
Contributor

@guitargeek guitargeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for this! I'm done with the review.

@vepadulano vepadulano requested a review from guitargeek July 29, 2025 09:53
@vepadulano
Copy link
Member Author

Thank you @guitargeek ! I addressed all the review comments, in particular:

  • I removed the extra subprocess.run call in __init__.py for the moment. It was meant to demonstrate a way to cope with missing dependencies, but probably we need to have a larger discussion about it
  • I kept for now the tutorials folder as I believe we can use it as a launcher for tests of the wheel, and the extra 25MB at the moment are not a real issue.

@vepadulano vepadulano force-pushed the experimental-pip-install-root branch 2 times, most recently from 570f735 to 870fbb0 Compare July 29, 2025 09:59
guitargeek
guitargeek previously approved these changes Jul 29, 2025
Copy link
Contributor

@guitargeek guitargeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments so quickly, it looks very good now!

Can we just do one CI run with the "clean build" flag please, since you changed "SearchInstalledSoftware.cmake"? And I guess you will also squash the commits a bit 😄

@vepadulano vepadulano dismissed guitargeek’s stale review July 29, 2025 15:32

After offline discussion, at the moment it looks like having #19470 and #19465 merged would simplify the RPATH logic for the CPython extensions hence improving also the logic for building the Python wheel. We can wait a couple more days before proceeding with this.

@vepadulano vepadulano force-pushed the experimental-pip-install-root branch from 870fbb0 to e38b8c0 Compare July 30, 2025 13:59
@vepadulano
Copy link
Member Author

Squashed all changes into a single commit. First commit currently is taken as-is from #19465

@vepadulano vepadulano requested a review from guitargeek July 31, 2025 08:56
Copy link
Contributor

@guitargeek guitargeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR! Works great, also locally for me with pip install root_src/!

This commit introduces the infrastructure required to build Python wheels from
the ROOT project. In turn, these can be used to effectively install the package
via `pip install ROOT`. For more details, see
https://indico.cern.ch/event/1338689/contributions/6010410/.

A Python wheel is a binary archive with the compiled artifacts of a package
build. When installing the ROOT package via pip, the package manager will simply
decompress the archive and all the ROOT libraries will be ready for use in the
user's Python environment.

In order to enable building a Python wheel of ROOT, the following changes have
been made.

First, it is required that all the components of the software do not explicitly
link against the Python shared library (e.g. 'libpython.so'). The requirement
comes from PEP503 which establishes a portable way to build Python wheels across
different platforms, see https://peps.python.org/pep-0513/#libpythonx-y-so-1.

Second, the directory structure of the destination user Python environment
follows a semi-defined logic, where all artifacts of a package are unpacked at a
root directory e.g. `site-packages` inside a sub-directory with the package
name, e.g. `site-packages/ROOT`. In order to respect this structure, proper
RPATH management of the ROOT libraries and, most notably, the CPython extensions
(libROOTPythonizations and cppyy) is required. The `-Drpath=ON` build option is
used in the build of the wheel. The CPython extensions are also modified with the
right RPATH location. This is handled by building ROOT with the CMake variables
CMAKE_INSTALL_BINDIR, CMAKE_INSTALL_INCLUDEDIR, CMAKE_INSTALL_LIBDIR and
CMAKE_INSTALL_PYTHONDIR following the `site-packages/ROOT` structure during the
build process and then moving the entire directory created this way directly to
the target user Python environment directory.

On top of being able to use ROOT as a Python package, the project provides a few
CLI tools such as `root`. This first commit demonstrates how to also expose
these tools in the Python packaging way, i.e. declaring a Python script to the
build backend that invokes the correct command. This is defined in the
[project.scripts] section of the newly created pyproject.toml file. At the
moment, only the `root` command is going to be available from the Python wheel,
other commands can be added in the future in a similar way.

The wheel of ROOT is currently built via setuptools and using the `manylinux`
platform tag to create portable wheels that can work across any Linux x86_64
distribution. One modification particular to the setuptools build is the
declaration of a dummy CPython extension module in the setuptools `setup`
function. This extensions has no code and takes no part in the build process,
but it is enough to signal to the build backend that this package needs to be
built for a specific Python version (required at the moment to ensure
compatibility of the CPython extensions with the target system). Another issue
is pypa/distutils#284, at this moment it looks like
more recent versions of setuptools break this CMake extension compatibility.
This should be addressed in the future.

A new github workflow is written to automatically create Python wheels for every
Python version supported by ROOT: 3.8, 3.9, 3.10, 3.11, 3.12 and 3.13 at the
time of writing. The workflow is set to run nightly and can be later adjusted to
run on specific triggers, e.g. during a PR build with a special label.
@vepadulano vepadulano force-pushed the experimental-pip-install-root branch from 3bfd16f to 7d6cd62 Compare July 31, 2025 15:27
Copy link
Contributor

@guitargeek guitargeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments!

@vepadulano vepadulano merged commit b0bd609 into master Aug 1, 2025
33 checks passed
@vepadulano vepadulano deleted the experimental-pip-install-root branch August 1, 2025 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants