Skip to content

Commit e2ebabb

Browse files
andrew-anyscaleandrew-anyscale
committed
wanda python-agnostic ray-cpp wheel (py3-none tag)
The ray-cpp wheel contains only C++ headers, libraries, and executables with no Python-specific code. Previously we built 4 identical wheels (one per Python version: cp310, cp311, cp312, cp313), wasting CI time and storage. This change produces a single wheel tagged py3-none-manylinux2014_* that works with any Python 3.x version. Changes: - Add ray-cpp-core.wanda.yaml and Dockerfile for cpp core - Add ray-cpp-wheel.wanda.yaml for cpp wheel builds - Add ci/build/build-ray-cpp-wheel.sh for Python-agnostic wheel builds - Add RayCppBdistWheel class to setup.py that forces py3-none tags (necessary because BinaryDistribution.has_ext_modules() causes bdist_wheel to use interpreter-specific ABI tags by default) - Update ray-cpp-wheel.wanda.yaml to build single wheel per architecture - Update .buildkite/build.rayci.yml to remove Python version matrix for cpp wheel build/upload steps Topic: ray-cpp-wheel Relative: crane-fix Signed-off-by: andrew <andrew@anyscale.com>
1 parent 9cb14c9 commit e2ebabb

File tree

7 files changed

+352
-1
lines changed

7 files changed

+352
-1
lines changed

‎.buildkite/build.rayci.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,55 @@ steps:
7676
- skip-on-premerge
7777
- oss
7878

79+
- name: ray-cpp-core-build
80+
label: "wanda: cpp core py{{matrix}} (x86_64)"
81+
wanda: ci/docker/ray-cpp-core.wanda.yaml
82+
matrix:
83+
- "3.10"
84+
- "3.11"
85+
- "3.12"
86+
- "3.13"
87+
env:
88+
PYTHON_VERSION: "{{matrix}}"
89+
ARCH_SUFFIX: ""
90+
HOSTTYPE: "x86_64"
91+
MANYLINUX_VERSION: "260103.868e54c"
92+
tags:
93+
- release_wheels
94+
- oss
95+
96+
- name: ray-cpp-wheel-build
97+
label: "wanda: cpp wheel (x86_64)"
98+
wanda: ci/docker/ray-cpp-wheel.wanda.yaml
99+
env:
100+
PYTHON_VERSION: "3.10" # Used for upstream images; wheel is Python-agnostic
101+
ARCH_SUFFIX: ""
102+
HOSTTYPE: "x86_64"
103+
MANYLINUX_VERSION: "260103.868e54c"
104+
tags:
105+
- release_wheels
106+
- linux_wheels
107+
- oss
108+
depends_on:
109+
- ray-core-build
110+
- ray-cpp-core-build
111+
- ray-java-build
112+
- ray-dashboard-build
113+
114+
# Upload cpp wheel to S3
115+
- label: ":s3: upload: cpp wheel (x86_64)"
116+
key: linux_cpp_wheels_upload
117+
instance_type: small
118+
commands:
119+
- ./ci/build/extract_wanda_wheels.sh ray-cpp-wheel
120+
- ./ci/build/copy_build_artifacts.sh wheel
121+
depends_on:
122+
- ray-cpp-wheel-build
123+
tags:
124+
- release_wheels
125+
- skip-on-premerge
126+
- oss
127+
79128
- label: ":tapioca: build: jar"
80129
key: java_wheels
81130
tags:

‎ci/build/build-ray-cpp-wheel.sh‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#!/bin/bash
2+
# Build Python-agnostic ray-cpp wheel.
3+
#
4+
# The ray-cpp wheel contains only C++ headers, libraries, and executables
5+
# with no Python version-specific content. This script builds a single wheel
6+
# tagged py3-none-manylinux2014_* that works with any Python 3 version.
7+
#
8+
# Usage: ./ci/build/build-cpp-wheel.sh
9+
#
10+
# Environment variables:
11+
# BUILDKITE_COMMIT - Git commit SHA for ray.__commit__ (required)
12+
13+
set -exuo pipefail
14+
15+
TRAVIS_COMMIT="${TRAVIS_COMMIT:-$BUILDKITE_COMMIT}"
16+
17+
# Use any Python 3 to run setuptools (wheel content is Python-agnostic)
18+
PYTHON="cp310-cp310"
19+
20+
mkdir -p .whl
21+
cd python
22+
23+
/opt/python/"${PYTHON}"/bin/pip install -q cython==3.0.12 setuptools==80.9.0
24+
25+
# Set the commit SHA in _version.py
26+
if [[ -n "$TRAVIS_COMMIT" ]]; then
27+
sed -i.bak "s/{{RAY_COMMIT_SHA}}/$TRAVIS_COMMIT/g" ray/_version.py && rm ray/_version.py.bak
28+
else
29+
echo "TRAVIS_COMMIT variable not set - required to populate ray.__commit__."
30+
exit 1
31+
fi
32+
33+
# Determine platform tag based on architecture
34+
ARCH=$(uname -m)
35+
if [[ "$ARCH" == "x86_64" ]]; then
36+
PLAT_NAME="manylinux2014_x86_64"
37+
elif [[ "$ARCH" == "aarch64" ]]; then
38+
PLAT_NAME="manylinux2014_aarch64"
39+
else
40+
echo "ERROR: Unsupported architecture: $ARCH"
41+
exit 1
42+
fi
43+
44+
echo "Building ray-cpp wheel (Python-agnostic, platform: $PLAT_NAME)..."
45+
46+
# Build wheel with py3-none tag (Python version-agnostic)
47+
# SKIP_BAZEL_BUILD=1 because artifacts are pre-built from wanda cache
48+
PATH="/opt/python/${PYTHON}/bin:$PATH" \
49+
SKIP_BAZEL_BUILD=1 RAY_INSTALL_JAVA=0 RAY_INSTALL_CPP=1 \
50+
"/opt/python/${PYTHON}/bin/python" setup.py bdist_wheel \
51+
--python-tag py3 \
52+
--plat-name "$PLAT_NAME"
53+
54+
mv dist/*.whl ../.whl/

‎ci/docker/ray-cpp-core.Dockerfile‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# syntax=docker/dockerfile:1.3-labs
2+
#
3+
# Ray C++ Core Artifacts Builder
4+
# ==============================
5+
# Builds ray_cpp_pkg.zip containing C++ headers, libraries, and examples.
6+
#
7+
ARG ARCH_SUFFIX=
8+
ARG HOSTTYPE=x86_64
9+
ARG MANYLINUX_VERSION
10+
FROM rayproject/manylinux2014:${MANYLINUX_VERSION}-jdk-${HOSTTYPE} AS builder
11+
12+
ARG PYTHON_VERSION=3.10
13+
ARG BUILDKITE_BAZEL_CACHE_URL
14+
ARG BUILDKITE_CACHE_READONLY
15+
ARG HOSTTYPE
16+
17+
WORKDIR /home/forge/ray
18+
19+
COPY . .
20+
21+
# Mounting cache dir for faster local rebuilds (architecture-specific to avoid toolchain conflicts)
22+
RUN --mount=type=cache,target=/home/forge/.cache,uid=2000,gid=100,id=bazel-cache-${HOSTTYPE}-${PYTHON_VERSION} \
23+
<<'EOF'
24+
#!/bin/bash
25+
set -euo pipefail
26+
27+
PY_CODE="${PYTHON_VERSION//./}"
28+
PY_BIN="cp${PY_CODE}-cp${PY_CODE}"
29+
export RAY_BUILD_ENV="manylinux_py${PY_BIN}"
30+
31+
sudo ln -sf "/opt/python/${PY_BIN}/bin/python3" /usr/local/bin/python3
32+
sudo ln -sf /usr/local/bin/python3 /usr/local/bin/python
33+
34+
if [[ "${BUILDKITE_CACHE_READONLY:-}" == "true" ]]; then
35+
echo "build --remote_upload_local_results=false" >> "$HOME/.bazelrc"
36+
fi
37+
38+
echo "build --repository_cache=/home/forge/.cache/bazel-repo" >> "$HOME/.bazelrc"
39+
40+
bazelisk build --config=ci //cpp:ray_cpp_pkg_zip
41+
42+
cp bazel-bin/cpp/ray_cpp_pkg.zip /home/forge/ray_cpp_pkg.zip
43+
44+
EOF
45+
46+
FROM scratch
47+
48+
COPY --from=builder /home/forge/ray_cpp_pkg.zip /

‎ci/docker/ray-cpp-core.wanda.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: "ray-cpp-core-py$PYTHON_VERSION$ARCH_SUFFIX"
2+
froms: ["rayproject/manylinux2014:$MANYLINUX_VERSION-jdk-$HOSTTYPE"]
3+
dockerfile: ci/docker/ray-cpp-core.Dockerfile
4+
srcs:
5+
- .bazelversion
6+
- .bazelrc
7+
- WORKSPACE
8+
- BUILD.bazel
9+
- bazel/
10+
- thirdparty/
11+
- src/
12+
- cpp/
13+
- python/ray/__init__.py
14+
- python/ray/_raylet.pxd
15+
- python/ray/_raylet.pyi
16+
- python/ray/_raylet.pyx
17+
- python/ray/includes/
18+
- java/BUILD.bazel
19+
- java/dependencies.bzl
20+
- release/BUILD.bazel
21+
- release/requirements.txt
22+
- release/requirements_py310.txt
23+
build_args:
24+
- PYTHON_VERSION
25+
- ARCH_SUFFIX
26+
- HOSTTYPE
27+
- MANYLINUX_VERSION
28+
- BUILDKITE_BAZEL_CACHE_URL
29+
build_hint_args:
30+
- BUILDKITE_CACHE_READONLY

‎ci/docker/ray-cpp-wheel.Dockerfile‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# syntax=docker/dockerfile:1.3-labs
2+
#
3+
# Ray C++ Wheel Builder
4+
# =====================
5+
# Builds manylinux2014-compatible ray-cpp wheel using pre-built C++ artifacts from wanda cache.
6+
#
7+
# This is a minimal Dockerfile for ray-cpp wheel builds only.
8+
# It copies only the files needed for the cpp wheel, reducing build context size.
9+
10+
ARG RAY_CORE_IMAGE
11+
ARG RAY_CPP_CORE_IMAGE
12+
ARG RAY_JAVA_IMAGE
13+
ARG RAY_DASHBOARD_IMAGE
14+
ARG MANYLINUX_VERSION
15+
ARG HOSTTYPE
16+
17+
FROM ${RAY_CORE_IMAGE} AS ray-core
18+
FROM ${RAY_CPP_CORE_IMAGE} AS ray-cpp-core
19+
FROM ${RAY_JAVA_IMAGE} AS ray-java
20+
FROM ${RAY_DASHBOARD_IMAGE} AS ray-dashboard
21+
22+
# Main build stage - manylinux2014 provides GLIBC 2.17
23+
FROM rayproject/manylinux2014:${MANYLINUX_VERSION}-jdk-${HOSTTYPE} AS builder
24+
25+
ARG BUILDKITE_COMMIT
26+
27+
WORKDIR /home/forge/ray
28+
29+
# Copy artifacts from all stages
30+
COPY --from=ray-core /ray_pkg.zip /tmp/
31+
COPY --from=ray-core /ray_py_proto.zip /tmp/
32+
COPY --from=ray-java /ray_java_pkg.zip /tmp/
33+
COPY --from=ray-dashboard /dashboard.tar.gz /tmp/
34+
35+
# Minimal source files needed for cpp wheel build
36+
COPY --chown=forge ci/build/build-ray-cpp-wheel.sh ci/build/
37+
COPY --chown=forge README.rst pyproject.toml ./
38+
COPY --chown=forge python/setup.py python/
39+
COPY --chown=forge python/LICENSE.txt python/
40+
COPY --chown=forge python/MANIFEST.in python/
41+
COPY --chown=forge python/ray/_version.py python/ray/
42+
43+
USER forge
44+
ENV BUILDKITE_COMMIT=${BUILDKITE_COMMIT:-unknown}
45+
RUN --mount=from=ray-cpp-core,source=/,target=/ray-cpp-core,ro \
46+
<<'EOF'
47+
#!/bin/bash
48+
set -euo pipefail
49+
50+
# Clean extraction dirs to avoid stale leftovers
51+
rm -rf /tmp/ray_pkg /tmp/ray_java_pkg /tmp/ray_cpp_pkg
52+
mkdir -p /tmp/ray_pkg /tmp/ray_java_pkg /tmp/ray_cpp_pkg
53+
54+
# Unpack pre-built artifacts
55+
unzip -o /tmp/ray_pkg.zip -d /tmp/ray_pkg
56+
unzip -o /tmp/ray_py_proto.zip -d python/
57+
unzip -o /tmp/ray_java_pkg.zip -d /tmp/ray_java_pkg
58+
mkdir -p python/ray/dashboard/client/build
59+
tar -xzf /tmp/dashboard.tar.gz -C python/ray/dashboard/client/build/
60+
61+
# C++ core artifacts
62+
cp -r /tmp/ray_pkg/ray/* python/ray/
63+
64+
# Java JARs
65+
cp -r /tmp/ray_java_pkg/ray/* python/ray/
66+
67+
# C++ API artifacts (headers, libs, examples)
68+
unzip -o /ray-cpp-core/ray_cpp_pkg.zip -d /tmp/ray_cpp_pkg
69+
cp -r /tmp/ray_cpp_pkg/ray/cpp python/ray/
70+
71+
# Build Python-agnostic cpp wheel
72+
./ci/build/build-ray-cpp-wheel.sh
73+
74+
# Sanity check: ensure wheels exist
75+
if [[ ! -d .whl ]]; then
76+
echo "ERROR: .whl directory not created"
77+
exit 1
78+
fi
79+
wheels=($(find .whl -maxdepth 1 -name '*.whl'))
80+
if (( ${#wheels[@]} == 0 )); then
81+
echo "ERROR: No wheels produced in .whl/"
82+
ls -la .whl
83+
exit 1
84+
fi
85+
86+
EOF
87+
88+
FROM scratch
89+
COPY --from=builder /home/forge/ray/.whl/*.whl /

‎ci/docker/ray-cpp-wheel.wanda.yaml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Python-agnostic ray-cpp wheel.
2+
# Uses upstream images from one Python version but produces a py3-none wheel
3+
# that works with any Python 3. PYTHON_VERSION is set at buildkite level.
4+
name: "ray-cpp-wheel$ARCH_SUFFIX"
5+
froms:
6+
- "rayproject/manylinux2014:$MANYLINUX_VERSION-jdk-$HOSTTYPE"
7+
- "cr.ray.io/rayproject/ray-core-py$PYTHON_VERSION$ARCH_SUFFIX" # C++ binaries (ray_pkg.zip)
8+
- "cr.ray.io/rayproject/ray-cpp-core-py$PYTHON_VERSION$ARCH_SUFFIX" # C++ headers/libs (ray_cpp_pkg.zip)
9+
- "cr.ray.io/rayproject/ray-java-build$ARCH_SUFFIX" # Java JARs
10+
- "cr.ray.io/rayproject/ray-dashboard" # Dashboard
11+
dockerfile: ci/docker/ray-cpp-wheel.Dockerfile
12+
srcs:
13+
- pyproject.toml
14+
- README.rst
15+
- ci/build/build-ray-cpp-wheel.sh
16+
- python/setup.py
17+
- python/LICENSE.txt
18+
- python/MANIFEST.in
19+
- python/ray/_version.py
20+
build_args:
21+
- MANYLINUX_VERSION
22+
- HOSTTYPE
23+
- BUILDKITE_COMMIT
24+
- ARCH_SUFFIX
25+
- RAY_CORE_IMAGE=cr.ray.io/rayproject/ray-core-py$PYTHON_VERSION$ARCH_SUFFIX
26+
- RAY_CPP_CORE_IMAGE=cr.ray.io/rayproject/ray-cpp-core-py$PYTHON_VERSION$ARCH_SUFFIX
27+
- RAY_JAVA_IMAGE=cr.ray.io/rayproject/ray-java-build$ARCH_SUFFIX
28+
- RAY_DASHBOARD_IMAGE=cr.ray.io/rayproject/ray-dashboard

‎python/setup.py‎

Lines changed: 54 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -757,6 +757,13 @@ def pip_run(build_ext):
757757
import setuptools
758758
import setuptools.command.build_ext
759759

760+
# bdist_wheel location varies: setuptools>=70.1 has it built-in,
761+
# older versions require the wheel package
762+
try:
763+
from setuptools.command.bdist_wheel import bdist_wheel
764+
except ImportError:
765+
from wheel.bdist_wheel import bdist_wheel
766+
760767
class build_ext(setuptools.command.build_ext.build_ext):
761768
def run(self):
762769
return pip_run(self)
@@ -765,6 +772,46 @@ class BinaryDistribution(setuptools.Distribution):
765772
def has_ext_modules(self):
766773
return True
767774

775+
class RayCppBdistWheel(bdist_wheel):
776+
"""Custom bdist_wheel that produces Python-agnostic wheels for ray-cpp.
777+
778+
PROBLEM
779+
-------
780+
The ray-cpp wheel contains only C++ files (headers, libraries, executables)
781+
with NO Python-specific code. It should work with any Python 3.x version.
782+
However, by default, setuptools/bdist_wheel generates wheels tagged with
783+
the specific Python interpreter used to build, e.g.:
784+
785+
ray_cpp-3.0.0-cp310-cp310-manylinux2014_x86_64.whl
786+
^^^^^-^^^^^
787+
Python 3.10 specific!
788+
789+
This class forces the wheel tag to be:
790+
791+
ray_cpp-3.0.0-py3-none-manylinux2014_x86_64.whl
792+
^^^-^^^^
793+
Works with ANY Python 3!
794+
795+
Tag meanings:
796+
- "py3" = compatible with any Python 3.x
797+
- "none" = no Python ABI dependency (no compiled .so using Python.h)
798+
- "manylinux2014_x86_64" = platform tag (still needed for C++ binaries)
799+
800+
"""
801+
802+
def finalize_options(self):
803+
super().finalize_options()
804+
# Mark as non-pure to ensure we get a platform tag (e.g., manylinux2014).
805+
# Pure wheels get "any" as the platform tag, which is wrong for ray-cpp
806+
# since it contains platform-specific C++ binaries.
807+
self.root_is_pure = False
808+
809+
def get_tag(self):
810+
# Get the platform tag from parent (e.g., "manylinux2014_x86_64")
811+
_, _, platform_tag = super().get_tag()
812+
# Force Python-agnostic tags: ("py3", "none", platform_tag)
813+
return "py3", "none", platform_tag
814+
768815
# Ensure no remaining lib files.
769816
build_dir = os.path.join(ROOT_DIR, "build")
770817
if os.path.isdir(build_dir):
@@ -781,6 +828,12 @@ def has_ext_modules(self):
781828
# If the license text has multiple lines, add an ending endline.
782829
license_text += "\n"
783830

831+
# Build cmdclass dict. Use RayCppBdistWheel for ray-cpp to produce
832+
# Python-agnostic wheels. See RayCppBdistWheel docstring for details.
833+
cmdclass = {"build_ext": build_ext}
834+
if setup_spec.type == SetupType.RAY_CPP:
835+
cmdclass["bdist_wheel"] = RayCppBdistWheel
836+
784837
setuptools.setup(
785838
name=setup_spec.name,
786839
version=setup_spec.version,
@@ -801,7 +854,7 @@ def has_ext_modules(self):
801854
"Programming Language :: Python :: 3.13",
802855
],
803856
packages=setup_spec.get_packages(),
804-
cmdclass={"build_ext": build_ext},
857+
cmdclass=cmdclass,
805858
distclass=( # Avoid building extensions for deps-only builds.
806859
BinaryDistribution if setup_spec.build_type != BuildType.DEPS_ONLY else None
807860
),

0 commit comments

Comments
 (0)