Skip to content

Commit f8a7872

Browse files
authored
Use native arm runners for building wheels (#18618)
This takes down the CI time to build wheels from 50 minutes to <10 minutes. **It also fixes macOS ARM builds, and includes more ARM builds in general** (we were ignoring pypy and musl before). This doesn't cost much for us to do this, reasons for not doing this is 1. space on PyPI and 2. keeping them 'officially' supported? This is the list of wheels this built (`+` are the ones added): ```diff matrix_synapse-1.133.0-cp39-abi3-macosx_10_9_x86_64.whl + matrix_synapse-1.133.0-cp39-abi3-macosx_11_0_arm64.whl matrix_synapse-1.133.0-cp39-abi3-manylinux_2_28_aarch64.whl matrix_synapse-1.133.0-cp39-abi3-manylinux_2_28_x86_64.whl + matrix_synapse-1.133.0-cp39-abi3-musllinux_1_2_aarch64.whl matrix_synapse-1.133.0-cp39-abi3-musllinux_1_2_x86_64.whl matrix_synapse-1.133.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl + matrix_synapse-1.133.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl + matrix_synapse-1.133.0-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl matrix_synapse-1.133.0-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl matrix_synapse-1.133.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl + matrix_synapse-1.133.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl + matrix_synapse-1.133.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl matrix_synapse-1.133.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl ``` And the numbers aaaaare 🥁 - [before](https://github.com/element-hq/synapse/actions/runs/16072488018): 54 minutes - [after](https://github.com/element-hq/synapse/actions/runs/16004034949?pr=18618): 10 minutes **Revert [e43b0f9](e43b0f9) before merging**
1 parent b8ad9bf commit f8a7872

File tree

3 files changed

+14
-27
lines changed

3 files changed

+14
-27
lines changed

.github/workflows/release-artifacts.yml

Lines changed: 11 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -107,12 +107,15 @@ jobs:
107107
path: debs/*
108108

109109
build-wheels:
110-
name: Build wheels on ${{ matrix.os }} for ${{ matrix.arch }}
110+
name: Build wheels on ${{ matrix.os }}
111111
runs-on: ${{ matrix.os }}
112112
strategy:
113113
matrix:
114-
os: [ubuntu-24.04, macos-13]
115-
arch: [x86_64, aarch64]
114+
os:
115+
- ubuntu-24.04
116+
- ubuntu-24.04-arm
117+
- macos-13 # This uses x86-64
118+
- macos-14 # This uses arm64
116119
# is_pr is a flag used to exclude certain jobs from the matrix on PRs.
117120
# It is not read by the rest of the workflow.
118121
is_pr:
@@ -122,12 +125,11 @@ jobs:
122125
# Don't build macos wheels on PR CI.
123126
- is_pr: true
124127
os: "macos-13"
125-
# Don't build aarch64 wheels on mac.
126-
- os: "macos-13"
127-
arch: aarch64
128+
- is_pr: true
129+
os: "macos-14"
128130
# Don't build aarch64 wheels on PR CI.
129131
- is_pr: true
130-
arch: aarch64
132+
os: "ubuntu-24.04-arm"
131133

132134
steps:
133135
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
@@ -141,33 +143,20 @@ jobs:
141143
- name: Install cibuildwheel
142144
run: python -m pip install cibuildwheel==3.0.0
143145

144-
- name: Set up QEMU to emulate aarch64
145-
if: matrix.arch == 'aarch64'
146-
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
147-
with:
148-
platforms: arm64
149-
150-
- name: Build aarch64 wheels
151-
if: matrix.arch == 'aarch64'
152-
run: echo 'CIBW_ARCHS_LINUX=aarch64' >> $GITHUB_ENV
153-
154146
- name: Only build a single wheel on PR
155147
if: startsWith(github.ref, 'refs/pull/')
156-
run: echo "CIBW_BUILD="cp39-manylinux_${{ matrix.arch }}"" >> $GITHUB_ENV
148+
run: echo "CIBW_BUILD="cp39-manylinux_*"" >> $GITHUB_ENV
157149

158150
- name: Build wheels
159151
run: python -m cibuildwheel --output-dir wheelhouse
160152
env:
161153
# Skip testing for platforms which various libraries don't have wheels
162154
# for, and so need extra build deps.
163155
CIBW_TEST_SKIP: pp3*-* *i686* *musl*
164-
# Fix Rust OOM errors on emulated aarch64: https://github.com/rust-lang/cargo/issues/10583
165-
CARGO_NET_GIT_FETCH_WITH_CLI: true
166-
CIBW_ENVIRONMENT_PASS_LINUX: CARGO_NET_GIT_FETCH_WITH_CLI
167156

168157
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
169158
with:
170-
name: Wheel-${{ matrix.os }}-${{ matrix.arch }}
159+
name: Wheel-${{ matrix.os }}
171160
path: ./wheelhouse/*.whl
172161

173162
build-sdist:

changelog.d/18618.misc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Speed up the building of arm-based wheels in CI.

pyproject.toml

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -375,13 +375,10 @@ build-backend = "poetry.core.masonry.api"
375375
# Skip unsupported platforms (by us or by Rust).
376376
# See https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip for the list of build targets.
377377
# We skip:
378-
# - CPython 3.6, 3.7 and 3.8: EOLed
379-
# - PyPy 3.7 and 3.8: we only support Python 3.9+
378+
# - CPython and PyPy 3.8: EOLed
380379
# - musllinux i686: excluded to reduce number of wheels we build.
381380
# c.f. https://github.com/matrix-org/synapse/pull/12595#discussion_r963107677
382-
# - PyPy on Aarch64 and musllinux on aarch64: too slow to build.
383-
# c.f. https://github.com/matrix-org/synapse/pull/14259
384-
skip = "cp36* cp37* cp38* pp37* pp38* *-musllinux_i686 pp*aarch64 *-musllinux_aarch64"
381+
skip = "cp38* pp38* *-musllinux_i686"
385382
# Enable non-default builds.
386383
# "pypy" used to be included by default up until cibuildwheel 3.
387384
enable = "pypy"

0 commit comments

Comments
 (0)