Skip to content

Commit 92ed6e0

Browse files
tiyash-basu-frequenztiyash-basu-frequenz
committed
[ci] Add tests for non-native architectures
This change introduces a new job to the CI to test the SDK on non-native architectures. Github CI ATM provides amd64 runners by default, so the new job targets testing on arch64 for now. The tests are run in containers running on the target platform(s). The container definitions are supposed to exist in the directory .github/containers/nox-cross-arch. A definition for an arm64-ubuntu20.04-python3.11 container can be found there. Since the tests take quite a while to run, given that they run on emulations, this step runs only on the `merge_group` event. Signed-off-by: Tiyash Basu <[email protected]>
1 parent 7bb970a commit 92ed6e0

File tree

7 files changed

+145
-0
lines changed

7 files changed

+145
-0
lines changed

‎.github/containers/nox-cross-arch/README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Cross-Arch Testing Containers
2+
3+
This directory containers dockerfiles that can be used in the CI to test the
4+
python package in non-native machine architectures, e.g., `aarch64`.
5+
6+
The dockerfiles here follow a naming scheme so that they can be easily used in
7+
build matrices in the CI, in `nox-cross-arch` job. The naming scheme is:
8+
9+
```
10+
<arch>-<os>-python-<python-version>.Dockerfile
11+
```
12+
13+
E.g.,
14+
15+
```
16+
arm64-ubuntu-20.04-python-3.11.Dockerfile
17+
```
18+
19+
If a dockerfile for your desired target architecture, OS, and python version
20+
does not exist here, please add one before proceeding to add your options to the
21+
test matrix.

‎.github/containers/nox-cross-arch/arm64-ubuntu-20.04-python-3.11.Dockerfile‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
FROM docker.io/library/ubuntu:20.04
2+
3+
ENV DEBIAN_FRONTEND=noninteractive
4+
5+
# Install Python 3.11 and curl to install pip later
6+
RUN apt-get update -y && \
7+
apt-get install --no-install-recommends -y \
8+
software-properties-common && \
9+
add-apt-repository ppa:deadsnakes/ppa && \
10+
apt-get install --no-install-recommends -y \
11+
ca-certificates \
12+
curl \
13+
git \
14+
python3.11 \
15+
python3.11-distutils && \
16+
apt-get clean && \
17+
rm -rf /var/lib/apt/lists/*
18+
19+
# Install pip
20+
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11
21+
22+
RUN update-alternatives --install \
23+
/usr/local/bin/python python /usr/bin/python3.11 1 && \
24+
python -m pip install --upgrade --no-cache-dir pip
25+
26+
COPY entrypoint.bash /usr/bin/entrypoint.bash
27+
28+
ENTRYPOINT ["/usr/bin/entrypoint.bash"]

‎.github/containers/nox-cross-arch/entrypoint.bash‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
set -e
3+
4+
echo "System details:" $(uname -a)
5+
echo "Machine:" $(uname -m)
6+
7+
pip install -e .[dev-noxfile]
8+
9+
exec "$@"

‎.github/workflows/ci.yaml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,86 @@ jobs:
7979
run: nox -R -e "$NOX_SESSION"
8080
timeout-minutes: 10
8181

82+
nox-cross-arch:
83+
name: Cross-arch tests with nox
84+
if: github.event_name == 'merge_group'
85+
strategy:
86+
fail-fast: false
87+
# Before adding new items to this matrix, make sure that a dockerfile
88+
# exists for the combination of items in the matrix.
89+
# Refer to .github/containers/nox-cross-arch/README.md to learn how to
90+
# add and name new dockerfiles.
91+
matrix:
92+
arch:
93+
- arm64
94+
os:
95+
- ubuntu-20.04
96+
python:
97+
- "3.11"
98+
nox-session:
99+
- "pytest_min"
100+
runs-on: ${{ matrix.os }}
101+
102+
steps:
103+
- name: Fetch sources
104+
uses: actions/checkout@v3
105+
106+
- name: Set up QEMU
107+
uses: docker/setup-qemu-action@v2
108+
with:
109+
platforms: linux/${{ matrix.arch }}
110+
111+
- name: Set up Docker Buildx
112+
uses: docker/setup-buildx-action@v2
113+
114+
# This is a workaround to prevent the cache from growing indefinitely.
115+
# https://docs.docker.com/build/ci/github-actions/cache/#local-cache
116+
# https://github.com/docker/build-push-action/issues/252
117+
# https://github.com/moby/buildkit/issues/1896
118+
- name: Cache container layers
119+
uses: actions/cache@v3
120+
with:
121+
path: /tmp/.buildx-cache
122+
key: ${{ runner.os }}-buildx-nox-${{ matrix.arch }}-${{ matrix.os }}-${{ matrix.python }}
123+
124+
- name: Build image
125+
uses: docker/build-push-action@v4
126+
with:
127+
context: .github/containers/nox-cross-arch
128+
file: .github/containers/nox-cross-arch/${{ matrix.arch }}-${{ matrix.os }}-python-${{ matrix.python }}.Dockerfile
129+
platforms: linux/${{ matrix.arch }}
130+
tags: localhost/nox-cross-arch:latest
131+
push: false
132+
load: true
133+
cache-from: type=local,src=/tmp/.buildx-cache
134+
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max
135+
136+
# Refer to the workaround mentioned above
137+
- name: Move cache
138+
run: |
139+
rm -rf /tmp/.buildx-cache
140+
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
141+
142+
# Cache nox downloads
143+
- name: Cache nox downloads
144+
uses: actions/cache@v3
145+
with:
146+
path: |
147+
.nox/${{ matrix.nox-session }}
148+
key: nox-${{ matrix.nox-session }}-${{ matrix.arch }}-${{ matrix.os }}-${{ matrix.python }}-${{ hashFiles('**/pyproject.toml') }}
149+
150+
- name: Run nox
151+
run: |
152+
docker run \
153+
--rm \
154+
-v $(pwd):/${{ github.workspace }} \
155+
-w ${{ github.workspace }} \
156+
--net=host \
157+
--platform linux/${{ matrix.arch }} \
158+
localhost/nox-cross-arch:latest \
159+
nox -R -e ${{ matrix.nox-session }}
160+
timeout-minutes: 30
161+
82162
build:
83163
name: Build distribution packages
84164
runs-on: ubuntu-20.04

‎CONTRIBUTING.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,10 @@ That said, if you want to test the actual website in **your fork**, you can
115115
always use `mike deploy --push --remote your-fork-remote`, and then access the
116116
GitHub pages produced for your fork.
117117

118+
## Cross-Arch Testing
119+
120+
This project has built-in support for testing across multiple architectures. Currently, our CI conducts tests on `aarch64` machines using QEMU emulation. We also have the flexibility to expand this support to include additional architectures in the future. For more information, see [Cross-Arch Testing](.github/containers/nox-cross-arch/README.md).
121+
118122
## Releasing
119123

120124
These are the steps to create a new release:

‎RELEASE_NOTES.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@
2626

2727
- Now when printing `FormulaEngine` for debugging purposes the the formula will be shown in infix notation, which should be easier to read.
2828

29+
- The CI now runs cross-arch tests on `arm64` architectures.
30+
2931
## Bug Fixes
3032

3133
<!-- Here goes notable bug fixes that are worth a special mention or explanation -->

‎docs/.github/containers/nox-cross-arch/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
--8<-- ".github/containers/nox-cross-arch/README.md"

0 commit comments

Comments
 (0)