-
Notifications
You must be signed in to change notification settings - Fork 271
feat: add initial ARM64 (aarch64) architecture support #1875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
dd15e97
feat: ARM64 architecture detection and build system
36b1781
feat: arch-aware Firecracker and kernel path resolution
0c9c939
feat: ARM64 runtime guards for SMT, UFFD WP, and seccomp
090f084
ci: add ARM64 cross-compilation and unit test workflow
a1b2124
fix: data races and test fixes exposed by ARM64 testing
2ffb97e
docs: ARM64 architecture support in orchestrator README
348579d
Merge branch 'main' into arm64-support
tomassrnka d9f545a
feat: enable UFFD write-protection on ARM64 (requires kernel 6.10+)
1bc1673
fix(shared): data races in multipart upload tests
bfb34cd
Merge branch 'main' into arm64-support
tomassrnka c9c2f76
chore: remove dead StracePfx code and revert package-lock.json drift
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Setup script for an ARM64 self-hosted GitHub Actions runner. | ||
| # Run this on a fresh ARM64 Ubuntu 22.04/24.04 machine with KVM support. | ||
| # | ||
| # Prerequisites: | ||
| # - ARM64 Linux host (Graviton, Ampere, etc.) | ||
| # - KVM enabled (/dev/kvm accessible) | ||
| # - At least 8GB RAM (for hugepage allocation) | ||
| # - Root access | ||
| # | ||
| # Usage: | ||
| # sudo ./setup-arm64-runner.sh | ||
| # | ||
| # After running this script, register the machine as a GitHub Actions | ||
| # self-hosted runner with the label: infra-tests-arm64 | ||
| # https://github.com/e2b-dev/infra/settings/actions/runners/new | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| PS4='[\D{%Y-%m-%d %H:%M:%S}] ' | ||
| set -x | ||
|
|
||
| if [ "$(id -u)" -ne 0 ]; then | ||
| echo "ERROR: This script must be run as root" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| ARCH=$(dpkg --print-architecture) | ||
| if [ "$ARCH" != "arm64" ]; then | ||
| echo "ERROR: This script is for ARM64 hosts (detected: $ARCH)" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "=== Setting up ARM64 GitHub Actions runner ===" | ||
|
|
||
| # KVM check | ||
| if [ ! -e /dev/kvm ]; then | ||
| echo "ERROR: /dev/kvm not found. KVM support is required." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Install base dependencies | ||
| apt-get update | ||
| apt-get install -y --no-install-recommends \ | ||
| build-essential \ | ||
| curl \ | ||
| git \ | ||
| jq \ | ||
| nbd-client \ | ||
| nbd-server | ||
|
|
||
| # Enable unprivileged userfaultfd | ||
| echo 1 > /proc/sys/vm/unprivileged_userfaultfd | ||
|
|
||
| # Hugepages | ||
| mkdir -p /mnt/hugepages | ||
| mount -t hugetlbfs none /mnt/hugepages 2>/dev/null || true | ||
| echo 2000 > /proc/sys/vm/nr_hugepages | ||
|
|
||
| grep -qF 'hugetlbfs /mnt/hugepages' /etc/fstab || \ | ||
| echo "hugetlbfs /mnt/hugepages hugetlbfs defaults 0 0" >> /etc/fstab | ||
|
|
||
| # Sysctl — write once (idempotent) | ||
| cat <<'EOF' > /etc/sysctl.d/99-e2b.conf | ||
| vm.unprivileged_userfaultfd=1 | ||
| vm.nr_hugepages=2000 | ||
| net.core.somaxconn=65535 | ||
| net.core.netdev_max_backlog=65535 | ||
| net.ipv4.tcp_max_syn_backlog=65535 | ||
| vm.max_map_count=1048576 | ||
| EOF | ||
| sysctl --system | ||
|
|
||
| # NBD | ||
| modprobe nbd nbds_max=256 | ||
| echo "nbd" > /etc/modules-load.d/e2b.conf | ||
| echo "options nbd nbds_max=256" > /etc/modprobe.d/e2b-nbd.conf | ||
|
|
||
| # Disable inotify for NBD devices | ||
| cat <<'EOF' > /etc/udev/rules.d/97-nbd-device.rules | ||
| ACTION=="add|change", KERNEL=="nbd*", OPTIONS:="nowatch" | ||
| EOF | ||
| udevadm control --reload-rules | ||
| udevadm trigger | ||
|
|
||
| # File descriptor limits | ||
| cat <<'EOF' > /etc/security/limits.d/99-e2b.conf | ||
| * soft nofile 1048576 | ||
| * hard nofile 1048576 | ||
| EOF | ||
|
|
||
| echo "" | ||
| echo "=== ARM64 runner setup complete ===" | ||
| echo "" | ||
| echo "Verify:" | ||
| echo " uname -m → aarch64" | ||
| echo " ls /dev/kvm → exists" | ||
| echo " cat /proc/meminfo | grep HugePages_Total" | ||
| echo " lsmod | grep nbd" | ||
| echo "" | ||
| echo "Next: register this machine as a GitHub Actions self-hosted runner" | ||
| echo " Label: infra-tests-arm64" | ||
| echo " https://github.com/e2b-dev/infra/settings/actions/runners/new" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,115 @@ | ||
| name: ARM64 tests on PRs | ||
|
|
||
| on: [workflow_call] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| cross-compile: | ||
| name: Cross-compile all packages for ARM64 | ||
| runs-on: ubuntu-24.04 | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: ./.github/actions/go-setup-cache | ||
|
|
||
| - name: Install ARM64 cross-compiler | ||
| run: sudo apt-get update && sudo apt-get install -y gcc-aarch64-linux-gnu | ||
|
|
||
| - name: Build and vet packages (pure Go) | ||
| run: | | ||
| for pkg in api client-proxy envd shared db docker-reverse-proxy; do | ||
| echo "::group::packages/$pkg" | ||
| pushd "packages/$pkg" > /dev/null | ||
| GOARCH=arm64 go build ./... | ||
| GOARCH=arm64 go vet ./... | ||
| popd > /dev/null | ||
| echo "::endgroup::" | ||
| done | ||
|
|
||
| - name: Build and vet orchestrator (CGO) | ||
| run: | | ||
| CGO_ENABLED=1 CC=aarch64-linux-gnu-gcc GOARCH=arm64 go build ./... | ||
| CGO_ENABLED=1 CC=aarch64-linux-gnu-gcc GOARCH=arm64 go vet ./... | ||
| working-directory: packages/orchestrator | ||
|
|
||
| arm64-unit-tests: | ||
| name: ARM64 tests for ${{ matrix.package }} | ||
| runs-on: ubuntu-24.04-arm | ||
| timeout-minutes: 30 | ||
| strategy: | ||
| matrix: | ||
| include: | ||
| - package: packages/api | ||
| test_path: ./... | ||
| sudo: false | ||
| - package: packages/client-proxy | ||
| test_path: ./... | ||
| sudo: false | ||
| - package: packages/db | ||
| test_path: ./... | ||
| sudo: false | ||
| - package: packages/docker-reverse-proxy | ||
| test_path: ./... | ||
| sudo: false | ||
| - package: packages/envd | ||
| test_path: ./... | ||
| sudo: true | ||
| - package: packages/orchestrator | ||
| test_path: ./... | ||
| sudo: true | ||
| - package: packages/shared | ||
| test_path: ./pkg/... | ||
| sudo: false | ||
| fail-fast: false | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: ./.github/actions/go-setup-cache | ||
| with: | ||
| cache-dependency-paths: | | ||
| go.work | ||
| ${{ matrix.package }}/go.mod | ||
| ${{ matrix.package }}/go.sum | ||
|
|
||
| - name: Setup envd tests | ||
| run: | | ||
| sudo apt-get update && sudo apt-get install -y bindfs | ||
| if: matrix.package == 'packages/envd' | ||
|
|
||
| - name: Setup orchestrator tests | ||
| run: | | ||
| # Enable unprivileged uffd (Ubuntu defaults to 0) | ||
| echo 1 | sudo tee /proc/sys/vm/unprivileged_userfaultfd | ||
|
|
||
| # Enable hugepages (256 × 2MB = 512MB). | ||
| # Tests that need more hugepages than available will skip gracefully. | ||
| sudo mkdir -p /mnt/hugepages | ||
| sudo mount -t hugetlbfs none /mnt/hugepages | ||
| echo 256 | sudo tee /proc/sys/vm/nr_hugepages | ||
|
|
||
| # Install extra kernel modules (nbd is not in base modules on GitHub-hosted runners) | ||
| sudo apt-get update | ||
| sudo apt-get install -y linux-modules-extra-$(uname -r) | ||
| sudo modprobe nbd nbds_max=256 | ||
|
|
||
| # Disable inotify watching of change events for NBD devices | ||
| echo 'ACTION=="add|change", KERNEL=="nbd*", OPTIONS:="nowatch"' | sudo tee /etc/udev/rules.d/97-nbd-device.rules | ||
| sudo udevadm control --reload-rules | ||
| sudo udevadm trigger | ||
| if: matrix.package == 'packages/orchestrator' | ||
|
|
||
| - name: Run tests that require sudo | ||
| working-directory: ${{ matrix.package }} | ||
| run: sudo -E `which go` test -race -v ${{ matrix.test_path }} | ||
| if: matrix.sudo == true | ||
|
|
||
| - name: Run tests | ||
| working-directory: ${{ matrix.package }} | ||
| run: go test -race -v ${{ matrix.test_path }} | ||
| if: matrix.sudo == false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.