Skip to content

[CI] build once for CPU and GPU #3434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
qwang98 wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

[CI] build once for CPU and GPU #3434

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e13e6a4
make init-testing-instance a function
qwang98 Nov 7, 2025
9d5c10e
just build once
qwang98 Nov 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/actions/init-testing-instance/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,22 @@
name: "Init testing instance"
description: "Initialises a testing instance with all required tools and fetches the precomputed tests archive named `tests_archive_cpu`"
description: "Initialises a testing instance with all required tools and fetches a precomputed tests archive"

inputs:
artifact_name:
description: "Name of the uploaded artifact to download (e.g. tests_archive_cpu)"
required: false
default: tests_archive

runs:
using: "composite"
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Download build artifacts (CPU)
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: tests_archive_cpu
name: ${{ inputs.artifact_name }}
- name: Install Rust toolchain nightly-2025-05-14 (with clippy and rustfmt)
shell: bash
run: rustup toolchain install nightly-2025-05-14 --component clippy,rustfmt,rust-src
Expand Down
92 changes: 34 additions & 58 deletions .github/workflows/pr-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,60 +18,8 @@ env:
POWDR_OPENVM_SEGMENT_DELTA: 50000

jobs:
build_cpu:
runs-on: warp-ubuntu-2404-x64-8x

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
- name: ⚡ Restore rust cache
id: cache
uses: WarpBuilds/cache/restore@v1
with:
path: |
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
Cargo.lock
key: ${{ runner.os }}-cargo-pr-tests
- name: Date of the restored cache
run: cat target/cache-build-date.txt
continue-on-error: true
- name: Check out cache commit state and update mtime accordingly.
run: git checkout "$(cat target/cache-commit-hash.txt || echo 'f02fd626e2bb9e46a22ea1cda96b4feb5c6bda43')" && git ls-files -z | xargs -0 -n1 touch -d "Fri, 18 Apr 2025 03:30:58 +0000" && git checkout HEAD@{1}

##### The block below is shared between cache build and PR build workflows #####
- name: Install Rust toolchain nightly-2025-10-01 (with clippy and rustfmt)
run: rustup toolchain install nightly-2025-10-01 --component clippy,rustfmt
- name: Install Rust toolchain 1.88 (stable)
run: rustup toolchain install 1.88
- name: Set cargo to perform shallow clones
run: echo "CARGO_NET_GIT_FETCH_WITH_CLI=true" >> $GITHUB_ENV
- name: Format
run: cargo fmt --all --check --verbose
- name: Cargo check with Rust 1.88 (default features)
run: cargo +1.88 check --all-targets
- name: Lint no default features
run: cargo clippy --all --all-targets --features metrics --profile pr-tests --verbose -- -D warnings
- name: Build (CPU)
run: cargo build --all-targets --features metrics --all --profile pr-tests --verbose
###############################################################################

- uses: taiki-e/install-action@nextest
- name: Create tests archive (CPU)
run: cargo nextest archive --archive-file tests_cpu.tar.zst --cargo-profile pr-tests --workspace --no-default-features
- name: Upload build artifacts (CPU)
uses: actions/upload-artifact@v4
with:
name: tests_archive_cpu
path: |
tests_cpu.tar.zst

test_quick_cpu:
needs: build_cpu
needs: build
runs-on: ubuntu-24.04
strategy:
matrix:
Expand All @@ -90,11 +38,13 @@ jobs:
submodules: recursive
- name: Init testing instance
uses: ./.github/actions/init-testing-instance
with:
artifact_name: tests_archive_cpu
- name: Run default tests
run: cargo nextest run --archive-file tests_cpu.tar.zst --workspace-remap . --verbose --partition count:"${{ matrix.test }}"/7 --no-tests=warn

test_medium_cpu:
needs: build_cpu
needs: build
runs-on: warp-ubuntu-2404-x64-16x
strategy:
matrix:
Expand All @@ -110,11 +60,13 @@ jobs:
submodules: recursive
- name: Init testing instance
uses: ./.github/actions/init-testing-instance
with:
artifact_name: tests_archive_cpu
- name: Run medium tests (ignored tests except large ones)
run: cargo nextest run --archive-file tests_cpu.tar.zst --workspace-remap . --verbose --partition count:"${{ matrix.test }}"/7 --test-threads=4 -E 'not (test(_large))' --run-ignored only --no-tests=warn

test_large_cpu:
needs: build_cpu
needs: build
runs-on: warp-ubuntu-2404-x64-32x

steps:
Expand All @@ -123,6 +75,8 @@ jobs:
submodules: recursive
- name: Init testing instance
uses: ./.github/actions/init-testing-instance
with:
artifact_name: tests_archive_cpu
- name: Run large tests
run: cargo nextest run --archive-file tests_cpu.tar.zst --workspace-remap . --verbose -E 'test(_large)' --run-ignored only --no-tests=warn

Expand Down Expand Up @@ -177,7 +131,7 @@ jobs:
echo "export RPC_1=${{ secrets.RPC_1 }}" >> .env
MODE="compile" APC=10 PGO_TYPE="instruction" /usr/bin/time -v ./run.sh

build_gpu:
build:
runs-on: [self-hosted, GPU]

steps:
Expand All @@ -186,6 +140,7 @@ jobs:
fetch-depth: 0
submodules: recursive

##### The block below is shared between cache build and PR build workflows #####
- name: Install Rust toolchain nightly-2025-10-01 (with clippy and rustfmt)
run: rustup toolchain install nightly-2025-10-01 --component clippy,rustfmt
- name: Install Rust toolchain 1.88 (stable)
Expand All @@ -197,6 +152,27 @@ jobs:
- name: Cargo check with Rust 1.88 (all features)
run: cargo +1.88 check --all-targets
- name: Lint no default features
run: cargo clippy --all --all-targets --no-default-features --profile pr-tests --verbose -- -D warnings
- name: Lint all features
run: cargo clippy --all --all-targets --all-features --profile pr-tests --verbose -- -D warnings
- name: Build (GPU)
run: cargo build --all-targets --all-features --all --profile pr-tests --verbose
- name: Build
run: cargo build --all-targets --all-features --all --profile pr-tests --verbose
###############################################################################

- uses: taiki-e/install-action@nextest
- name: Create tests archive (CPU)
run: cargo nextest archive --archive-file tests_cpu.tar.zst --cargo-profile pr-tests --workspace --no-default-features
- name: Upload build artifacts (CPU)
uses: actions/upload-artifact@v4
with:
name: tests_archive_cpu
path: |
tests_cpu.tar.zst
- name: Create tests archive (GPU)
run: cargo nextest archive --archive-file tests_gpu.tar.zst --cargo-profile pr-tests --workspace --features cuda
- name: Upload build artifacts (GPU)
uses: actions/upload-artifact@v4
with:
name: tests_archive_gpu
path: |
tests_gpu.tar.zst
Loading