Skip to content

support multiple labels for manual bench workflows #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
richkcho merged 1 commit into from
Nov 28, 2025
Merged

support multiple labels for manual bench workflows #64

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions .github/workflows/bench-mpmc.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
description: 'Pull request number to benchmark'
required: true
type: number
runner_label:
description: 'Label for the self-hosted benchmark runner'
runner_labels:
description: 'Comma-separated labels for the self-hosted benchmark runner (e.g. "bench,linux")'
required: true
type: string

Expand All @@ -21,9 +21,24 @@
CARGO_TARGET_DIR: ${{ github.workspace }}/.bench-target

jobs:
prepare-runner-labels:
runs-on: ubuntu-latest
outputs:
runner_labels_list: ${{ steps.build-labels.outputs.runner_labels_list }}
steps:
- name: Build runner label list
id: build-labels
shell: bash
run: |
labels="${{ inputs.runner_labels }}"
labels="${labels// /}" # strip spaces
labels="${labels//,/\",\"}" # turn comma list into quoted csv
echo "runner_labels_list=[\"self-hosted\"${labels:+,\"$labels\"}]" >> "$GITHUB_OUTPUT"

benchmark:
name: Bench PR #${{ inputs.pr_number }}
runs-on: [self-hosted, '${{ inputs.runner_label }}']
needs: prepare-runner-labels
runs-on: ${{ fromJson(needs.prepare-runner-labels.outputs.runner_labels_list) }}
outputs:
pr_number: ${{ steps.parse-pr.outputs.pr_number }}

Expand All @@ -47,7 +62,7 @@
script: |
// Use the parsed PR number integer
const prNumber = ${{ steps.parse-pr.outputs.pr_number }};
const body = `Benchmark run started on runner label \`${{ inputs.runner_label }}\`...`;
const body = `Benchmark run started on runner label(s) \`${{ inputs.runner_labels }}\`...`;
const { data: comment } = await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
Expand Down Expand Up @@ -81,7 +96,7 @@
- name: Cache Rust build artifacts
uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
shared-key: bench-${{ inputs.runner_label }}

Check failure on line 99 in .github/workflows/bench-mpmc.yml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

Raw Output:
e:.github/workflows/bench-mpmc.yml:99:33: property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]
workspaces: |
.
main-baseline
Expand All @@ -92,7 +107,7 @@
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ env.CARGO_TARGET_DIR }}/criterion
key: criterion-main-${{ inputs.runner_label }}-${{ steps.main_sha.outputs.sha }}

Check failure on line 110 in .github/workflows/bench-mpmc.yml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

Raw Output:
e:.github/workflows/bench-mpmc.yml:110:35: property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

- name: Run baseline on main (only if missing)
if: steps.baseline-cache.outputs.cache-hit != 'true'
Expand All @@ -104,7 +119,7 @@
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ env.CARGO_TARGET_DIR }}/criterion
key: criterion-main-${{ inputs.runner_label }}-${{ steps.main_sha.outputs.sha }}

Check failure on line 122 in .github/workflows/bench-mpmc.yml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

Raw Output:
e:.github/workflows/bench-mpmc.yml:122:35: property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

- name: Run benchmarks for PR against main baseline
id: bench-pr
Expand Down
23 changes: 19 additions & 4 deletions .github/workflows/bench-spsc.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
description: 'Pull request number to benchmark'
required: true
type: number
runner_label:
description: 'Label for the self-hosted benchmark runner'
runner_labels:
description: 'Comma-separated labels for the self-hosted benchmark runner (e.g. "bench,linux")'
required: true
type: string

Expand All @@ -21,9 +21,24 @@
CARGO_TARGET_DIR: ${{ github.workspace }}/.bench-target

jobs:
prepare-runner-labels:
runs-on: ubuntu-latest
outputs:
runner_labels_list: ${{ steps.build-labels.outputs.runner_labels_list }}
steps:
- name: Build runner label list
id: build-labels
shell: bash
run: |
labels="${{ inputs.runner_labels }}"
labels="${labels// /}" # strip spaces
labels="${labels//,/\",\"}" # turn comma list into quoted csv
echo "runner_labels_list=[\"self-hosted\"${labels:+,\"$labels\"}]" >> "$GITHUB_OUTPUT"

benchmark:
name: Bench PR #${{ inputs.pr_number }}
runs-on: [self-hosted, '${{ inputs.runner_label }}']
needs: prepare-runner-labels
runs-on: ${{ fromJson(needs.prepare-runner-labels.outputs.runner_labels_list) }}
outputs:
pr_number: ${{ steps.parse-pr.outputs.pr_number }}

Expand All @@ -47,7 +62,7 @@
script: |
// Use the parsed PR number integer
const prNumber = ${{ steps.parse-pr.outputs.pr_number }};
const body = `Benchmark run started on runner label \`${{ inputs.runner_label }}\`...`;
const body = `Benchmark run started on runner label(s) \`${{ inputs.runner_labels }}\`...`;
const { data: comment } = await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
Expand Down Expand Up @@ -81,7 +96,7 @@
- name: Cache Rust build artifacts
uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
shared-key: bench-${{ inputs.runner_label }}

Check failure on line 99 in .github/workflows/bench-spsc.yml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

Raw Output:
e:.github/workflows/bench-spsc.yml:99:33: property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]
workspaces: |
.
main-baseline
Expand All @@ -92,7 +107,7 @@
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ env.CARGO_TARGET_DIR }}/criterion
key: criterion-main-${{ inputs.runner_label }}-${{ steps.main_sha.outputs.sha }}

Check failure on line 110 in .github/workflows/bench-spsc.yml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

Raw Output:
e:.github/workflows/bench-spsc.yml:110:35: property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

- name: Run baseline on main (only if missing)
if: steps.baseline-cache.outputs.cache-hit != 'true'
Expand All @@ -104,7 +119,7 @@
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ env.CARGO_TARGET_DIR }}/criterion
key: criterion-main-${{ inputs.runner_label }}-${{ steps.main_sha.outputs.sha }}

Check failure on line 122 in .github/workflows/bench-spsc.yml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

Raw Output:
e:.github/workflows/bench-spsc.yml:122:35: property "runner_label" is not defined in object type {pr_number: number; runner_labels: string} [expression]

- name: Run benchmarks for PR against main baseline
id: bench-pr
Expand Down
Loading