Skip to content

Commit bd89831

Browse files
committed
ci(bench): run on changes under bench/, in a parallel per-platform matrix
之前只能手动触发,于是这套东西**从来没有在真实 CI 上跑过** —— `bench.yml` 一度连合法 YAML 都不是,而没有任何东西会告诉你。 改为 **path-scoped**:`bench/**` 或本文件变动时触发(push 与 pull_request 都是), 手动触发保留。这样它在「我是不是改坏了 harness / 这次改动有没有挪动数字」 真正被问到的时候跑,而不在每个无关 PR 上跑。矩阵按平台并行、`fail-fast: false` —— 一个平台缺某个引擎,不该把其他平台已经采到的数据取消掉。 ⚠️ **每一个 `inputs.*` 都补了兜底值**:push/pull_request 触发时它们**全是空的**, 而空的 `--engines` 会让 bench 什么都不跑、然后报告成功 —— 那正是这条 workflow 想避免的那种「绿得没有意义」。默认尺寸用 `smoke`, 因为路径触发是回归检查,不是发布测量。
1 parent 5306c42 commit bd89831

1 file changed

Lines changed: 37 additions & 14 deletions

File tree

.github/workflows/bench.yml

Lines changed: 37 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,36 @@
11
name: bench
22

3-
# Build-engine benchmark. MANUAL TRIGGER ONLY, and that is a design decision:
3+
# Build-engine benchmark. Runs on changes under `bench/` and on demand.
4+
#
5+
# WHY IT IS PATH-SCOPED RATHER THAN ON EVERY PUSH:
46
#
57
# * it is heavy — a full matrix compiles the same fixture six ways per platform
68
# * it is noisy — cloud runners are shared, and the CPU model changes under you
79
# * it asserts nothing — no threshold, no pass/fail on timings
810
#
9-
# Attaching it to every PR would drown the signal it exists to produce, and a
10-
# timing threshold on a shared runner turns normal variance into red crosses that
11-
# people learn to ignore. Results are uploaded as artifacts; comparing them is a
12-
# human act.
11+
# So it fires when the SUITE itself changes, where the question "did I break the
12+
# harness / did this shift the numbers" is actually being asked, and stays off
13+
# every unrelated PR. A timing threshold on a shared runner would turn normal
14+
# variance into red crosses people learn to ignore, so there is none: results
15+
# are uploaded as artifacts and comparing them is a human act.
16+
#
17+
# The matrix runs platforms in parallel and `fail-fast: false`, because one
18+
# platform missing an engine must not cancel the data from the others.
1319
#
1420
# See bench/README.md for the measurement contract before quoting any number.
1521

1622
on:
23+
# Changes to the suite itself — including its own tests and the project
24+
# descriptions it measures. Not `paths: ['**']`: the point is to fire where
25+
# the numbers can move, not on every commit.
26+
push:
27+
paths:
28+
- 'bench/**'
29+
- '.github/workflows/bench.yml'
30+
pull_request:
31+
paths:
32+
- 'bench/**'
33+
- '.github/workflows/bench.yml'
1734
workflow_dispatch:
1835
inputs:
1936
engines:
@@ -73,7 +90,10 @@ jobs:
7390
shell: bash
7491
run: |
7592
set -euo pipefail
76-
want="${{ inputs.platforms }}"
93+
# `inputs.*` is empty on a push/pull_request trigger, so every input needs
94+
# a fallback here — an empty `platforms` would otherwise plan an empty
95+
# matrix and the job would silently do nothing.
96+
want="${{ inputs.platforms || 'linux,macos,windows' }}"
7797
entries=()
7898
case ",$want," in *,linux,*) entries+=('{"os":"ubuntu-24.04","name":"linux"}');; esac
7999
case ",$want," in *,macos,*) entries+=('{"os":"macos-14","name":"macos"}');; esac
@@ -137,16 +157,19 @@ jobs:
137157
# positive number. Passing raw numbers unconditionally would make every
138158
# run's size an accident of this file rather than a named, comparable
139159
# workload — and --preset must come first so the overrides still win.
140-
args=( --preset "${{ inputs.preset }}" )
141-
[ "${{ inputs.units }}" -gt 0 ] 2>/dev/null && args+=( --units "${{ inputs.units }}" )
142-
[ "${{ inputs.fanin }}" -gt 0 ] 2>/dev/null && args+=( --fanin "${{ inputs.fanin }}" )
160+
# Every `inputs.*` needs a fallback: on a push/pull_request trigger
161+
# they are all EMPTY, and an empty --engines would run nothing while
162+
# still reporting success.
163+
args=( --preset "${{ inputs.preset || 'smoke' }}" )
164+
[ "${{ inputs.units || 0 }}" -gt 0 ] 2>/dev/null && args+=( --units "${{ inputs.units }}" )
165+
[ "${{ inputs.fanin || 0 }}" -gt 0 ] 2>/dev/null && args+=( --fanin "${{ inputs.fanin }}" )
143166
"$BENCH" \
144-
--engines '${{ inputs.engines }}' \
145-
--variants '${{ inputs.variants }}' \
146-
--scenarios '${{ inputs.scenarios }}' \
147-
--profile '${{ inputs.profile }}' \
167+
--engines '${{ inputs.engines || 'mcpp,cmake,xmake,meson,bazel' }}' \
168+
--variants '${{ inputs.variants || 'headers,modules,modules-impl' }}' \
169+
--scenarios '${{ inputs.scenarios || 'cold,noop,touch-hub,touch-leaf,edit-body,edit-comment' }}' \
170+
--profile '${{ inputs.profile || 'release' }}' \
148171
"${args[@]}" \
149-
--runs '${{ inputs.runs }}' \
172+
--runs '${{ inputs.runs || 0 }}' \
150173
--work "$RUNNER_TEMP/bench-work" \
151174
--out "bench-${{ matrix.name }}.json"
152175

0 commit comments

Comments
 (0)