Skip to content

Commit ed679cc

Browse files
committed
Add README for performance regression
1 parent b954b7e commit ed679cc

File tree

1 file changed

+73
-0
lines changed

1 file changed

+73
-0
lines changed

tests/perf-regression/README.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# About Performance Regression — Developer Guide
2+
3+
The performance regression framework is designed to detect regressions in
4+
ZkProgram execution (compile, prove, verify) and compile-time performance of
5+
constraint systems (CS) and zkApps.
6+
7+
## Overview
8+
9+
- **ZkProgram regression** — handled by
10+
[src/lib/testing/perf-regression.ts](../../src/lib/testing/perf-regression.ts),
11+
measuring individual ZkPrograms (e.g., SHA256, ECDSA).
12+
- **CS & zkApp regression** — handled by
13+
[tests/perf-regression/perf-regression.ts](./perf-regression.ts), measuring
14+
compile-time performance for zkApp examples.
15+
16+
Both use the same JSON file,
17+
[tests/perf-regression/perf-regression.json](./perf-regression.json) and is
18+
automatically used by CI to compare new performance data against existing
19+
baselines and should only be dumped by triggering the CI workflow (see
20+
[CI Integration](#ci-integration)).
21+
22+
## Modes
23+
24+
- `--check`: Compares current benchmarks with the stored baseline to detect
25+
regressions. Used by CI in PRs.
26+
- `--dump`: Regenerates the baseline with fresh values, replacing the JSON data.
27+
Used only via GitHub runners for consistency.
28+
29+
## Usage
30+
31+
### Dump all examples (local)
32+
33+
```bash
34+
npm run regression:dump-perf
35+
```
36+
37+
### Check all examples (local)
38+
39+
```bash
40+
npm run regression:check-perf
41+
```
42+
43+
## CI Integration
44+
45+
- **PR checks**: Automatically run in `--check` mode. CI compares results
46+
against the committed baseline.
47+
- **Baseline updates**: Manually trigger the workflow
48+
[dump-perf-baseline.yml](../../.github/workflows/dump-perf-baseline.yml) to
49+
run in `--dump`, regenerate the baseline, and commit it to the same branch.
50+
- To trigger the CI workflow manually:
51+
52+
```bash
53+
gh workflow run dump-perf-baseline.yml -r $(git rev-parse --abbrev-ref HEAD)
54+
```
55+
56+
- **Note:** Baselines must be generated by CI to avoid local machine variance.
57+
That's why you should NOT commit local dumps.
58+
59+
## Local vs CI
60+
61+
| Environment | Mode | Description |
62+
| -------------------- | -------------------- | --------------------------------------------------------------------------------------------- |
63+
| **Local** | `--dump` / `--check` | Developers can test performance locally. Results vary by hardware; do not commit local dumps. |
64+
| **CI (PR)** | `--check` | Ensures performance consistency against the repo’s baseline. |
65+
| **CI (Manual Dump)** | `--dump` | Refreshes the baseline via GitHub runners and commits updates. |
66+
67+
## Notes
68+
69+
- To see which examples are covered, check
70+
[perf-regression.sh](./perf-regression.sh).
71+
72+
- Without explicit `--dump` or `--check`, the framework behaves like `tic/toc`,
73+
enabling seamless use in standard runs.

0 commit comments

Comments
 (0)