|
| 1 | +# About Performance Regression — Developer Guide |
| 2 | + |
| 3 | +The performance regression framework is designed to detect regressions in |
| 4 | +ZkProgram execution (compile, prove, verify) and compile-time performance of |
| 5 | +constraint systems (CS) and zkApps. |
| 6 | + |
| 7 | +## Overview |
| 8 | + |
| 9 | +- **ZkProgram regression** — handled by |
| 10 | + [src/lib/testing/perf-regression.ts](../../src/lib/testing/perf-regression.ts), |
| 11 | + measuring individual ZkPrograms (e.g., SHA256, ECDSA). |
| 12 | +- **CS & zkApp regression** — handled by |
| 13 | + [tests/perf-regression/perf-regression.ts](./perf-regression.ts), measuring |
| 14 | + compile-time performance for zkApp examples. |
| 15 | + |
| 16 | +Both use the same JSON file, |
| 17 | +[tests/perf-regression/perf-regression.json](./perf-regression.json) and is |
| 18 | +automatically used by CI to compare new performance data against existing |
| 19 | +baselines and should only be dumped by triggering the CI workflow (see |
| 20 | +[CI Integration](#ci-integration)). |
| 21 | + |
| 22 | +## Modes |
| 23 | + |
| 24 | +- `--check`: Compares current benchmarks with the stored baseline to detect |
| 25 | + regressions. Used by CI in PRs. |
| 26 | +- `--dump`: Regenerates the baseline with fresh values, replacing the JSON data. |
| 27 | + Used only via GitHub runners for consistency. |
| 28 | + |
| 29 | +## Usage |
| 30 | + |
| 31 | +### Dump all examples (local) |
| 32 | + |
| 33 | +```bash |
| 34 | +npm run regression:dump-perf |
| 35 | +``` |
| 36 | + |
| 37 | +### Check all examples (local) |
| 38 | + |
| 39 | +```bash |
| 40 | +npm run regression:check-perf |
| 41 | +``` |
| 42 | + |
| 43 | +## CI Integration |
| 44 | + |
| 45 | +- **PR checks**: Automatically run in `--check` mode. CI compares results |
| 46 | + against the committed baseline. |
| 47 | +- **Baseline updates**: Manually trigger the workflow |
| 48 | + [dump-perf-baseline.yml](../../.github/workflows/dump-perf-baseline.yml) to |
| 49 | + run in `--dump`, regenerate the baseline, and commit it to the same branch. |
| 50 | + - To trigger the CI workflow manually: |
| 51 | + |
| 52 | + ```bash |
| 53 | + gh workflow run dump-perf-baseline.yml -r $(git rev-parse --abbrev-ref HEAD) |
| 54 | + ``` |
| 55 | + |
| 56 | + - **Note:** Baselines must be generated by CI to avoid local machine variance. |
| 57 | + That's why you should NOT commit local dumps. |
| 58 | +
|
| 59 | +## Local vs CI |
| 60 | +
|
| 61 | +| Environment | Mode | Description | |
| 62 | +| -------------------- | -------------------- | --------------------------------------------------------------------------------------------- | |
| 63 | +| **Local** | `--dump` / `--check` | Developers can test performance locally. Results vary by hardware; do not commit local dumps. | |
| 64 | +| **CI (PR)** | `--check` | Ensures performance consistency against the repo’s baseline. | |
| 65 | +| **CI (Manual Dump)** | `--dump` | Refreshes the baseline via GitHub runners and commits updates. | |
| 66 | +
|
| 67 | +## Notes |
| 68 | +
|
| 69 | +- To see which examples are covered, check |
| 70 | + [perf-regression.sh](./perf-regression.sh). |
| 71 | +
|
| 72 | +- Without explicit `--dump` or `--check`, the framework behaves like `tic/toc`, |
| 73 | + enabling seamless use in standard runs. |
0 commit comments