No seeding

Author: ntucker

examples/benchmark-react/README.md

@@ -14,7 +14,7 @@ The repo has two benchmark suites:
 - **What we measure:** Wall-clock time from triggering an action (e.g. `mount(100)` or `updateAuthor('author-0')`) until the harness sets `data-bench-complete` (after two `requestAnimationFrame` callbacks). Optionally we also record React Profiler commit duration and, with `BENCH_TRACE=true`, Chrome trace duration.
 - **Why:** Normalized caching should show wins on shared-entity updates (one store write, many components update), ref stability (fewer new object references), and derived-view memoization (`Query` schema avoids re-sorting when entities haven't changed). See [js-framework-benchmark "How the duration is measured"](https://github.com/krausest/js-framework-benchmark/wiki/How-the-duration-is-measured) for a similar timeline-based approach.
 - **Statistical:** Warmup runs are discarded; we report median and 95% CI. Libraries are interleaved per round to reduce environmental variance.
-- **No CPU throttling:** Runs at native speed with more samples (3 warmup + 30 measurement locally, 15 in CI) for statistical significance rather than artificial slowdown.
+- **No CPU throttling:** Runs at native speed with more samples for statistical significance rather than artificial slowdown. Small (cheap) scenarios use 3 warmup + 15 measurement runs locally (10 in CI); large (expensive) scenarios use 1 warmup + 4 measurement runs.
 
 ## Scenario categories
 
@@ -133,6 +133,44 @@ Regressions >5% on stable scenarios or >15% on volatile scenarios are worth inve
    - `BENCH_PORT=<port>` — port for `preview` server and bench runner (default `5173`)
    - `BENCH_BASE_URL=<url>` — full base URL override (takes precedence over `BENCH_PORT`)
 
+4. **Filtering scenarios**
+
+   The runner supports CLI flags (with env var fallbacks) to select a subset of scenarios:
+
+   | CLI flag | Env var | Description |
+   |---|---|---|
+   | `--lib <names>` | `BENCH_LIB` | Comma-separated library names (e.g. `data-client,swr`) |
+   | `--size <small\|large>` | `BENCH_SIZE` | Run only `small` (cheap, full rigor) or `large` (expensive, reduced runs) scenarios |
+   | `--action <group\|action>` | `BENCH_ACTION` | Filter by action group (`mount`, `update`, `mutation`, `memory`) or exact action name |
+   | `--scenario <pattern>` | `BENCH_SCENARIO` | Substring filter on scenario name |
+
+   CLI flags take precedence over env vars. Examples:
+
+   ```bash
+   yarn bench --lib data-client                # only data-client
+   yarn bench --size small                      # only cheap scenarios (full warmup/measurement)
+   yarn bench --action mount                    # mount, mountSortedView, bulkIngest
+   yarn bench --action update --lib swr         # update scenarios for swr only
+   yarn bench --scenario sorted-view            # only sorted-view scenarios
+   ```
+
+   Convenience scripts:
+
+   ```bash
+   yarn bench:small       # --size small
+   yarn bench:large       # --size large
+   yarn bench:dc          # --lib data-client
+   ```
+
+5. **Scenario sizes**
+
+   Scenarios are classified as `small` or `large` based on their cost:
+
+   - **Small** (3 warmup + 15 measurement): `mount-100-items`, `update-single-entity`, `update-shared-author-duration`, `ref-stability-*`, `optimistic-update`, `invalidate-and-resolve`, `create-item`, `delete-item`
+   - **Large** (1 warmup + 4 measurement): `mount-500-items`, `update-shared-author-500-mounted`, `update-shared-author-2000-mounted`, `memory-mount-unmount-cycle`, `update-shared-author-with-network`, `bulk-ingest-500`, `sorted-view-mount-500`, `sorted-view-update-entity`
+
+   When running all scenarios (`yarn bench`), each group runs with its own warmup/measurement count. Use `--size` to run only one group.
+
 ## Output
 
 The runner prints a JSON array in `customSmallerIsBetter` format (name, unit, value, range) to stdout. In CI this is written to `react-bench-output.json` and sent to the benchmark action.