Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .cursor/rules/benchmarking.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Use this mapping when deciding which React benchmark scenarios are relevant to a
- Relevant for: `@data-client/react` hooks, `@data-client/core` store initialization
- All libraries

- **Update propagation** (`update-single-entity`, `update-shared-user-500-mounted`, `update-shared-user-10000-mounted`)
- **Update propagation** (`update-entity`, `update-user`, `update-user-10000`)
- Exercises: store update → React rerender → DOM mutation
- Relevant for: `@data-client/core` dispatch/reducer, `@data-client/react` subscription/selector
- All libraries (normalization advantage shows with shared user at scale)
Expand All @@ -41,7 +41,7 @@ Use this mapping when deciding which React benchmark scenarios are relevant to a
- Relevant for: `@data-client/normalizr` denormalize memoization, Entity identity
- All libraries (data-client should show fewest changed refs)

- **Sorted/derived view** (`sorted-view-mount-500`, `sorted-view-update-entity`)
- **Sorted/derived view** (`getlist-500-sorted`, `update-entity-sorted`)
- Exercises: `Query` schema memoization via `useQuery` (data-client) vs `useMemo` sort (competitors)
- Relevant for: `@data-client/endpoint` Query, `@data-client/normalizr` MemoCache, `@data-client/react` useQuery
- All libraries
Expand All @@ -58,8 +58,8 @@ Use this mapping when deciding which React benchmark scenarios are relevant to a

| Category | Scenarios | Typical run-to-run spread |
|---|---|---|
| **Stable** | `getlist-*`, `update-single-entity`, `ref-stability-*`, `sorted-view-mount-*` | 2–5% |
| **Moderate** | `update-shared-user-*`, `sorted-view-update-*` | 5–10% |
| **Stable** | `getlist-*`, `update-entity`, `ref-stability-*` | 2–5% |
| **Moderate** | `update-user-*`, `update-entity-sorted` | 5–10% |
| **Volatile** | `memory-mount-unmount-cycle`, `startup-*`, `(react commit)` suffixes | 10–25% |

Regressions >5% on stable scenarios or >15% on volatile scenarios are worth investigating.
Expand Down
2 changes: 1 addition & 1 deletion examples/benchmark-react/.babelrc.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
const options = { polyfillMethod: false };
if (process.env.REACT_COMPILER === 'true') {
if (process.env.REACT_COMPILER !== 'false') {
options.reactCompiler = {};
}

Expand Down
36 changes: 18 additions & 18 deletions examples/benchmark-react/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ The repo has two benchmark suites:
**Hot path (CI)**

- **Get list** (`getlist-100`, `getlist-500`) — Time to show a ListView component that auto-fetches 100 or 500 issues from the list endpoint, then renders (unit: ms). Exercises the full fetch + normalization + render pipeline.
- **Update single entity** (`update-single-entity`) — Time to update one issue and propagate to the UI (unit: ms).
- **Update shared user (scaling)** (`update-shared-user-500-mounted`, `update-shared-user-10000-mounted`) — Update one shared user with 500 or 10,000 mounted issues to test subscriber scaling. Normalized cache: one store update, all views of that user update.
- **Get list sorted** (`getlist-500-sorted`) — Mount 500 issues through a sorted/derived view. data-client uses `useQuery(sortedIssuesQuery)` with `Query` schema memoization; competitors use `useMemo` + sort.
- **Update entity** (`update-entity`) — Time to update one issue and propagate to the UI (unit: ms).
- **Update entity sorted** (`update-entity-sorted`) — After mounting a sorted view, update one entity. data-client's `Query` memoization avoids re-sorting when sort keys are unchanged.
- **Update user (scaling)** (`update-user`, `update-user-10000`) — Update one shared user with 1,000 or 10,000 mounted issues to test subscriber scaling. Normalized cache: one store update, all views of that user update.
- **Ref-stability** (`ref-stability-issue-changed`, `ref-stability-user-changed`) — Count of components that received a **new** object reference after an update (unit: count; smaller is better). Normalization keeps referential equality for unchanged entities.
- **Sorted view mount** (`sorted-view-mount-500`) — Mount 500 issues through a sorted/derived view. data-client uses `useQuery(sortedIssuesQuery)` with `Query` schema memoization; competitors use `useMemo` + sort.
- **Sorted view update** (`sorted-view-update-entity`) — After mounting a sorted view, update one entity. data-client's `Query` memoization avoids re-sorting when sort keys are unchanged.
- **Invalidate and resolve** (`invalidate-and-resolve`) — data-client only; invalidates a cached endpoint and immediately re-resolves. Measures Suspense boundary round-trip.

**With network (local comparison)**
Expand All @@ -55,17 +55,17 @@ These are approximate values to help calibrate expectations. Exact numbers vary
| Scenario | data-client | tanstack-query | swr |
|---|---|---|---|
| `getlist-100` | ~similar | ~similar | ~similar |
| `update-shared-user-500-mounted` | Low (one store write propagates) | Higher (list refetch) | Higher (list refetch) |
| `update-user` | Low (one store write propagates) | Higher (list refetch) | Higher (list refetch) |
| `ref-stability-issue-changed` (100 mounted) | ~1 changed | ~100 changed (list refetch) | ~100 changed (list refetch) |
| `ref-stability-user-changed` (100 mounted) | ~5 changed | ~100 changed (list refetch) | ~100 changed (list refetch) |
| `sorted-view-update-entity` | Fast (Query memoization skips re-sort) | Re-sorts on every issue change | Re-sorts on every issue change |
| `update-entity-sorted` | Fast (Query memoization skips re-sort) | Re-sorts on every issue change | Re-sorts on every issue change |

## Expected variance

| Category | Scenarios | Typical run-to-run spread |
|---|---|---|
| **Stable** | `getlist-*`, `update-single-entity`, `ref-stability-*`, `sorted-view-mount-*` | 2-5% |
| **Moderate** | `update-shared-user-*`, `sorted-view-update-*` | 5-10% |
| **Stable** | `getlist-*`, `update-entity`, `ref-stability-*` | 2-5% |
| **Moderate** | `update-user-*`, `update-entity-sorted` | 5-10% |
| **Volatile** | `memory-mount-unmount-cycle`, `startup-*`, `(react commit)` suffixes | 10-25% |

Regressions >5% on stable scenarios or >15% on volatile scenarios are worth investigating.
Expand Down Expand Up @@ -107,24 +107,24 @@ Regressions >5% on stable scenarios or >15% on volatile scenarios are worth inve

Or from repo root after a build: start preview in one terminal, then in another run `yarn workspace example-benchmark-react bench`.

3. **With React Compiler**
3. **Without React Compiler**

To measure the impact of React Compiler, build and bench with it enabled:
The default build includes React Compiler. To measure impact without it:

```bash
cd examples/benchmark-react
yarn build:compiler # builds with babel-plugin-react-compiler
yarn build:no-compiler # builds without babel-plugin-react-compiler
yarn preview &
sleep 5
yarn bench:compiler # labels results with [compiler] suffix
yarn bench:no-compiler # labels results with [no-compiler] suffix
```

Or as a single command: `yarn bench:run:compiler`.
Or as a single command: `yarn bench:run:no-compiler`.

Results are labelled `[compiler]` so you can compare side-by-side with a normal run by loading both JSON files into the report viewer's history feature.
Results are labelled `[no-compiler]` so you can compare side-by-side with the default run by loading both JSON files into the report viewer's history feature.

You can also set the env vars directly for custom combinations:
- `REACT_COMPILER=true` — enables the Babel plugin at build time
Env vars for custom combinations:
- `REACT_COMPILER=false` — disables the Babel plugin at build time
- `BENCH_LABEL=<tag>` — appends `[<tag>]` to all result names at bench time
- `BENCH_PORT=<port>` — port for `preview` server and bench runner (default `5173`)
- `BENCH_BASE_URL=<url>` — full base URL override (takes precedence over `BENCH_PORT`)
Expand Down Expand Up @@ -163,8 +163,8 @@ Regressions >5% on stable scenarios or >15% on volatile scenarios are worth inve

Scenarios are classified as `small` or `large` based on their cost:

- **Small** (3 warmup + 15 measurement): `getlist-100`, `update-single-entity`, `ref-stability-*`, `invalidate-and-resolve`, `unshift-item`, `delete-item`
- **Large** (1 warmup + 4 measurement): `getlist-500`, `update-shared-user-500-mounted`, `update-shared-user-10000-mounted`, `update-shared-user-with-network`, `sorted-view-mount-500`, `sorted-view-update-entity`
- **Small** (3 warmup + 15 measurement): `getlist-100`, `update-entity`, `ref-stability-*`, `invalidate-and-resolve`, `unshift-item`, `delete-item`
- **Large** (1 warmup + 4 measurement): `getlist-500`, `getlist-500-sorted`, `update-user`, `update-user-10000`, `update-entity-sorted`, `list-detail-switch`
- **Memory** (opt-in, 1 warmup + 3 measurement): `memory-mount-unmount-cycle` — run with `--action memory`

When running all scenarios (`yarn bench`), each group runs with its own warmup/measurement count. Use `--size` to run only one group.
Expand Down
7 changes: 7 additions & 0 deletions examples/benchmark-react/bench/runner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,13 @@ async function runScenario(
);
}

if (scenario.renderLimit != null) {
await (bench as any).evaluate(
(api: any, n: number) => api.setRenderLimit(n),
scenario.renderLimit,
);
}

const isMemory =
scenario.action === 'mountUnmountCycle' &&
scenario.resultMetric === 'heapDelta';
Expand Down
40 changes: 29 additions & 11 deletions examples/benchmark-react/bench/scenarios.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ interface BaseScenario {
onlyLibs?: string[];
/** Result is deterministic (zero variance); run exactly once with no warmup. */
deterministic?: boolean;
/** Cap DOM rendering to first N items while keeping all data in the store. */
renderLimit?: number;
}

const BASE_SCENARIOS: BaseScenario[] = [
Expand All @@ -73,17 +75,21 @@ const BASE_SCENARIOS: BaseScenario[] = [
size: 'large',
},
{
nameSuffix: 'update-single-entity',
nameSuffix: 'update-entity',
action: 'updateEntity',
args: [1],
category: 'hotPath',
mountCount: 1000,
renderLimit: 100,
},
{
nameSuffix: 'ref-stability-issue-changed',
action: 'updateEntity',
args: [1],
resultMetric: 'issueRefChanged',
category: 'hotPath',
mountCount: 1000,
renderLimit: 100,
deterministic: true,
},
{
Expand All @@ -92,14 +98,17 @@ const BASE_SCENARIOS: BaseScenario[] = [
args: ['user0'],
resultMetric: 'userRefChanged',
category: 'hotPath',
mountCount: 1000,
renderLimit: 100,
deterministic: true,
},
{
nameSuffix: 'update-shared-user-500-mounted',
nameSuffix: 'update-user',
action: 'updateUser',
args: ['user0'],
category: 'hotPath',
mountCount: 500,
mountCount: 1000,
renderLimit: 100,
size: 'large',
},
{
Expand All @@ -111,63 +120,71 @@ const BASE_SCENARIOS: BaseScenario[] = [
size: 'large',
},
{
nameSuffix: 'sorted-view-mount-500',
nameSuffix: 'getlist-500-sorted',
action: 'mountSortedView',
args: [500],
category: 'hotPath',
size: 'large',
},
{
nameSuffix: 'sorted-view-update-entity',
nameSuffix: 'update-entity-sorted',
action: 'updateEntity',
args: [1],
category: 'hotPath',
mountCount: 500,
mountCount: 1000,
renderLimit: 100,
preMountAction: 'mountSortedView',
size: 'large',
},
{
nameSuffix: 'list-detail-switch',
action: 'listDetailSwitch',
args: [500],
args: [1000],
category: 'hotPath',
size: 'large',
renderLimit: 100,
},
{
nameSuffix: 'update-shared-user-10000-mounted',
nameSuffix: 'update-user-10000',
action: 'updateUser',
args: ['user0'],
category: 'hotPath',
mountCount: 10000,
size: 'large',
renderLimit: 100,
},
{
nameSuffix: 'invalidate-and-resolve',
action: 'invalidateAndResolve',
args: [1],
category: 'hotPath',
mountCount: 1000,
renderLimit: 100,
onlyLibs: ['data-client'],
},
{
nameSuffix: 'unshift-item',
action: 'unshiftItem',
args: [],
category: 'hotPath',
mountCount: 100,
mountCount: 1000,
renderLimit: 100,
},
{
nameSuffix: 'delete-item',
action: 'deleteEntity',
args: [1],
category: 'hotPath',
mountCount: 100,
mountCount: 1000,
renderLimit: 100,
},
{
nameSuffix: 'move-item',
action: 'moveItem',
args: [1],
category: 'hotPath',
mountCount: 100,
mountCount: 1000,
renderLimit: 100,
preMountAction: 'initDoubleList',
},
];
Expand All @@ -188,6 +205,7 @@ export const SCENARIOS: Scenario[] = LIBRARIES.flatMap(lib =>
mountCount: base.mountCount,
preMountAction: base.preMountAction,
deterministic: base.deterministic,
renderLimit: base.renderLimit,
}),
),
);
1 change: 0 additions & 1 deletion examples/benchmark-react/bench/validate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ const BASE_URL =
process.env.BENCH_BASE_URL ??
`http://localhost:${process.env.BENCH_PORT ?? '5173'}`;

// react-window virtualises; keep test counts within the visible window
const TEST_ISSUE_COUNT = 20;

// ---------------------------------------------------------------------------
Expand Down
7 changes: 3 additions & 4 deletions examples/benchmark-react/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@
"description": "React rendering benchmark comparing @data-client/react against other data libraries",
"scripts": {
"build": "BROWSERSLIST_ENV=2026 webpack --mode=production",
"build:compiler": "BROWSERSLIST_ENV=2026 REACT_COMPILER=true webpack --mode=production",
"build:no-compiler": "BROWSERSLIST_ENV=2026 REACT_COMPILER=false webpack --mode=production",
"preview": "serve dist -l ${BENCH_PORT:-5173} --no-request-logging",
"bench": "npx tsx bench/runner.ts",
"bench:compiler": "BENCH_LABEL=compiler npx tsx bench/runner.ts",
"bench:no-compiler": "BENCH_LABEL=no-compiler npx tsx bench/runner.ts",
"bench:small": "npx tsx bench/runner.ts --size small",
"bench:large": "npx tsx bench/runner.ts --size large",
"bench:dc": "npx tsx bench/runner.ts --lib data-client",
"bench:run": "yarn build && (yarn preview &) && sleep 5 && yarn bench",
"bench:run:compiler": "yarn build:compiler && (yarn preview &) && sleep 5 && yarn bench:compiler",
"bench:run:no-compiler": "yarn build:no-compiler && (yarn preview &) && sleep 5 && yarn bench:no-compiler",
"validate": "npx tsx bench/validate.ts",
"validate:run": "yarn build && (yarn preview &) && sleep 5 && yarn validate"
},
Expand All @@ -28,7 +28,6 @@
"@tanstack/react-query": "5.62.7",
"react": "19.2.3",
"react-dom": "19.2.3",
"react-window": "^2.2.7",
"swr": "2.4.1"
},
"devDependencies": {
Expand Down
Loading
Loading