Skip to content
Open
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
4943251
demo: Add benchmark-react with normalization and ref-stability scenarios
ntucker Mar 9, 2026
eb261cb
more of the plan
ntucker Mar 9, 2026
ddd4c8f
More scenarios
ntucker Mar 9, 2026
4080f7b
more scenarios
ntucker Mar 9, 2026
678b359
add react compiler option
ntucker Mar 9, 2026
972621e
bugbot: dead code
ntucker Mar 9, 2026
4498e1c
No throttling
ntucker Mar 9, 2026
f3d5327
yarn lock
ntucker Mar 10, 2026
53e85ee
bugbot + fix test data client correctness
ntucker Mar 10, 2026
54a70e7
fair comparisons
ntucker Mar 10, 2026
944f3ba
ts 6
ntucker Mar 10, 2026
393050c
bugbot
ntucker Mar 10, 2026
b9317e5
update website types
ntucker Mar 10, 2026
1530cab
improve ci
ntucker Mar 10, 2026
7090705
internal: Bench runs concurrently
ntucker Mar 11, 2026
2606262
better abstractions
ntucker Mar 12, 2026
830199e
CRUD
ntucker Mar 13, 2026
60e8c10
virtualize
ntucker Mar 13, 2026
58e4f85
create adds to list; larger high end case
ntucker Mar 13, 2026
0253c11
No seeding
ntucker Mar 14, 2026
f5ea087
fix test conditions to be more accurate
ntucker Mar 14, 2026
6b15581
dynamic accuracy
ntucker Mar 14, 2026
5f34537
fix bench measurements
ntucker Mar 14, 2026
b0ab3fb
remove unneeded bench
ntucker Mar 14, 2026
ba040ef
fix measurement by eliminating paint timings from measurement
ntucker Mar 14, 2026
c9d5d60
increase scale, remove redundant
ntucker Mar 15, 2026
0281018
bench name updates
ntucker Mar 15, 2026
0de0caf
more realistic data
ntucker Mar 15, 2026
aed6a64
MutationObserver timeout silently fails without signaling completion
ntucker Mar 15, 2026
bb70f1e
Make sorted mount consistent
ntucker Mar 15, 2026
3cd8079
fix sorted-view-update-entity for some frameworks
ntucker Mar 15, 2026
d023380
change reporting
ntucker Mar 15, 2026
41fcaa9
move scenario
ntucker Mar 15, 2026
802d8a7
Init scenarios capture wrong react-commit-update measurement
ntucker Mar 15, 2026
ac6bcc1
Upgrade benchmark baseline from dual list to triple list
ntucker Mar 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 71 additions & 4 deletions .cursor/rules/benchmarking.mdc
Original file line number Diff line number Diff line change
@@ -1,12 +1,79 @@
---
description: Benchmarking guidelines using examples/benchmark (suite selection, commands, and what packages are exercised)
globs: examples/benchmark/**, .github/workflows/benchmark.yml, packages/normalizr/src/**, packages/core/src/**, packages/endpoint/src/schemas/**
description: Benchmarking guidelines using examples/benchmark and examples/benchmark-react (suite selection, commands, and what packages are exercised)
globs: examples/benchmark/**, examples/benchmark-react/**, .github/workflows/benchmark.yml, .github/workflows/benchmark-react.yml, packages/normalizr/src/**, packages/core/src/**, packages/endpoint/src/schemas/**, packages/react/src/**
alwaysApply: false
---

# Benchmarking (`@examples/benchmark`)
# Benchmarking

When working on performance investigations or changes that might impact performance, use **`@examples/benchmark`** as the canonical benchmark harness.
## Node benchmark (`@examples/benchmark`)

When working on performance investigations or changes that might impact **core, normalizr, or endpoint** (no browser, no React), use **`@examples/benchmark`** as the canonical harness.

## React benchmark (`@examples/benchmark-react`)

When working on **`packages/react`** or comparing data-client to other React data libraries (TanStack Query, SWR, baseline), use **`@examples/benchmark-react`**.

- **Where it lives**: `examples/benchmark-react/`
- **How to run**: From repo root: `yarn build:benchmark-react`, then `yarn workspace example-benchmark-react preview &` and in another terminal `cd examples/benchmark-react && yarn bench`
- **What it measures**: Browser-based init/update duration, ref-stability counts, sorted-view (Query memoization), optional memory (heap delta), startup metrics (FCP/TBT), and React Profiler commit times. Compares data-client, TanStack Query, SWR, and a plain React baseline.
- **CI**: `.github/workflows/benchmark-react.yml` runs on changes to `packages/react/src/**`, `packages/core/src/**`, `packages/endpoint/src/schemas/**`, `packages/normalizr/src/**`, or `examples/benchmark-react/**` and reports via `rhysd/github-action-benchmark` (customSmallerIsBetter). CI runs **data-client only** (hot-path scenarios) to track regressions; competitor libraries (TanStack Query, SWR, baseline) are for local comparison only.
- **Report viewer**: Open `examples/benchmark-react/bench/report-viewer.html` in a browser and paste `react-bench-output.json` to view a comparison table and charts. Toggle "React commit" and "Trace" filters. Use "Load history" for time-series.

See `@examples/benchmark-react/README.md` for methodology, adding a new library, and interpreting results.

### Scenarios and what they exercise

Use this mapping when deciding which React benchmark scenarios are relevant to a change:

- **Get list scenarios** (`getlist-100`, `getlist-500`)
- Exercises: full fetch + normalization + render pipeline (ListView auto-fetches from list endpoint)
- Relevant for: `@data-client/react` hooks, `@data-client/core` store initialization
- All libraries

- **Update propagation** (`update-single-entity`, `update-shared-author-500-mounted`, `update-shared-author-10000-mounted`)
- Exercises: store update → React rerender → DOM mutation
- Relevant for: `@data-client/core` dispatch/reducer, `@data-client/react` subscription/selector
- All libraries (normalization advantage shows with shared author at scale)

- **Ref-stability** (`ref-stability-item-changed`, `ref-stability-author-changed`)
- Exercises: referential equality preservation through normalization
- Relevant for: `@data-client/normalizr` denormalize memoization, Entity identity
- All libraries (data-client should show fewest changed refs)

- **Sorted/derived view** (`sorted-view-mount-500`, `sorted-view-update-entity`)
- Exercises: `Query` schema memoization via `useQuery` (data-client) vs `useMemo` sort (competitors)
- Relevant for: `@data-client/endpoint` Query, `@data-client/normalizr` MemoCache, `@data-client/react` useQuery
- All libraries

- **Optimistic update** (`optimistic-update`) — data-client only
- Exercises: `getOptimisticResponse` + `controller.fetch` pipeline
- Relevant for: `@data-client/core` optimistic dispatch

- **Invalidation** (`invalidate-and-resolve`) — data-client only
- Exercises: `controller.invalidate` → Suspense fallback → `controller.setResponse` re-resolve
- Relevant for: `@data-client/core` invalidation, `@data-client/react` Suspense integration

### Expected variance

| Category | Scenarios | Typical run-to-run spread |
|---|---|---|
| **Stable** | `getlist-*`, `update-single-entity`, `ref-stability-*`, `sorted-view-mount-*` | 2–5% |
| **Moderate** | `update-shared-author-*`, `sorted-view-update-*` | 5–10% |
| **Volatile** | `memory-mount-unmount-cycle`, `startup-*`, `(react commit)` suffixes | 10–25% |

Regressions >5% on stable scenarios or >15% on volatile scenarios are worth investigating.

### When to use Node vs React benchmark

- **Core/normalizr/endpoint changes only** (no rendering impact): Run `examples/benchmark` (Node). Faster iteration, no browser needed.
- **React hook or Provider changes**: Run `examples/benchmark-react`. Captures real rendering cost.
- **Schema changes** (Entity, Query, All): Run both — Node benchmark for raw throughput, React benchmark for rendering impact.
- **Performance investigation**: Start with Node benchmark to isolate the JS layer, then validate with React benchmark for end-to-end confirmation.

---

# Node benchmark details (`@examples/benchmark`)

## Optimization workflow

Expand Down
101 changes: 101 additions & 0 deletions .github/workflows/benchmark-react.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: Benchmark React

on:
pull_request:
branches:
- master
paths:
- 'packages/react/src/**'
- 'packages/core/src/**'
- 'packages/endpoint/src/schemas/**'
- 'packages/normalizr/src/**'
- 'examples/benchmark-react/**'
- '.github/workflows/benchmark-react.yml'
push:
branches:
- master
paths:
- 'packages/react/src/**'
- 'packages/core/src/**'
- 'packages/endpoint/src/schemas/**'
- 'packages/normalizr/src/**'
- 'examples/benchmark-react/**'
- '.github/workflows/benchmark-react.yml'

concurrency:
group: ${{ github.event_name == 'push' && 'gh-pages-bench-react-push' || format('benchmark-react-{0}', github.head_ref) }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

permissions:
contents: write
pull-requests: write

jobs:
benchmark-react:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
with:
fetch-depth: 1
- uses: actions/setup-node@v6
with:
node-version: '24'
cache: 'yarn'
- name: Install packages
run: |
corepack enable
yarn install --immutable
- name: Install Playwright (Chromium + system deps)
run: npx playwright install chromium --with-deps
- name: Build packages
run: yarn build:benchmark-react
- name: Run benchmark
run: |
yarn workspace example-benchmark-react preview &
sleep 10
cd examples/benchmark-react && yarn bench | tee react-bench-output.json

# PR comments on changes
- name: Download previous benchmark data (PR)
if: ${{ github.event_name == 'pull_request' }}
uses: actions/cache@v5
with:
path: ./cache
key: ${{ runner.os }}-benchmark-react-pr-${{ github.run_number }}
restore-keys: |
${{ runner.os }}-benchmark-react
- name: Store benchmark result (PR)
if: ${{ github.event_name == 'pull_request' }}
uses: rhysd/github-action-benchmark@v1
with:
tool: 'customSmallerIsBetter'
output-file-path: examples/benchmark-react/react-bench-output.json
github-token: "${{ secrets.GITHUB_TOKEN }}"
gh-pages-branch: 'gh-pages-bench'
benchmark-data-dir-path: react-bench
alert-threshold: '150%'
comment-always: true
fail-on-alert: false
alert-comment-cc-users: '@ntucker'
save-data-file: false
auto-push: false

# master reports to history
- name: Download previous benchmark data (main)
if: ${{ github.event_name == 'push' }}
uses: actions/cache@v5
with:
path: ./cache
key: ${{ runner.os }}-benchmark-react
- name: Store benchmark result (main)
if: ${{ github.event_name == 'push' }}
uses: rhysd/github-action-benchmark@v1
with:
tool: 'customSmallerIsBetter'
output-file-path: examples/benchmark-react/react-bench-output.json
github-token: "${{ secrets.GITHUB_TOKEN }}"
gh-pages-branch: 'gh-pages-bench'
benchmark-data-dir-path: react-bench
auto-push: true
fail-on-alert: false
10 changes: 9 additions & 1 deletion .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ on:
- 'packages/core/src/**'
- 'examples/benchmark/**'
- '.github/workflows/benchmark.yml'
permissions:
contents: write
pull-requests: write

concurrency:
group: ${{ github.event_name == 'push' && 'gh-pages-bench-node-push' || format('benchmark-node-{0}', github.head_ref) }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
benchmark:

Expand Down Expand Up @@ -48,7 +56,7 @@ jobs:
uses: actions/cache@v5
with:
path: ./cache
key: ${{ runner.os }}-benchmark-pr-${{ env.GITHUB_RUN_NUMBER }}
key: ${{ runner.os }}-benchmark-pr-${{ github.run_number }}
restore-keys: |
${{ runner.os }}-benchmark
- name: Store benchmark result (PR)
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/bundle_size.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ on:
- 'yarn.lock'
- 'examples/test-bundlesize/**'
- '.github/workflows/bundle_size.yml'

concurrency:
group: bundle-size-${{ github.head_ref || github.ref }}
cancel-in-progress: true

jobs:
build:

Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ on:
permissions:
contents: read

concurrency:
group: codeql-${{ github.head_ref || github.ref }}
cancel-in-progress: true

jobs:
analyze:
permissions:
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/site-preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ on:
- 'website/**'
- 'docs/**'
- '.github/workflows/site-preview.yml'

concurrency:
group: site-preview-${{ github.head_ref || github.ref }}
cancel-in-progress: true

jobs:
deploy:
runs-on: ubuntu-latest
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/site-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ on:
- 'website/**'
- 'docs/**'
- '.github/workflows/site-release.yml'

concurrency:
group: site-release-${{ github.ref }}
cancel-in-progress: true

jobs:
deploy:
runs-on: ubuntu-latest
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ Any user-facing change in `packages/*` requires a changeset. Core packages are v
- **Examples**: `examples/todo-app`, `examples/github-app`, `examples/nextjs`
- **Documentation**: `docs/core/api`, `docs/rest`, `docs/core/guides`
- **Tests**: `packages/*/src/**/__tests__`
- **Benchmarks**: `examples/benchmark` (Node: core/normalizr/endpoint throughput), `examples/benchmark-react` (browser: React rendering and data-library comparison). See `.cursor/rules/benchmarking.mdc` and each example’s README.

## Key Principles

Expand Down
2 changes: 1 addition & 1 deletion examples/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

### Monorepo workspace members

`benchmark`, `test-bundlesize`, `normalizr-*`, `coin-app` — listed in root `package.json` workspaces, managed by yarn. Most use `workspace:*` for `@data-client/*` deps.
`benchmark`, `benchmark-react`, `test-bundlesize`, `normalizr-*`, `coin-app` — listed in root `package.json` workspaces, managed by yarn. Most use `workspace:*` for `@data-client/*` deps.

### Standalone (StackBlitz demos)

Expand Down
8 changes: 8 additions & 0 deletions examples/benchmark-react/.babelrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
const options = { polyfillMethod: false };
if (process.env.REACT_COMPILER === 'true') {
options.reactCompiler = {};
}

module.exports = {
presets: [['@anansi', options]],
};
Loading
Loading