demo: Add benchmark-react with normalization and ref-stability scenarios by ntucker · Pull Request #3783 · reactive/data-client

ntucker · 2026-03-09T03:29:24Z

Motivation

We need a browser-based benchmark for @data-client/react that highlights the performance advantages of normalization and referential equality: when one entity is updated, only components that use that entity get new references; others keep the same object reference so React can skip rerenders. This enables comparison against other React data libraries (e.g. TanStack Query, SWR) and surfaces regressions in CI.

Solution

New example examples/benchmark-react: Playwright-driven benchmark that runs a small React app with shared authors, measures mount/update duration via performance.measure(), and reports in customSmallerIsBetter format for rhysd/github-action-benchmark.
CI runs data-client only: Hot-path scenarios for data-client track our regressions; competitor libraries (TanStack Query, SWR, baseline) are benchmarked locally for comparison only.
React Compiler support: yarn bench:run:compiler builds with babel-plugin-react-compiler (via @anansi/babel-preset's reactCompiler option) and labels results [compiler] for side-by-side comparison in the report viewer.

Scenarios

Hot path (CI, data-client only)

Mount (100/500 items), update single entity, update shared author (duration + scaling to 500/1000 mounted)
Ref-stability (count of components receiving new object references — smaller is better)
Sorted/derived view (Query schema memoization vs useMemo sort)
Bulk ingest (500 items through normalization pipeline)
Optimistic update (data-client only)
Invalidate and resolve (data-client only — Suspense boundary round-trip)

Local comparison only

With-network: simulated delay per request (data-client = 1 update vs N refetches for non-normalized libs)
Memory: heap delta after mount/unmount cycles
Startup: FCP and task duration via CDP

Key design decisions

4x CPU throttling via CDP to amplify small differences on fast machines
Interleaved library execution with shuffled order per round to reduce environmental variance
Three measurement layers: performance.measure() (JS-driven), React Profiler actualDuration (reconciliation), Chrome trace duration (full rendering pipeline)
Warmup runs discarded; report median with 95% CI
Report viewer (bench/report-viewer.html): filterable table (base/react-commit/trace), time-series charting via history file loading

Scripts

Script	Description
`yarn bench:run`	Build + serve + bench (all libraries locally)
`yarn bench:run:compiler`	Same but with React Compiler enabled, results labelled `[compiler]`
`yarn build:compiler`	Build with React Compiler only
`yarn bench:compiler`	Run bench with `[compiler]` label only

Open questions

N/A

Made with Cursor

Note

Medium Risk
Adds a large new Playwright-driven benchmarking app plus a new CI workflow that will run on PRs/pushes; while it doesn’t affect runtime library code, it changes CI behavior and introduces many new scripts/deps that could be flaky or slow.

Overview
Introduces a new browser-based React benchmarking harness in examples/benchmark-react to measure end-to-end render performance (mount/update durations, ref-stability counts, derived/sorted views, optional memory/trace metrics) across @data-client/react and local competitor baselines.

Adds benchmark-react GitHub Actions workflow to build and run the Playwright benchmarks and report results via rhysd/github-action-benchmark, and updates existing CI workflows with concurrency/permissions tweaks (including the Node benchmark workflow cache key fix).

Wires the new benchmark app into the monorepo (examples/benchmark-react workspace + build:benchmark-react script) and updates benchmarking documentation/agent notes; also updates the website playground generated @data-client/rest type definitions.

^{Written by Cursor Bugbot for commit 802d8a7. This will update automatically on new commits. Configure here.}

vercel · 2026-03-09T03:29:29Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs-site	Ignored	Preview	Mar 15, 2026 3:18am

changeset-bot · 2026-03-09T03:29:32Z

⚠️ No Changeset found

Latest commit: 802d8a7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-03-09T03:33:59Z

Size Change: 0 B

Total Size: 80.5 kB

ℹ️ View Unchanged

Filename	Size
`examples/test-bundlesize/dist/App.js`	3.18 kB
`examples/test-bundlesize/dist/polyfill.js`	307 B
`examples/test-bundlesize/dist/rdcClient.js`	10.2 kB
`examples/test-bundlesize/dist/rdcEndpoint.js`	6.33 kB
`examples/test-bundlesize/dist/react.js`	59.7 kB
`examples/test-bundlesize/dist/webpack-runtime.js`	726 B

_{compressed-size-action}

codecov · 2026-03-09T03:36:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.05%. Comparing base (84744e0) to head (802d8a7).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3783   +/-   ##
=======================================
  Coverage   98.05%   98.05%           
=======================================
  Files         151      151           
  Lines        2834     2834           
  Branches      555      555           
=======================================
  Hits         2779     2779           
  Misses         11       11           
  Partials       44       44

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

examples/benchmark-react/bench/stats.ts

examples/benchmark-react/bench/tracing.ts

examples/benchmark-react/src/data-client/resources.ts

examples/benchmark-react/bench/runner.ts

examples/benchmark-react/bench/scenarios.ts

examples/benchmark-react/bench/tracing.ts

examples/benchmark-react/package.json

examples/benchmark-react/bench/runner.ts

examples/benchmark-react/src/data-client/index.tsx

examples/benchmark-react/bench/runner.ts

examples/benchmark-react/src/swr/index.tsx

github-actions

Benchmark

Details

Benchmark suite	Current: `802d8a7`	Previous: `a8c10ca`	Ratio
`normalizeLong`	`447` ops/sec (`±1.33%`)	`448` ops/sec (`±0.82%`)	`1.00`
`normalizeLong Values`	`412` ops/sec (`±1.04%`)	`404` ops/sec (`±0.48%`)	`0.98`
`denormalizeLong`	`277` ops/sec (`±3.48%`)	`275` ops/sec (`±2.97%`)	`0.99`
`denormalizeLong Values`	`264` ops/sec (`±2.22%`)	`264` ops/sec (`±2.36%`)	`1`
`denormalizeLong donotcache`	`1055` ops/sec (`±0.17%`)	`1043` ops/sec (`±0.15%`)	`0.99`
`denormalizeLong Values donotcache`	`784` ops/sec (`±0.19%`)	`759` ops/sec (`±0.20%`)	`0.97`
`denormalizeShort donotcache 500x`	`1572` ops/sec (`±0.14%`)	`1431` ops/sec (`±0.16%`)	`0.91`
`denormalizeShort 500x`	`817` ops/sec (`±2.15%`)	`785` ops/sec (`±2.10%`)	`0.96`
`denormalizeShort 500x withCache`	`6388` ops/sec (`±0.68%`)	`4769` ops/sec (`±0.62%`)	`0.75`
`queryShort 500x withCache`	`2593` ops/sec (`±0.13%`)	`2611` ops/sec (`±0.31%`)	`1.01`
`buildQueryKey All`	`55460` ops/sec (`±0.31%`)	`58214` ops/sec (`±0.33%`)	`1.05`
`query All withCache`	`6734` ops/sec (`±0.29%`)	`5813` ops/sec (`±0.25%`)	`0.86`
`denormalizeLong with mixin Entity`	`278` ops/sec (`±2.37%`)	`277` ops/sec (`±2.02%`)	`1.00`
`denormalizeLong withCache`	`7077` ops/sec (`±0.21%`)	`6431` ops/sec (`±0.09%`)	`0.91`
`denormalizeLong Values withCache`	`5109` ops/sec (`±0.14%`)	`4658` ops/sec (`±0.10%`)	`0.91`
`denormalizeLong All withCache`	`6462` ops/sec (`±0.17%`)	`5468` ops/sec (`±0.09%`)	`0.85`
`denormalizeLong Query-sorted withCache`	`6755` ops/sec (`±0.26%`)	`6294` ops/sec (`±0.09%`)	`0.93`
`denormalizeLongAndShort withEntityCacheOnly`	`1691` ops/sec (`±0.33%`)	`1551` ops/sec (`±0.25%`)	`0.92`
`getResponse`	`4682` ops/sec (`±0.47%`)	`3770` ops/sec (`±0.23%`)	`0.81`
`getResponse (null)`	`10747864` ops/sec (`±0.99%`)	`9815415` ops/sec (`±0.77%`)	`0.91`
`getResponse (clear cache)`	`269` ops/sec (`±2.05%`)	`272` ops/sec (`±2.12%`)	`1.01`
`getSmallResponse`	`3365` ops/sec (`±0.13%`)	`3231` ops/sec (`±0.39%`)	`0.96`
`getSmallInferredResponse`	`2501` ops/sec (`±0.31%`)	`2476` ops/sec (`±0.21%`)	`0.99`
`getResponse Collection`	`4513` ops/sec (`±0.46%`)	`3763` ops/sec (`±0.15%`)	`0.83`
`get Collection`	`4512` ops/sec (`±0.32%`)	`3539` ops/sec (`±0.31%`)	`0.78`
`get Query-sorted`	`5241` ops/sec (`±0.20%`)	`5072` ops/sec (`±0.11%`)	`0.97`
`setLong`	`457` ops/sec (`±0.18%`)	`452` ops/sec (`±0.54%`)	`0.99`
`setLongWithMerge`	`261` ops/sec (`±0.12%`)	`258` ops/sec (`±0.29%`)	`0.99`
`setLongWithSimpleMerge`	`276` ops/sec (`±0.16%`)	`273` ops/sec (`±0.18%`)	`0.99`
`setSmallResponse 500x`	`945` ops/sec (`±0.11%`)	`903` ops/sec (`±0.17%`)	`0.96`

This comment was automatically generated by workflow using github-action-benchmark.

examples/benchmark-react/src/swr/index.tsx

examples/benchmark-react/bench/runner.ts

.github/workflows/benchmark-react.yml

examples/benchmark-react/bench/runner.ts

examples/benchmark-react/bench/scenarios.ts

examples/benchmark-react/src/data-client/index.tsx

- Browser benchmark comparing @data-client/react (Playwright, customSmallerIsBetter). - Scenarios: mount, update entity/author, ref-stability (item/author ref counts). - Hot-path (CI) vs with-network (local): simulated delay for overfetch comparison. - CI workflow runs hot-path only; reports to rhysd/github-action-benchmark. Made-with: Cursor

examples/benchmark-react/src/data-client/index.tsx

fix yarn lock

examples/benchmark-react/bench/runner.ts

cursor · 2026-03-14T18:47:41Z

examples/benchmark-react/src/shared/data.ts

+    });
+  }
+  return { items, authors };
+}


generateFreshData exported but never imported anywhere

Low Severity

generateFreshData is exported but has zero consumers — no file in the benchmark suite imports it. The comment mentions "bulk ingestion scenarios" but no such scenario exists in the codebase. This is dead code that adds maintenance burden without providing value.

examples/benchmark-react/src/baseline/index.tsx

examples/benchmark-react/src/shared/server.ts

examples/benchmark-react/src/shared/benchHarness.tsx

examples/benchmark-react/src/shared/resources.ts

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

examples/benchmark-react/bench/runner.ts

ntucker force-pushed the benchmark-react-normalization branch from af004f6 to a2cb25d Compare March 9, 2026 03:32

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Show resolved Hide resolved

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Outdated Show resolved Hide resolved

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/scenarios.ts Outdated Show resolved Hide resolved

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/tracing.ts Outdated Show resolved Hide resolved

examples/benchmark-react/package.json Outdated Show resolved Hide resolved

cursor bot reviewed Mar 10, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Show resolved Hide resolved

cursor bot reviewed Mar 10, 2026

View reviewed changes

examples/benchmark-react/src/data-client/index.tsx Show resolved Hide resolved

cursor bot reviewed Mar 10, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Outdated Show resolved Hide resolved

ntucker force-pushed the benchmark-react-normalization branch from d8d904f to 98de6fa Compare March 10, 2026 13:31

vercel bot deployed to Preview March 10, 2026 13:36 View deployment

cursor bot reviewed Mar 10, 2026

View reviewed changes

examples/benchmark-react/src/swr/index.tsx Outdated Show resolved Hide resolved

github-actions bot reviewed Mar 10, 2026

View reviewed changes

cursor bot reviewed Mar 11, 2026

View reviewed changes

examples/benchmark-react/src/swr/index.tsx Outdated Show resolved Hide resolved

cursor bot reviewed Mar 12, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Show resolved Hide resolved

.github/workflows/benchmark-react.yml Show resolved Hide resolved

ntucker force-pushed the benchmark-react-normalization branch from 5ad1b81 to 4df9d01 Compare March 12, 2026 03:10

cursor bot reviewed Mar 13, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Show resolved Hide resolved

cursor bot reviewed Mar 13, 2026

View reviewed changes

examples/benchmark-react/bench/scenarios.ts Show resolved Hide resolved

cursor bot reviewed Mar 14, 2026

View reviewed changes

examples/benchmark-react/src/data-client/index.tsx Show resolved Hide resolved

ntucker force-pushed the benchmark-react-normalization branch 2 times, most recently from c8bc040 to 081e48d Compare March 14, 2026 02:56

ntucker added 5 commits March 13, 2026 23:00

more of the plan

eb261cb

More scenarios

ddd4c8f

more scenarios

4080f7b

add react compiler option

678b359

ntucker added 5 commits March 13, 2026 23:00

internal: Bench runs concurrently

7090705

better abstractions

2606262

CRUD

830199e

virtualize

60e8c10

create adds to list; larger high end case

58e4f85

ntucker force-pushed the benchmark-react-normalization branch from 081e48d to 06986ed Compare March 14, 2026 03:00

cursor bot reviewed Mar 14, 2026

View reviewed changes

examples/benchmark-react/src/data-client/index.tsx Show resolved Hide resolved

ntucker force-pushed the benchmark-react-normalization branch from 1f5d167 to a496880 Compare March 14, 2026 17:37

No seeding

0253c11

fix yarn lock

ntucker force-pushed the benchmark-react-normalization branch from a496880 to 0253c11 Compare March 14, 2026 18:35

cursor bot reviewed Mar 14, 2026

View reviewed changes

fix test conditions to be more accurate

f5ea087

cursor bot reviewed Mar 14, 2026

View reviewed changes

examples/benchmark-react/src/baseline/index.tsx Show resolved Hide resolved

dynamic accuracy

6b15581

cursor bot reviewed Mar 14, 2026

View reviewed changes

examples/benchmark-react/src/shared/server.ts Outdated Show resolved Hide resolved

ntucker added 5 commits March 14, 2026 16:53

fix bench measurements

5f34537

remove unneeded bench

b0ab3fb

fix measurement by eliminating paint timings from measurement

ba040ef

increase scale, remove redundant

c9d5d60

bench name updates

0281018

cursor bot reviewed Mar 15, 2026

View reviewed changes

examples/benchmark-react/src/shared/benchHarness.tsx Show resolved Hide resolved

more realistic data

0de0caf

cursor bot reviewed Mar 15, 2026

View reviewed changes

examples/benchmark-react/src/shared/resources.ts Show resolved Hide resolved

ntucker added 4 commits March 14, 2026 21:26

MutationObserver timeout silently fails without signaling completion

aed6a64

Make sorted mount consistent

bb70f1e

fix sorted-view-update-entity for some frameworks

3cd8079

change reporting

d023380

cursor bot reviewed Mar 15, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Show resolved Hide resolved

ntucker added 2 commits March 14, 2026 23:10

move scenario

41fcaa9

Init scenarios capture wrong react-commit-update measurement

802d8a7

Uh oh!

Conversation

ntucker commented Mar 9, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Solution

Scenarios

Key design decisions

Scripts

Open questions

Uh oh!

vercel bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot Mar 14, 2026

Choose a reason for hiding this comment

generateFreshData exported but never imported anywhere

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ntucker commented Mar 9, 2026 •

edited by cursor bot

Loading

vercel bot commented Mar 9, 2026 •

edited

Loading

changeset-bot bot commented Mar 9, 2026 •

edited

Loading

github-actions bot commented Mar 9, 2026 •

edited

Loading

codecov bot commented Mar 9, 2026 •

edited

Loading

github-actions bot left a comment •

edited

Loading

`generateFreshData` exported but never imported anywhere