Skip to content

Commit ee441c6

Browse files
authored
demo(benchmark-react): add baseline framework for raw React comparison (#3809)
* demo(benchmark-react): add baseline framework for raw React comparison Add a "baseline" framework that uses only React primitives (useState + useEffect) with zero global caching. Each component independently fetches its own data, providing a true zero-library comparison point for all existing benchmark scenarios. Made-with: Cursor * docs: Update readme with results
1 parent e75a181 commit ee441c6

File tree

4 files changed

+334
-25
lines changed

4 files changed

+334
-25
lines changed

examples/benchmark-react/README.md

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -51,14 +51,14 @@ The repo has two benchmark suites:
5151

5252
## Expected results
5353

54-
Illustrative **relative** results with **SWR = 100%** (baseline). For **throughput** rows, each value is (library ops/s ÷ SWR ops/s) × 100 — **higher is faster**. For **ref-stability** rows, the ratio uses the “refs changed” count — **lower is fewer components that saw a new object reference**. Figures are rounded from the **Latest measured results** table below (network simulation on); absolute ops/s will vary by machine, but **library-to-library ratios** are usually similar.
54+
Illustrative **relative** results with **baseline = 100%** (plain React useState/useEffect, no data library). For **throughput** rows, each value is (library ops/s ÷ baseline ops/s) × 100 — **higher is faster**. For **ref-stability** rows, the ratio uses the “refs changed” count — **lower is fewer components that saw a new object reference**. Figures are rounded from the **Latest measured results** table below (network simulation on); absolute ops/s will vary by machine, but **library-to-library ratios** are usually similar.
5555

56-
| Category | Scenarios (representative) | data-client | tanstack-query | swr |
57-
|---|---|---:|---:|---:|
58-
| Navigation | `getlist-100`, `getlist-500`, `getlist-500-sorted` | ~97% | ~99% | **100%** |
59-
| Navigation | `list-detail-switch-10` | **~468%** | ~98% | 100% |
60-
| Mutations | `update-entity`, `update-user`, `update-entity-sorted`, `update-entity-multi-view`, `unshift-item`, `delete-item`, `move-item` | **~4600%** | ~98% | 100% |
61-
| Scaling (10k items) | `update-user-10000` | **~1953%** | ~82% | 100% |
56+
| Category | Scenarios (representative) | data-client | tanstack-query | swr | baseline |
57+
|---|---|---:|---:|---:|---:|
58+
| Navigation | `getlist-100`, `getlist-500`, `getlist-500-sorted` | ~96% | ~98% | ~99% | **100%** |
59+
| Navigation | `list-detail-switch-10` | **~949%** | ~199% | ~203% | 100% |
60+
| Mutations | `update-entity`, `update-user`, `update-entity-sorted`, `update-entity-multi-view`, `unshift-item`, `delete-item`, `move-item` | **~4486%** | ~96% | ~99% | 100% |
61+
| Scaling (10k items) | `update-user-10000` | **~2006%** | ~84% | ~103% | 100% |
6262

6363

6464
## Latest measured results (network simulation on)
@@ -67,22 +67,22 @@ Median ops/s per scenario; range is approximate 95% CI margin from the runner (`
6767

6868
Run: **2026-03-21**, Linux (WSL2), `yarn build:benchmark-react`, static preview + `env -u CI npx tsx bench/runner.ts --network-sim true` (all libraries; memory scenarios not included). Numbers are **machine-specific**; use them for relative comparison between libraries, not as absolutes.
6969

70-
| Scenario | data-client | tanstack-query | swr |
71-
|---|---:|---:|---:|
72-
| **Navigation** | | | |
73-
| `getlist-100` | 11.20 ± 0.03 | 11.27 ± 0.02 | 11.43 ± 0.07 |
74-
| `getlist-500` | 9.78 ± 0.12 | 10.01 ± 0.13 | 10.16 ± 0.13 |
75-
| `getlist-500-sorted` | 9.82 ± 0.16 | 10.08 ± 0.13 | 10.21 ± 0.07 |
76-
| `list-detail-switch-10` | 6.93 ± 1.02 | 1.45 ± 0.04 | 1.48 ± 0.08 |
77-
| **Mutations** | | | |
78-
| `update-entity` | 357.14 ± 11.48 | 7.01 ± 0.02 | 7.02 ± 0.02 |
79-
| `update-user` | 333.33 ± 14.44 | 7.01 ± 0.02 | 7.17 ± 0.03 |
80-
| `update-entity-sorted` | 312.50 ± 23.44 | 7.08 ± 0.00 | 7.07 ± 0.03 |
81-
| `update-entity-multi-view` | 357.14 ± 52.30 | 6.82 ± 0.34 | 6.88 ± 0.39 |
82-
| `update-user-10000` | 97.09 ± 7.73 | 4.07 ± 0.02 | 4.97 ± 0.02 |
83-
| `unshift-item` | 285.71 ± 4.90 | 6.92 ± 0.02 | 7.16 ± 0.00 |
84-
| `delete-item` | 312.50 ± 9.77 | 6.93 ± 0.01 | 7.15 ± 0.01 |
85-
| `move-item` | 285.71 ± 10.61 | 6.39 ± 0.02 | 6.83 ± 0.00 |
70+
| Scenario | data-client | tanstack-query | swr | baseline |
71+
|---|---:|---:|---:|---:|
72+
| **Navigation** | | | | |
73+
| `getlist-100` | 11.20 ± 0.03 | 11.27 ± 0.02 | 11.43 ± 0.07 | 11.55 ± 0.02 |
74+
| `getlist-500` | 9.78 ± 0.12 | 10.01 ± 0.13 | 10.16 ± 0.13 | 10.22 ± 0.07 |
75+
| `getlist-500-sorted` | 9.82 ± 0.16 | 10.08 ± 0.13 | 10.21 ± 0.07 | 10.29 ± 0.06 |
76+
| `list-detail-switch-10` | 6.93 ± 1.02 | 1.45 ± 0.04 | 1.48 ± 0.08 | 0.73 ± 0.00 |
77+
| **Mutations** | | | | |
78+
| `update-entity` | 357.14 ± 11.48 | 7.01 ± 0.02 | 7.02 ± 0.02 | 7.22 ± 0.00 |
79+
| `update-user` | 333.33 ± 14.44 | 7.01 ± 0.02 | 7.17 ± 0.03 | 7.22 ± 0.01 |
80+
| `update-entity-sorted` | 312.50 ± 23.44 | 7.08 ± 0.00 | 7.07 ± 0.03 | 7.28 ± 0.01 |
81+
| `update-entity-multi-view` | 357.14 ± 52.30 | 6.82 ± 0.34 | 6.88 ± 0.39 | 7.14 ± 0.36 |
82+
| `update-user-10000` | 97.09 ± 7.73 | 4.07 ± 0.02 | 4.97 ± 0.02 | 4.84 ± 0.03 |
83+
| `unshift-item` | 285.71 ± 4.90 | 6.92 ± 0.02 | 7.16 ± 0.00 | 7.16 ± 0.02 |
84+
| `delete-item` | 312.50 ± 9.77 | 6.93 ± 0.01 | 7.15 ± 0.01 | 7.16 ± 0.01 |
85+
| `move-item` | 285.71 ± 10.61 | 6.39 ± 0.02 | 6.83 ± 0.00 | 6.82 ± 0.00 |
8686

8787
[Measured on a Ryzen 9 7950X; 64 GB RAM; Ubuntu (WSL2); Node 24.12.0; Chromium (Playwright)]
8888

examples/benchmark-react/bench/scenarios.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,12 @@ const BASE_SCENARIOS: BaseScenario[] = [
186186
},
187187
];
188188

189-
export const LIBRARIES = ['data-client', 'tanstack-query', 'swr'] as const;
189+
export const LIBRARIES = [
190+
'data-client',
191+
'tanstack-query',
192+
'swr',
193+
'baseline',
194+
] as const;
190195

191196
export const SCENARIOS: Scenario[] = LIBRARIES.flatMap(lib =>
192197
BASE_SCENARIOS.filter(
Lines changed: 304 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,304 @@
1+
import {
2+
moveItemIsReady,
3+
renderBenchApp,
4+
useBenchState,
5+
} from '@shared/benchHarness';
6+
import {
7+
DOUBLE_LIST_STYLE,
8+
IssueRow,
9+
PINNED_STRIP_STYLE,
10+
PinnedCardView,
11+
PlainIssueList,
12+
} from '@shared/components';
13+
import {
14+
FIXTURE_USERS,
15+
FIXTURE_USERS_BY_LOGIN,
16+
FIXTURE_ISSUES_BY_NUMBER,
17+
sortByTitle,
18+
} from '@shared/data';
19+
import { setCurrentIssues } from '@shared/refStability';
20+
import {
21+
fetchIssue,
22+
fetchIssueList,
23+
updateIssue,
24+
updateUser as serverUpdateUser,
25+
createIssue,
26+
deleteIssue,
27+
} from '@shared/server';
28+
import type { Issue } from '@shared/types';
29+
import React, { useCallback, useEffect, useMemo, useState } from 'react';
30+
31+
function SortedListView({
32+
limit,
33+
refetchKey,
34+
}: {
35+
limit?: number;
36+
refetchKey: number;
37+
}) {
38+
const [issues, setIssues] = useState<Issue[] | null>(null);
39+
useEffect(() => {
40+
fetchIssueList().then(setIssues);
41+
}, [refetchKey]);
42+
const sorted = useMemo(() => (issues ? sortByTitle(issues) : []), [issues]);
43+
if (!sorted.length) return null;
44+
return (
45+
<div data-sorted-list>
46+
<PlainIssueList issues={sorted} limit={limit} />
47+
</div>
48+
);
49+
}
50+
51+
function DetailView({
52+
number,
53+
refetchKey,
54+
}: {
55+
number: number;
56+
refetchKey: number;
57+
}) {
58+
const [issue, setIssue] = useState<Issue | null>(null);
59+
useEffect(() => {
60+
fetchIssue({ number }).then(setIssue);
61+
}, [number, refetchKey]);
62+
if (!issue) return null;
63+
return (
64+
<div data-detail-view data-issue-number={number}>
65+
<IssueRow issue={issue} />
66+
</div>
67+
);
68+
}
69+
70+
function PinnedCard({
71+
number,
72+
refetchKey,
73+
}: {
74+
number: number;
75+
refetchKey: number;
76+
}) {
77+
const [issue, setIssue] = useState<Issue | null>(null);
78+
useEffect(() => {
79+
fetchIssue({ number }).then(setIssue);
80+
}, [number, refetchKey]);
81+
if (!issue) return null;
82+
return <PinnedCardView issue={issue} />;
83+
}
84+
85+
function PinnedStrip({
86+
numbers,
87+
refetchKey,
88+
}: {
89+
numbers: number[];
90+
refetchKey: number;
91+
}) {
92+
return (
93+
<div data-pinned-strip style={PINNED_STRIP_STYLE}>
94+
{numbers.map(n => (
95+
<PinnedCard key={n} number={n} refetchKey={refetchKey} />
96+
))}
97+
</div>
98+
);
99+
}
100+
101+
function ListView({
102+
count,
103+
limit,
104+
refetchKey,
105+
}: {
106+
count: number;
107+
limit?: number;
108+
refetchKey: number;
109+
}) {
110+
const [issues, setIssues] = useState<Issue[] | null>(null);
111+
useEffect(() => {
112+
fetchIssueList({ count }).then(setIssues);
113+
}, [count, refetchKey]);
114+
if (!issues) return null;
115+
setCurrentIssues(issues);
116+
return <PlainIssueList issues={issues} limit={limit} />;
117+
}
118+
119+
function StateListView({
120+
state,
121+
count,
122+
limit,
123+
refetchKey,
124+
}: {
125+
state: string;
126+
count: number;
127+
limit?: number;
128+
refetchKey: number;
129+
}) {
130+
const [issues, setIssues] = useState<Issue[] | null>(null);
131+
useEffect(() => {
132+
fetchIssueList({ state, count }).then(setIssues);
133+
}, [state, count, refetchKey]);
134+
if (!issues) return null;
135+
return (
136+
<div data-state-list={state}>
137+
<span data-state-count>{issues.length}</span>
138+
<PlainIssueList issues={issues} limit={limit} />
139+
</div>
140+
);
141+
}
142+
143+
function DoubleListView({
144+
count,
145+
limit,
146+
refetchKey,
147+
}: {
148+
count: number;
149+
limit?: number;
150+
refetchKey: number;
151+
}) {
152+
return (
153+
<div style={DOUBLE_LIST_STYLE}>
154+
<StateListView
155+
state="open"
156+
count={count}
157+
limit={limit}
158+
refetchKey={refetchKey}
159+
/>
160+
<StateListView
161+
state="closed"
162+
count={count}
163+
limit={limit}
164+
refetchKey={refetchKey}
165+
/>
166+
</div>
167+
);
168+
}
169+
170+
function BenchmarkHarness() {
171+
const [refetchKey, setRefetchKey] = useState(0);
172+
const triggerRefetch = useCallback(() => setRefetchKey(k => k + 1), []);
173+
174+
const {
175+
listViewCount,
176+
showSortedView,
177+
showDoubleList,
178+
doubleListCount,
179+
detailIssueNumber,
180+
pinnedNumbers,
181+
renderLimit,
182+
containerRef,
183+
measureUpdate,
184+
registerAPI,
185+
} = useBenchState();
186+
187+
const updateEntity = useCallback(
188+
(number: number) => {
189+
const issue = FIXTURE_ISSUES_BY_NUMBER.get(number);
190+
if (!issue) return;
191+
measureUpdate(() =>
192+
updateIssue({
193+
number,
194+
title: `${issue.title} (updated)`,
195+
}).then(triggerRefetch),
196+
);
197+
},
198+
[measureUpdate, triggerRefetch],
199+
);
200+
201+
const updateUser = useCallback(
202+
(login: string) => {
203+
const user = FIXTURE_USERS_BY_LOGIN.get(login);
204+
if (!user) return;
205+
measureUpdate(() =>
206+
serverUpdateUser({
207+
login,
208+
name: `${user.name} (updated)`,
209+
}).then(triggerRefetch),
210+
);
211+
},
212+
[measureUpdate, triggerRefetch],
213+
);
214+
215+
const unshiftItem = useCallback(() => {
216+
const user = FIXTURE_USERS[0];
217+
measureUpdate(() =>
218+
createIssue({ title: 'New Issue', user }).then(triggerRefetch),
219+
);
220+
}, [measureUpdate, triggerRefetch]);
221+
222+
const deleteEntity = useCallback(
223+
(number: number) => {
224+
measureUpdate(() => deleteIssue({ number }).then(triggerRefetch));
225+
},
226+
[measureUpdate, triggerRefetch],
227+
);
228+
229+
const moveItem = useCallback(
230+
(number: number) => {
231+
measureUpdate(
232+
() => updateIssue({ number, state: 'closed' }).then(triggerRefetch),
233+
() => moveItemIsReady(containerRef, number),
234+
);
235+
},
236+
[measureUpdate, triggerRefetch, containerRef],
237+
);
238+
239+
const updateEntityMultiView = useCallback(
240+
(number: number) => {
241+
const issue = FIXTURE_ISSUES_BY_NUMBER.get(number);
242+
if (!issue) return;
243+
const expected = `${issue.title} (updated)`;
244+
measureUpdate(
245+
() => updateIssue({ number, title: expected }).then(triggerRefetch),
246+
() => {
247+
const container = containerRef.current!;
248+
const listTitle = container.querySelector(
249+
`[data-issue-number="${number}"] [data-title]`,
250+
);
251+
const detailTitle = container.querySelector(
252+
'[data-detail-view] [data-title]',
253+
);
254+
const pinnedTitle = container.querySelector(
255+
`[data-pinned-number="${number}"] [data-title]`,
256+
);
257+
return [listTitle, detailTitle, pinnedTitle].every(
258+
el => el?.textContent === expected,
259+
);
260+
},
261+
);
262+
},
263+
[measureUpdate, triggerRefetch, containerRef],
264+
);
265+
266+
registerAPI({
267+
updateEntity,
268+
updateUser,
269+
updateEntityMultiView,
270+
unshiftItem,
271+
deleteEntity,
272+
moveItem,
273+
});
274+
275+
return (
276+
<div ref={containerRef} data-bench-harness>
277+
{listViewCount != null && (
278+
<ListView
279+
count={listViewCount}
280+
limit={renderLimit}
281+
refetchKey={refetchKey}
282+
/>
283+
)}
284+
{showSortedView && (
285+
<SortedListView limit={renderLimit} refetchKey={refetchKey} />
286+
)}
287+
{showDoubleList && doubleListCount != null && (
288+
<DoubleListView
289+
count={doubleListCount}
290+
limit={renderLimit}
291+
refetchKey={refetchKey}
292+
/>
293+
)}
294+
{detailIssueNumber != null && (
295+
<DetailView number={detailIssueNumber} refetchKey={refetchKey} />
296+
)}
297+
{pinnedNumbers.length > 0 && (
298+
<PinnedStrip numbers={pinnedNumbers} refetchKey={refetchKey} />
299+
)}
300+
</div>
301+
);
302+
}
303+
304+
renderBenchApp(BenchmarkHarness);

examples/benchmark-react/webpack.config.cjs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ const { makeConfig } = require('@anansi/webpack-config');
22
const HtmlWebpackPlugin = require('html-webpack-plugin');
33
const path = require('path');
44

5-
const LIBRARIES = ['data-client', 'tanstack-query', 'swr'];
5+
const LIBRARIES = ['data-client', 'tanstack-query', 'swr', 'baseline'];
66

77
const entries = {};
88
for (const lib of LIBRARIES) {

0 commit comments

Comments
 (0)