fix(db): batch D2 output callbacks to prevent duplicate key errors in joins #1114

KyleAMathews · 2026-01-09T20:45:59Z

Summary

Fixes isReady() returning true while toArray() returns []. Also fixes a latent bug that was exposed by the timing fix.

The Bug Report

I have a case where collection.toArray is empty, but collection.isReady() is already true. Just to confirm, isReady should only be true once the query/collection got all query results for the first time, right?

const collection = createLiveQueryCollection((q) =>
  q.from({ room: roomCollection }).where(({ room }) => eq(room.id, roomId))
);

await collection.toArrayWhenReady();
// isReady() === true, but toArray() === [] ← BUG

Root Cause

markReady() was being called in subscribeToAllCollections() before the D2 graph had a chance to run and process data:

// In subscribeToAllCollections():
syncState.subscribedToAllCollections = true
this.updateLiveQueryStatus(config)  // ← Called here, before graph.run()!

The graph only runs in maybeRunGraph(), which happens after subscriptions are set up. So isReady became true while the data pipeline was still empty.

The Fix

Add a guard to updateLiveQueryStatus() so it only marks ready after subscriptions are set up AND the graph has had a chance to run:

if (
  this.currentSyncState?.subscribedToAllCollections &&  // ← NEW: must have subscriptions
  this.allCollectionsReady() &&
  !this.liveQueryCollection?.isLoadingSubset
) {
  markReady()
}

And remove the premature call from subscribeToAllCollections(). The canonical place to mark ready is now after graph.run() in maybeRunGraph().

The Journey: Exposing a Latent Bug

When I made the timing fix, a test started failing: "should allow custom getKey with joins (1:1 relationships)"

Error: Cannot insert document with key '1' from sync because it already exists

Why did this test "pass" before?

The old code called markReady() before the graph ran, which resolved preload() early. The duplicate key error was thrown after the promise resolved, so it was swallowed. The test appeared to pass, but was actually broken.

What was the actual bug?

D2's incremental join produces multiple outputs during a single graph.run():

First: partial join [key=1, undefined] with multiplicity +1
Second: full join [key=1, data] with multiplicity +1, AND delete of partial with multiplicity -1

Previously, each output committed immediately. The first commit added key "1" to syncedData, and the second output's insert failed because the key already existed.

The fix: Batch all outputs within a single graph.run() into one transaction. Changes for the same key are accumulated (+1, +1, -1 = +1) before committing.

Key Invariants

markReady() only after graph processes data - the subscribedToAllCollections check ensures we don't mark ready before the graph runs
All outputs within a graph.run() share one transaction - prevents intermediate join states from causing duplicate key errors

Verification

pnpm vitest run packages/db/tests/query/live-query-collection.test.ts
# All 46 tests pass

pnpm vitest run packages/db/tests
# 1716 tests pass

Files Changed

File	Change
`collection-config-builder.ts`	Add `subscribedToAllCollections` guard to `updateLiveQueryStatus()`, remove premature call from `subscribeToAllCollections()`, batch outputs in `pendingChanges` Map
`types.ts`	Add `flushPendingChanges` to `SyncState` type
`sync.ts`	Minor refactor: extract `valuesEqual` variable
`live-query-collection.test.ts`	Fix test timing to properly await async operations

🤖 Generated with Claude Code

… joins When D2's incremental join processes data, it can produce multiple outputs during a single graph.run() iteration - first a partial join result, then the full result plus a delete of the partial. Previously, each output callback had its own begin()/commit() cycle, causing the second insert for the same key to fail with DuplicateKeySyncError. This fix: - Accumulates all changes from output callbacks into a pendingChanges Map - Flushes accumulated changes in a single transaction after each graph.run() - Adds subscribedToAllCollections check to updateLiveQueryStatus() to ensure markReady() is only called after the graph has processed data - Properly types flushPendingChanges on SyncState to avoid type assertions Co-Authored-By: Claude Opus 4.5 <[email protected]>

changeset-bot · 2026-01-09T20:46:02Z

🦋 Changeset detected

Latest commit: 5ad89bc

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@tanstack/db	Patch
@tanstack/angular-db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/offline-transactions	Patch
@tanstack/powersync-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/rxdb-db-collection	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2026-01-09T20:48:32Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1114

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1114

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1114

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1114

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1114

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1114

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1114

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1114

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1114

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1114

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1114

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1114

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1114

commit: bcf9627

github-actions · 2026-01-09T20:49:08Z

Size Change: +43 B (+0.05%)

Total Size: 90.2 kB

Filename	Size	Change
`./packages/db/dist/esm/query/live/collection-config-builder.js`	5.4 kB	+43 B (+0.8%)

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/collection/change-events.js`	1.39 kB
`./packages/db/dist/esm/collection/changes.js`	1.19 kB
`./packages/db/dist/esm/collection/events.js`	388 B
`./packages/db/dist/esm/collection/index.js`	3.32 kB
`./packages/db/dist/esm/collection/indexes.js`	1.1 kB
`./packages/db/dist/esm/collection/lifecycle.js`	1.67 kB
`./packages/db/dist/esm/collection/mutations.js`	2.34 kB
`./packages/db/dist/esm/collection/state.js`	3.46 kB
`./packages/db/dist/esm/collection/subscription.js`	3.62 kB
`./packages/db/dist/esm/collection/sync.js`	2.38 kB
`./packages/db/dist/esm/deferred.js`	207 B
`./packages/db/dist/esm/errors.js`	4.49 kB
`./packages/db/dist/esm/event-emitter.js`	748 B
`./packages/db/dist/esm/index.js`	2.69 kB
`./packages/db/dist/esm/indexes/auto-index.js`	742 B
`./packages/db/dist/esm/indexes/base-index.js`	766 B
`./packages/db/dist/esm/indexes/btree-index.js`	1.93 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.1 kB
`./packages/db/dist/esm/indexes/reverse-index.js`	513 B
`./packages/db/dist/esm/local-only.js`	837 B
`./packages/db/dist/esm/local-storage.js`	2.1 kB
`./packages/db/dist/esm/optimistic-action.js`	359 B
`./packages/db/dist/esm/paced-mutations.js`	496 B
`./packages/db/dist/esm/proxy.js`	3.75 kB
`./packages/db/dist/esm/query/builder/functions.js`	733 B
`./packages/db/dist/esm/query/builder/index.js`	4.01 kB
`./packages/db/dist/esm/query/builder/ref-proxy.js`	917 B
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.35 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	430 B
`./packages/db/dist/esm/query/compiler/group-by.js`	1.8 kB
`./packages/db/dist/esm/query/compiler/index.js`	1.96 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	1.46 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.07 kB
`./packages/db/dist/esm/query/expression-helpers.js`	1.43 kB
`./packages/db/dist/esm/query/ir.js`	673 B
`./packages/db/dist/esm/query/live-query-collection.js`	360 B
`./packages/db/dist/esm/query/live/collection-registry.js`	264 B
`./packages/db/dist/esm/query/live/collection-subscriber.js`	1.93 kB
`./packages/db/dist/esm/query/live/internal.js`	130 B
`./packages/db/dist/esm/query/optimizer.js`	2.56 kB
`./packages/db/dist/esm/query/predicate-utils.js`	2.97 kB
`./packages/db/dist/esm/query/subset-dedupe.js`	921 B
`./packages/db/dist/esm/scheduler.js`	1.3 kB
`./packages/db/dist/esm/SortedMap.js`	1.3 kB
`./packages/db/dist/esm/strategies/debounceStrategy.js`	247 B
`./packages/db/dist/esm/strategies/queueStrategy.js`	428 B
`./packages/db/dist/esm/strategies/throttleStrategy.js`	246 B
`./packages/db/dist/esm/transactions.js`	2.9 kB
`./packages/db/dist/esm/utils.js`	924 B
`./packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`./packages/db/dist/esm/utils/btree.js`	5.61 kB
`./packages/db/dist/esm/utils/comparison.js`	852 B
`./packages/db/dist/esm/utils/cursor.js`	457 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.51 kB
`./packages/db/dist/esm/utils/type-guards.js`	157 B

_{compressed-size-action::db-package-size}

github-actions · 2026-01-09T20:49:43Z

Size Change: 0 B

Total Size: 3.47 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	225 B
`./packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.17 kB
`./packages/react-db/dist/esm/useLiveQuery.js`	1.12 kB
`./packages/react-db/dist/esm/useLiveSuspenseQuery.js`	559 B
`./packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

Co-Authored-By: Claude Opus 4.5 <[email protected]>

kevin-dp

Overall the changes LGTM.
I left a comment to improve code quality because a helper function was inlined which makes the code more complex than it needs to be.

packages/db/src/query/live/collection-config-builder.ts

Restore the accumulateChanges helper function instead of inlining its logic. This improves code readability and maintainability by keeping the accumulation logic in a separate, testable function. Co-authored-by: Kevin <[email protected]>

github-actions · 2026-01-13T02:58:24Z

🎉 This PR has been released!

@tanstack/[email protected]

Thank you for your contribution!

ci: apply automated fixes

1dde0d2

chore: add changeset for live query join fix

718cbc2

Co-Authored-By: Claude Opus 4.5 <[email protected]>

KyleAMathews requested a review from samwillis January 9, 2026 20:56

chore: update changeset to mention isReady fix

bcf9627

Co-Authored-By: Claude Opus 4.5 <[email protected]>

kevin-dp approved these changes Jan 12, 2026

View reviewed changes

packages/db/src/query/live/collection-config-builder.ts Show resolved Hide resolved

KyleAMathews merged commit 29033b8 into main Jan 12, 2026

KyleAMathews deleted the fix/live-query-batch-outputs branch January 12, 2026 18:59

github-actions bot mentioned this pull request Jan 12, 2026

ci: Version Packages #1117

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(db): batch D2 output callbacks to prevent duplicate key errors in joins #1114

fix(db): batch D2 output callbacks to prevent duplicate key errors in joins #1114

Uh oh!

KyleAMathews commented Jan 9, 2026 •

edited

Loading

Uh oh!

changeset-bot bot commented Jan 9, 2026 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Jan 9, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 9, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 9, 2026 •

edited

Loading

Uh oh!

kevin-dp left a comment

Uh oh!

Uh oh!

github-actions bot commented Jan 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fix(db): batch D2 output callbacks to prevent duplicate key errors in joins #1114

fix(db): batch D2 output callbacks to prevent duplicate key errors in joins #1114

Uh oh!

Conversation

KyleAMathews commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

The Bug Report

Root Cause

The Fix

The Journey: Exposing a Latent Bug

Key Invariants

Verification

Files Changed

Uh oh!

changeset-bot bot commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevin-dp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jan 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

KyleAMathews commented Jan 9, 2026 •

edited

Loading

changeset-bot bot commented Jan 9, 2026 •

edited

Loading

pkg-pr-new bot commented Jan 9, 2026 •

edited

Loading

github-actions bot commented Jan 9, 2026 •

edited

Loading

github-actions bot commented Jan 9, 2026 •

edited

Loading