Skip to content

Commit d8f95da

Browse files
committed
Fix potential infinite loops in graph execution and data loading
Add iteration safeguards to prevent infinite loops that can occur when using Electric with large datasets and ORDER BY/LIMIT queries: 1. `maybeRunGraph` while loop (collection-config-builder.ts): - Can loop infinitely when data loading triggers graph updates - Happens when WHERE filters out most data, causing dataNeeded() > 0 - Loading more data triggers updates that get filtered out - Added 10,000 iteration limit with error logging 2. `requestLimitedSnapshot` while loop (subscription.ts): - Can loop if index iteration has issues - Added 10,000 iteration limit with error logging - Removed unused `insertedKeys` tracking 3. `D2.run()` while loop (d2.ts): - Can loop infinitely on circular data flow bugs - Added 100,000 iteration limit with error logging The safeguards log errors to help debug the root cause while preventing the app from freezing.
1 parent 05130f2 commit d8f95da

File tree

3 files changed

+51
-2
lines changed

3 files changed

+51
-2
lines changed

packages/db-ivm/src/d2.ts

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,22 @@ export class D2 implements ID2 {
5757
}
5858

5959
run(): void {
60+
// Safety limit to prevent infinite loops in case of circular data flow
61+
// or other bugs that cause operators to perpetually produce output.
62+
// For legitimate pipelines, data should flow through in finite steps.
63+
const MAX_RUN_ITERATIONS = 100000
64+
let iterations = 0
65+
6066
while (this.pendingWork()) {
67+
iterations++
68+
if (iterations > MAX_RUN_ITERATIONS) {
69+
console.error(
70+
`[D2 Graph] Execution exceeded ${MAX_RUN_ITERATIONS} iterations. ` +
71+
`This may indicate an infinite loop in the dataflow graph. ` +
72+
`Breaking out to prevent app freeze.`,
73+
)
74+
break
75+
}
6176
this.step()
6277
}
6378
}

packages/db/src/collection/subscription.ts

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -502,8 +502,23 @@ export class CollectionSubscription
502502
? compileExpression(new PropRef(orderByExpression.path), true)
503503
: null
504504

505+
// Safety limit to prevent infinite loops if the index iteration or filtering
506+
// logic has issues. The loop should naturally terminate when the index is
507+
// exhausted, but this provides a backstop. 10000 iterations is generous
508+
// for any legitimate use case.
509+
const MAX_SNAPSHOT_ITERATIONS = 10000
510+
let snapshotIterations = 0
511+
505512
while (valuesNeeded() > 0 && !collectionExhausted()) {
506-
const insertedKeys = new Set<string | number>() // Track keys we add to `changes` in this iteration
513+
snapshotIterations++
514+
if (snapshotIterations > MAX_SNAPSHOT_ITERATIONS) {
515+
console.error(
516+
`[TanStack DB] requestLimitedSnapshot exceeded ${MAX_SNAPSHOT_ITERATIONS} iterations. ` +
517+
`This may indicate an infinite loop in index iteration or filtering. ` +
518+
`Breaking out to prevent app freeze. Collection: ${this.collection.id}`,
519+
)
520+
break
521+
}
507522

508523
for (const key of keys) {
509524
const value = this.collection.get(key)!
@@ -515,7 +530,6 @@ export class CollectionSubscription
515530
// Extract the indexed value (e.g., salary) from the row, not the full row
516531
// This is needed for index.take() to work correctly with the BTree comparator
517532
biggestObservedValue = valueExtractor ? valueExtractor(value) : value
518-
insertedKeys.add(key) // Track this key
519533
}
520534

521535
keys = index.take(valuesNeeded(), biggestObservedValue, filterFn)

packages/db/src/query/live/collection-config-builder.ts

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -336,7 +336,27 @@ export class CollectionConfigBuilder<
336336

337337
// Always run the graph if subscribed (eager execution)
338338
if (syncState.subscribedToAllCollections) {
339+
// Safety limit to prevent infinite loops when data loading and graph processing
340+
// create a feedback cycle. This can happen when:
341+
// 1. OrderBy/limit queries filter out most data, causing dataNeeded() > 0
342+
// 2. Loading more data triggers updates that get filtered out
343+
// 3. The cycle continues indefinitely
344+
// 10000 iterations is generous for legitimate use cases but prevents hangs.
345+
const MAX_GRAPH_ITERATIONS = 10000
346+
let iterations = 0
347+
339348
while (syncState.graph.pendingWork()) {
349+
iterations++
350+
if (iterations > MAX_GRAPH_ITERATIONS) {
351+
console.error(
352+
`[TanStack DB] Graph execution exceeded ${MAX_GRAPH_ITERATIONS} iterations. ` +
353+
`This may indicate an infinite loop caused by data loading triggering ` +
354+
`continuous graph updates. Breaking out of the loop to prevent app freeze. ` +
355+
`Query ID: ${this.id}`,
356+
)
357+
break
358+
}
359+
340360
syncState.graph.run()
341361
// Flush accumulated changes after each graph step to commit them as one transaction.
342362
// This ensures intermediate join states (like null on one side) don't cause

0 commit comments

Comments
 (0)