Skip to content

Conversation

@KyleAMathews
Copy link
Collaborator

No description provided.

@changeset-bot
Copy link

changeset-bot bot commented Dec 2, 2025

🦋 Changeset detected

Latest commit: f2796da

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages
Name Type
@tanstack/db Minor
@tanstack/angular-db Patch
@tanstack/electric-db-collection Patch
@tanstack/offline-transactions Patch
@tanstack/powersync-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch
todos Patch
@tanstack/db-example-paced-mutations-demo Patch
offline-transactions-react-native Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Dec 2, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@950

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@950

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@950

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@950

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@950

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@950

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@950

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@950

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@950

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@950

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@950

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@950

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@950

commit: f2796da

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2025

Size Change: +1.94 kB (+2.14%)

Total Size: 92.5 kB

Filename Size Change
./packages/db/dist/esm/collection/index.js 3.14 kB -183 B (-5.51%)
./packages/db/dist/esm/collection/indexes.js 845 B -253 B (-23.04%) 🎉
./packages/db/dist/esm/collection/lifecycle.js 1.58 kB -86 B (-5.16%)
./packages/db/dist/esm/collection/subscription.js 3.66 kB +35 B (+0.97%)
./packages/db/dist/esm/index.js 2.67 kB -23 B (-0.85%)
./packages/db/dist/esm/indexes/auto-index.js 935 B +193 B (+26.01%) 🚨
./packages/db/dist/esm/indexes/base-index.js 729 B -37 B (-4.83%)
./packages/db/dist/esm/indexes/btree-index.js 0 B -1.93 kB (removed) 🏆
./packages/db/dist/esm/indexes/index-registry.js 797 B +797 B (new file) 🆕
./packages/db/dist/esm/indexes/lazy-index.js 0 B -1.1 kB (removed) 🏆
./packages/db/dist/esm/indexes/read-optimized-index.js 1.93 kB +1.93 kB (new file) 🆕
./packages/db/dist/esm/indexes/write-optimized-index.js 1.94 kB +1.94 kB (new file) 🆕
./packages/db/dist/esm/indexing.js 371 B +371 B (new file) 🆕
./packages/db/dist/esm/query/builder/functions.js 714 B -19 B (-2.59%)
./packages/db/dist/esm/query/compiler/order-by.js 1.45 kB -5 B (-0.34%)
./packages/db/dist/esm/query/live/collection-subscriber.js 1.97 kB +36 B (+1.86%)
./packages/db/dist/esm/utils/array-utils.js 273 B +273 B (new file) 🆕
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.39 kB
./packages/db/dist/esm/collection/changes.js 1.19 kB
./packages/db/dist/esm/collection/events.js 388 B
./packages/db/dist/esm/collection/mutations.js 2.34 kB
./packages/db/dist/esm/collection/state.js 3.49 kB
./packages/db/dist/esm/collection/sync.js 2.4 kB
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.49 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/indexes/reverse-index.js 513 B
./packages/db/dist/esm/local-only.js 837 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/builder/index.js 4.08 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 1.05 kB
./packages/db/dist/esm/query/compiler/evaluators.js 1.42 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 1.87 kB
./packages/db/dist/esm/query/compiler/index.js 1.96 kB
./packages/db/dist/esm/query/compiler/joins.js 2 kB
./packages/db/dist/esm/query/compiler/select.js 1.06 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/ir.js 673 B
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-config-builder.js 5.4 kB
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/internal.js 145 B
./packages/db/dist/esm/query/optimizer.js 2.56 kB
./packages/db/dist/esm/query/predicate-utils.js 2.97 kB
./packages/db/dist/esm/query/subset-dedupe.js 921 B
./packages/db/dist/esm/scheduler.js 1.3 kB
./packages/db/dist/esm/SortedMap.js 1.3 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 924 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/comparison.js 852 B
./packages/db/dist/esm/utils/cursor.js 457 B
./packages/db/dist/esm/utils/index-optimization.js 1.51 kB
./packages/db/dist/esm/utils/type-guards.js 157 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2025

Size Change: 0 B

Total Size: 3.7 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.34 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 559 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

Copy link
Contributor

@kevin-dp kevin-dp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some comments that i would like to see addressed. Nothing major. There are no unit tests for the new BasicIndex. We should add unit tests.

// Dev mode detection settings - ON by default in non-production
let devModeConfig: IndexDevModeConfig = {
enabled: true,
collectionSizeThreshold: 1000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This threshold seems arbitrary. Going through an non-indexed collection of 1000 rows should be pretty fast. I guess modern machines can easily go to perhaps 1M rows before it really becomes slow and needs indexes. Time-based suggestions (slowQueryThresholdMs) makes much more sense imo.

@kevin-dp kevin-dp force-pushed the claude/estimate-bundle-without-indexing-01RpFe1MCKR6TXmrXaPSEXRx branch from 6df29a2 to 79db466 Compare December 11, 2025 11:02
@KyleAMathews KyleAMathews moved this from Todo to In Progress in TanStack DB 1.0.0 release Jan 7, 2026
claude and others added 11 commits January 12, 2026 10:27
This POC explores making indexing a tree-shakeable opt-in feature to reduce
default bundle size. Key changes:

- Create new `@tanstack/db/indexing` entry point for BTreeIndex and related utils
- Add index registry to replace hard-coded BTreeIndex defaults
- Add dev-mode auto-detection for when indexes would help (collection size,
  slow queries)
- Remove BTreeIndex value export from main entry (types still exported)

Bundle size improvements when NOT using indexing:
- Minified: ~15 KB saved (6.9%)
- Gzipped: ~5.4 KB saved (8.5%)

Usage after this change:
```ts
// Option 1: Enable indexing globally
import { enableIndexing } from '@tanstack/db/indexing'
enableIndexing()

// Option 2: Use explicit index type (best for tree-shaking)
import { BTreeIndex } from '@tanstack/db/indexing'
collection.createIndex((row) => row.userId, { indexType: BTreeIndex })

// Dev mode for index suggestions
import { configureIndexDevMode } from '@tanstack/db'
configureIndexDevMode({ enabled: true, collectionSizeThreshold: 100 })
```

Note: Additional savings (~25KB more) possible by making index-optimization.ts
lazy-loaded, but would require more extensive refactoring of change-events.ts
and order-by.ts.
…y default

Breaking change: autoIndex now defaults to 'off' instead of 'eager'.
This reduces default bundle size by not requiring indexing code.

Changes:
- autoIndex defaults to 'off' - users must explicitly enable or add indexes
- Dev mode suggestions are ON by default (in non-production) to help
  developers identify when indexes would improve performance
- Updated tests to reflect new default behavior

Users who want auto-indexing can either:
1. Set autoIndex: 'eager' on individual collections
2. Import and register BTreeIndex globally:
   ```ts
   import { enableIndexing } from '@tanstack/db/indexing'
   enableIndexing()
   ```

Dev mode will warn in console when queries could benefit from indexes,
and this will also be available in devtools.
Most collections won't have perf issues until much larger sizes.
Slow query detection (10ms) is the more actionable metric.
- Add MapIndex for equality lookups (eq, in) without BTree overhead
- Change enableIndexing() to use MapIndex by default
- Add enableBTreeIndexing() for ORDER BY optimization on large collections
- BTreeIndex is only needed for sorted iteration (10k+ items with ORDER BY)

Bundle size impact:
- MapIndex: ~5 KB (~1.3 KB gzipped)
- BTreeIndex: ~33 KB (~7.8 KB gzipped)
- Savings with MapIndex: ~27 KB (~6 KB gzipped)
…ndexType

- Add BasicIndex using Map + sorted Array for both equality and range queries
- Remove registry pattern - pass defaultIndexType to collection constructor instead
- Remove lazy index infrastructure (LazyIndexWrapper, IndexProxy)
- Simplify indexing.ts entry point to just export BasicIndex and BTreeIndex
- Update all tests to explicitly set defaultIndexType where needed
- Update changeset to reflect simplified approach

Breaking changes:
- createIndex() requires defaultIndexType on collection or indexType in config
- enableIndexing()/enableBTreeIndexing() removed, use defaultIndexType instead
@kevin-dp kevin-dp force-pushed the claude/estimate-bundle-without-indexing-01RpFe1MCKR6TXmrXaPSEXRx branch from feaba11 to ddcb1d7 Compare January 12, 2026 09:33
autofix-ci bot and others added 3 commits January 12, 2026 09:34
- Remove duplicate findInsertionIndex method from BasicIndex
- Add findInsertPositionInArray helper for simple arrays
- Add deleteInSortedArray helper for deleting from sorted arrays
- Keep findInsertPosition for tuple arrays (backward compatibility)
- Update BasicIndex to use the new shared utility functions

Co-authored-by: Kevin <[email protected]>
Replace three-iteration approach (spread, reverse, map) with a single
backwards for loop for better performance.

Co-authored-by: Kevin <[email protected]>
@github-actions github-actions bot force-pushed the claude/estimate-bundle-without-indexing-01RpFe1MCKR6TXmrXaPSEXRx branch from fe9b441 to 50a5fa2 Compare January 12, 2026 10:41
@kevin-dp kevin-dp marked this pull request as ready for review January 12, 2026 13:19
Copy link
Contributor

@kevin-dp kevin-dp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I addressed my own review.

@kevin-dp kevin-dp moved this from In Progress to Ready for review in TanStack DB 1.0.0 release Jan 13, 2026
@KyleAMathews
Copy link
Collaborator Author

/review

1 similar comment
@KyleAMathews
Copy link
Collaborator Author

/review

@claude
Copy link
Contributor

claude bot commented Jan 20, 2026

🔍 Comprehensive PR Review: Make Indexing Opt-In

PR: #950 | Branch: claude/estimate-bundle-without-indexing-01RpFe1MCKR6TXmrXaPSEXRx

This PR makes indexing opt-in for better tree-shaking, introduces a new @tanstack/db/indexing subpath export, and adds two index implementations: BasicIndex (read-optimized) and BTreeIndex (write-optimized).


📊 Summary

Category Critical Important Suggestions
Code Quality 0 4 3
Tests 3 2 2
Error Handling 2 3 2
Type Design 0 2 4
Comments 2 4 3

🔴 Critical Issues (Must Fix)

1. Silent Failure in Index remove() Leaves Index Inconsistent

Files: read-optimized-index.ts:113-123, write-optimized-index.ts:101-111

The remove() method catches errors and returns early with only a console.warn, leaving the index in an inconsistent state:

remove(key: TKey, item: any): void {
  try {
    indexedValue = this.evaluateIndexExpression(item)
  } catch (error) {
    console.warn(`Failed to evaluate...`)
    return  // ⚠️ indexedKeys.delete(key) is NEVER called!
  }
  // ...
  this.indexedKeys.delete(key)  // This line never executes on error
}

Impact: Index corruption over time. Queries return stale/phantom data. Debugging is extremely difficult.

Fix: Either throw the error (preferred) or still call this.indexedKeys.delete(key) to maintain consistency.

2. Missing Tests for ReadOptimizedIndex (BasicIndex)

File: packages/db/src/indexes/read-optimized-index.ts

The ReadOptimizedIndex class (458 lines, exported as BasicIndex) has zero dedicated test coverage. All tests use WriteOptimizedIndex exclusively.

What could break undetected:

  • Range queries (gt, gte, lt, lte) edge cases
  • take() and takeReversed() pagination
  • Binary search bugs in findInsertPositionInArray
  • Memory leaks when removing values from sorted array

3. Missing Tests for Dev Mode Utilities

File: packages/db/src/indexes/index-registry.ts

The new dev mode utilities are completely untested:

  • configureIndexDevMode()
  • trackQuery()
  • checkCollectionSizeForIndex()
  • emitIndexSuggestion()

4. Missing Error Handling Test for createIndex() Without Index Type

When createIndex() is called without indexType and no defaultIndexType is set, an error should be thrown. This breaking change behavior is not tested.


🟠 Important Issues (Should Fix)

1. Config Mismatch: autoIndex: 'eager' Without defaultIndexType Fails Silently

File: auto-index.ts:17-28

When a user sets autoIndex: 'eager' but forgets defaultIndexType, auto-indexing is silently disabled with only a dev-mode warning. In production, there's no warning at all.

if (!collection.config.defaultIndexType) {
  if (isDevModeEnabled()) {  // Only warns in dev mode!
    console.warn(`Auto-indexing is disabled...`)
  }
  return false  // Silent in production
}

Recommendation: Throw at collection creation time if autoIndex: 'eager' is set without defaultIndexType.

2. Naming Inconsistency Between Internal and Public API

File: indexing.ts:27-36

Confusing name mapping:

  • BasicIndex → internally ReadOptimizedIndex
  • BTreeIndex → internally WriteOptimizedIndex

Error messages use internal names, causing confusion:

throw new Error(`Operation ${operation} not supported by BasicIndex`)  // But class is ReadOptimizedIndex

3. equalityLookup() Returns Internal Set Reference

Files: read-optimized-index.ts:241-244, write-optimized-index.ts

equalityLookup(value: any): Set<TKey> {
  return this.valueMap.get(normalizedValue) ?? new Set()  // Returns internal reference!
}

Callers can mutate the internal index state. Return a new Set instead.

4. Dev Mode Enabled by Default Can Cause Unexpected Console Warnings

File: index-registry.ts:8-13

Users who intentionally chose not to use indexes will see noisy console warnings about index suggestions.

5. Auto-Index Creation Failures Silently Swallowed

File: auto-index.ts:73-93

try {
  collection.createIndex(...)
} catch (error) {
  console.warn(`Failed to create auto-index...`)
  // Continues as if nothing happened
}

User expects indexes but queries silently fall back to full scans.


🟡 Suggestions (Nice to Have)

Type Design

  1. IndexSuggestion: Use proper discriminated union to correlate optional fields with type:

    export type IndexSuggestion =
      | { type: 'collection-size'; collectionSize: number; ... }
      | { type: 'slow-query'; queryTimeMs: number; queryCount: number; ... }
  2. RangeQueryOptions: Consider generic type parameter for type-safe boundaries instead of any

  3. getIndexDevModeConfig(): Return frozen copy to prevent external mutation

Code Comments

  1. Update class docs to explain the name aliasing (ReadOptimizedIndex exported as BasicIndex)
  2. Add concrete thresholds to help developers choose between index types
  3. Document trackQuery() is called internally - most developers don't need it

Code Simplification

  1. Significant duplicate code between ReadOptimizedIndex and WriteOptimizedIndex could be extracted to BaseIndex
  2. queryPatterns Map in index-registry.ts grows unbounded - potential memory leak in long-running apps

✅ Strengths

  1. Excellent tree-shaking setup - vite.config.ts and package.json exports are correctly configured
  2. Simplified API - Removing async IndexResolver and LazyIndexWrapper reduces complexity significantly
  3. Great changeset documentation - Clear migration paths with code examples
  4. Well-documented entry point - indexing.ts module docs show all three usage patterns
  5. Tests properly updated - All tests now explicitly pass defaultIndexType: WriteOptimizedIndex

📋 Recommended Action Plan

Before Merge (Critical)

  1. Fix remove() method error handling in both index implementations
  2. Add dedicated tests for ReadOptimizedIndex
  3. Add tests for dev mode utilities

Before Merge (Important)

  1. Throw error at collection creation if autoIndex: 'eager' without defaultIndexType
  2. Fix naming inconsistency in error messages

Post-Merge

  1. Consider type design improvements
  2. Extract duplicate code to BaseIndex
  3. Add memory management for query patterns tracking

🤖 Generated by multi-agent PR review

KyleAMathews pushed a commit that referenced this pull request Jan 20, 2026
Researched JavaScript full-text search libraries that could integrate with
TanStack DB's indexing architecture following PR #950's opt-in pattern.

Key findings:
- Orama recommended (2KB, used by tanstack.com, full features)
- MiniSearch as alternative (7KB, class-based API)
- FlexSearch for high-performance needs
- Custom inverted index for minimal bundle

Includes implementation approach and bundle size comparison.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Ready for review

Development

Successfully merging this pull request may close these issues.

4 participants