Skip to content

Conversation

@KyleAMathews
Copy link
Collaborator

@KyleAMathews KyleAMathews commented Jan 15, 2026

Summary

Fixes SQL compilation for on-demand sync to properly encode column names using the configured columnMapper. Previously, queries with camelCase property names would generate incorrect SQL when snakeCamelMapper was configured. Also adds developer warnings when on-demand collections are passed directly to useLiveQuery without a query builder.


Root Cause

When using snakeCamelMapper (which converts between camelCase JS properties and snake_case DB columns), the SQL compiler was not applying the encoding transformation. For example:

// User writes a query using camelCase (JS convention):
q.from({ p: programs }).where(({ p }) => p.programTemplateId === someId)

// SQL was incorrectly generated as:
// WHERE "programTemplateId" = $1  ❌ (column doesn't exist in Postgres)

// Should be:
// WHERE "program_template_id" = $1  ✅

The columnMapper.encode function existed and was used during data sync, but was never passed to the SQL compilation step.

Approach

SQL Compiler Enhancement:

  • Added optional encodeColumnName parameter to compileSQL()
  • Threading the encoder through all expression compilation functions (compileBasicExpression, compileOrderBy, compileFunction)
  • Applied encoding in quoteIdentifier() before quoting column names

Integration:

  • createLoadSubsetDedupe() now receives shapeOptions.columnMapper?.encode
  • All compileSQL() calls pass the compile options when encoding is available

Developer Experience:

  • Added console warnings in React, Vue, and Svelte useLiveQuery hooks when an on-demand collection is passed directly (which loads no data since there's no predicate)

Key Invariants

  1. Column names in generated SQL must match the actual database column names
  2. The columnMapper transformation must be applied consistently: during sync (already worked) AND during query compilation (this fix)
  3. When no columnMapper is configured, column names pass through unchanged

Non-goals

  • Did not change how columnMapper works for data sync (already correct)
  • Did not add runtime validation that column names exist in the schema
  • Did not make the on-demand + direct collection usage an error (just a warning)

Trade-offs

Warning vs Error for on-demand misuse: Chose a warning because some developers might intentionally pass an on-demand collection and rely on external mechanisms to load data. An error would be breaking; a warning educates without blocking.


Verification

pnpm test --filter @tanstack/electric-db-collection

The new tests in sql-compiler.test.ts verify:

  • camelCase → snake_case encoding in WHERE clauses
  • Encoding in ORDER BY clauses
  • Encoding in IS NULL / IS NOT NULL expressions
  • Encoding in nested AND/OR expressions
  • Encoding in LIKE expressions
  • No transformation when encoder is absent

Files Changed

File Change
packages/electric-db-collection/src/sql-compiler.ts Added ColumnEncoder type and CompileSQLOptions interface; threaded encodeColumnName through all compilation functions
packages/electric-db-collection/src/electric.ts Passed shapeOptions.columnMapper?.encode to createLoadSubsetDedupe() and through to compileSQL()
packages/electric-db-collection/tests/sql-compiler.test.ts Added 9 tests for column name encoding scenarios
packages/react-db/src/useLiveQuery.ts Added warning when on-demand collection passed directly
packages/vue-db/src/useLiveQuery.ts Added warning when on-demand collection passed directly
packages/svelte-db/src/useLiveQuery.svelte.ts Added warning when on-demand collection passed directly

Investigation of Discord-reported issues with snake_case to camelCase
column mapping when using Electric collections with on-demand sync.

Key findings:
- columnMapper transformations not applied during SQL compilation
- compileSQL uses TypeScript property names (camelCase) directly
- Electric's structured expression support (whereExpr) not utilized
- Results in PostgreSQL errors expecting snake_case column names

Includes reproduction steps, root cause analysis, and proposed solutions.
…lation

When using columnMapper (e.g., snakeCamelMapper()) with on-demand sync mode,
camelCase column names in queries were not being converted back to snake_case
before sending to PostgreSQL.

Root cause: The SQL compiler quoted identifiers directly using TypeScript
property names without applying the columnMapper's encode function.

Changes:
- Add encodeColumnName option to compileSQL() to transform column names
- Pass columnMapper.encode from shapeOptions through createLoadSubsetDedupe
- Apply encoding in quoteIdentifier() before quoting
- Add comprehensive tests for column name encoding

This fixes the issue where queries like eq(d.programTemplateId, value) would
generate "programTemplateId" = $1 instead of "program_template_id" = $1.

Fixes: Discord-reported issue with snakeCamelMapper in on-demand mode
… directly to useLiveQuery

Add console warning when a collection with syncMode "on-demand" is passed
directly to useLiveQuery. In on-demand mode, data is only loaded when
queries with predicates request it. Passing the collection directly doesn't
provide any predicates, so no data loads.

The warning guides users to either:
1. Use a query builder function: useLiveQuery((q) => q.from({c: collection}))
2. Switch to syncMode "eager" for automatic sync

This helps prevent confusion when users expect data to appear but nothing
loads due to the on-demand sync behavior.
@changeset-bot
Copy link

changeset-bot bot commented Jan 15, 2026

⚠️ No Changeset found

Latest commit: 3ee5a1e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Jan 15, 2026

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1141

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1141

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1141

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1141

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1141

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1141

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1141

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1141

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1141

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1141

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1141

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1141

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1141

commit: 3ee5a1e

@github-actions
Copy link
Contributor

Size Change: 0 B

Total Size: 90.5 kB

ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.39 kB
./packages/db/dist/esm/collection/changes.js 1.19 kB
./packages/db/dist/esm/collection/events.js 388 B
./packages/db/dist/esm/collection/index.js 3.32 kB
./packages/db/dist/esm/collection/indexes.js 1.1 kB
./packages/db/dist/esm/collection/lifecycle.js 1.67 kB
./packages/db/dist/esm/collection/mutations.js 2.34 kB
./packages/db/dist/esm/collection/state.js 3.46 kB
./packages/db/dist/esm/collection/subscription.js 3.62 kB
./packages/db/dist/esm/collection/sync.js 2.38 kB
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.49 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/index.js 2.69 kB
./packages/db/dist/esm/indexes/auto-index.js 742 B
./packages/db/dist/esm/indexes/base-index.js 766 B
./packages/db/dist/esm/indexes/btree-index.js 1.93 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.1 kB
./packages/db/dist/esm/indexes/reverse-index.js 513 B
./packages/db/dist/esm/local-only.js 837 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/builder/functions.js 733 B
./packages/db/dist/esm/query/builder/index.js 4.08 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 1.05 kB
./packages/db/dist/esm/query/compiler/evaluators.js 1.42 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 1.87 kB
./packages/db/dist/esm/query/compiler/index.js 1.96 kB
./packages/db/dist/esm/query/compiler/joins.js 2 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.45 kB
./packages/db/dist/esm/query/compiler/select.js 1.06 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/ir.js 673 B
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-config-builder.js 5.4 kB
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/collection-subscriber.js 1.93 kB
./packages/db/dist/esm/query/live/internal.js 145 B
./packages/db/dist/esm/query/optimizer.js 2.56 kB
./packages/db/dist/esm/query/predicate-utils.js 2.97 kB
./packages/db/dist/esm/query/subset-dedupe.js 921 B
./packages/db/dist/esm/scheduler.js 1.3 kB
./packages/db/dist/esm/SortedMap.js 1.3 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 924 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/comparison.js 852 B
./packages/db/dist/esm/utils/cursor.js 457 B
./packages/db/dist/esm/utils/index-optimization.js 1.51 kB
./packages/db/dist/esm/utils/type-guards.js 157 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

Size Change: 0 B

Total Size: 3.47 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.12 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 559 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

@ButlerBorst
Copy link

Hey hey! Long time follower first time commenter.

I reported this bug 1/15 after noticing it the day prior. I've installed this PR locally and can confirm the fix works perfectly. Thanks for all y'all are doing. Keep up the good work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Ready for review

Development

Successfully merging this pull request may close these issues.

4 participants