Skip to content

Conversation

@samwillis
Copy link
Collaborator

@samwillis samwillis commented Jun 29, 2025

stacked on #185 - brings more composability to queries, and looking for feedback.

The first part is essentially done, a defineQuery function that allows to you define a query outside of creating a live query collection. It can then be passed to either liveQueryCollectionOptions or to useLiveQuery as both a root from or used in a join along with any number of rested subqueries.

// Create a predefined query somewhere
const predefinedQuery = defineQuery((q) =>
  q
    .from({ persons: collection })
    .where(({ persons }) => gt(persons.age, 30))
    .select(({ persons }) => ({
      id: persons.id,
      name: persons.name,
      age: persons.age,
    }))
)

// Then use it later
const {data} =  useLiveQuery(predefinedQuery)

The second part is very much draft, I've been looking at ways to define parts of a query for later composing. after much experimenting this is what I have, an api to help define reusable callbacks to pass to the any method that takes a callback (where/having/orderby/join etc).

It's slightly odd, in order to make the type inference work:

type User = {
  name: string
  age: number
}

// Can be used in where/having/groupBy, anything that has a callback returning an expression
export const userIsAdult = defineForRow<{ user: User }>().callback(({ user }) =>
  gt(user.age, 18)
)

// Can be used in select
export const userNameUpper = defineForRow<{ user: User }>().select(
  ({ user }) => ({
    name: upper(user.name),
  })
)

const users = useLiveQuery((q) => 
  q
    .from({ user: usersCollection }) // usersCollection is a Collection<User>
    .where(userIsAdult)
    .select(userNameUpper)
)

Note how you have to call defineForRow<interface>() first, then a method on the returned object. This is required to make the full type inference work, mostly for the return type of the callback.

(I've named it defineForRow, but I don't like that name)

@changeset-bot
Copy link

changeset-bot bot commented Jun 29, 2025

⚠️ No Changeset found

Latest commit: 436196d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Jun 29, 2025

@tanstack/db-example-react-todo

npm i https://pkg.pr.new/@tanstack/db@216
npm i https://pkg.pr.new/@tanstack/db-collections@216
npm i https://pkg.pr.new/@tanstack/react-db@216
npm i https://pkg.pr.new/@tanstack/vue-db@216

commit: 436196d

@github-actions
Copy link
Contributor

github-actions bot commented Jun 29, 2025

Size Change: +233 B (+0.79%)

Total Size: 29.9 kB

Filename Size Change
./packages/db/dist/esm/index.js 554 B -15 B (-2.64%)
./packages/db/dist/esm/query/builder/ref-proxy.js 840 B -11 B (-1.29%)
./packages/db/dist/esm/query/live-query-collection.js 2.02 kB +46 B (+2.33%)
./packages/db/dist/esm/query/builder/composables.js 213 B +213 B (new file) 🆕
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection.js 7.69 kB
./packages/db/dist/esm/deferred.js 230 B
./packages/db/dist/esm/errors.js 150 B
./packages/db/dist/esm/optimistic-action.js 294 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/builder/functions.js 523 B
./packages/db/dist/esm/query/builder/index.js 3.19 kB
./packages/db/dist/esm/query/compiler/evaluators.js 1.34 kB
./packages/db/dist/esm/query/compiler/group-by.js 2.08 kB
./packages/db/dist/esm/query/compiler/index.js 1.4 kB
./packages/db/dist/esm/query/compiler/joins.js 1.16 kB
./packages/db/dist/esm/query/compiler/order-by.js 933 B
./packages/db/dist/esm/query/compiler/select.js 657 B
./packages/db/dist/esm/query/ir.js 310 B
./packages/db/dist/esm/SortedMap.js 1.24 kB
./packages/db/dist/esm/transactions.js 1.33 kB

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Jun 29, 2025

Size Change: 0 B

Total Size: 665 B

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 152 B
./packages/react-db/dist/esm/useLiveQuery.js 513 B

compressed-size-action::react-db-package-size

@KyleAMathews
Copy link
Collaborator

KyleAMathews commented Jun 29, 2025

Really nice!

Combining them is nice too:

const usersQuery = defineQuery(q => q.from({ user: usersCollection }))

const users = useLiveQuery(usersQuery
    .where(userIsAdult)
    .select(userNameUpper)
)

Why define vs. create? I generally prefer sticking with as few of verbs as possible and create seems fine here? E.g. createQuery and createFragment

@samwillis
Copy link
Collaborator Author

@KyleAMathews

As there is two steps, crating a query definition and then using that query definition to crate a live query that materialises to a collection (once it started), I wanted to give it a distinct name to indicate that.

A defined query can be used in multiple createLiveQueryOption or useLiveQuery calls. Calling it creteLiveQuery felt like it would conflate the two.

I suppose it's a bit like a class, you define it once, then create instances of it with arguments... (FYI you can use defineQuery inside a function to make it parameterised)

I'm very open to naming suggestions on both!

@samwillis samwillis marked this pull request as ready for review June 30, 2025 10:43
@samwillis
Copy link
Collaborator Author

I've fleshed out jsdoc and tests for both methods.

We just need to decide on naming.

@kevin-dp
Copy link
Contributor

I was wondering why we need a callback based approach to define reusable queries. I went into the implementation and defineQuery is implemented like this:

export function defineQuery<TQueryBuilder extends QueryBuilder<any>>(
  fn: (builder: InitialQueryBuilder) => TQueryBuilder
): TQueryBuilder {
  return fn(new BaseQueryBuilder())
}

So the callback approach is unnecessary here as the user could directly define the query like this:

new BaseQueryBuilder()
  .from({ persons: collection })
  .where(({ persons }) => gt(persons.age, 30))
  .select(({ persons }) => ({
    id: persons.id,
    name: persons.name,
    age: persons.age,
  }))

If we don't like the verbosity of new BaseQueryBuilder() we could hide it behind a function such that it becomes:

q()
  .from({ persons: collection })
  .where(({ persons }) => gt(persons.age, 30))
  .select(({ persons }) => ({
    id: persons.id,
    name: persons.name,
    age: persons.age,
  }))

* )
* ```
*/
export function defineForRow<TNamespaceRow extends NamespacedRow>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The defineForRow function seems to be used only for typing purposes. I'm against introducing runtime functions for compile time concerns (like typing).

* query.where(userIsActive)
* ```
*/
const callback = <TResult>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function as well as select are confusing me. Both are basically identity functions. I don't see how this is useful. Seems like unnecessary wrapping.

Since they are identity functions, it seems that your example from the PR description is equivalent to:

type User = {
  name: string
  age: number
}

const userIsAdult = ({ user: User }) => gt(user.age, 18)

export const userNameUpper = ({ user }) => ({
  name: upper(user.name),
})

const users = useLiveQuery((q) => 
  q
    .from({ user: usersCollection }) // usersCollection is a Collection<User>
    .where(userIsAdult)
    .select(userNameUpper)
)

This seems more compact and simpler than the original code in the PR description.

@kevin-dp
Copy link
Contributor

kevin-dp commented Jun 30, 2025

I was chatting with @samwillis about this PR. I'm proposing this simpler approach:

const query = (q: QueryBuilder) =>
  q
    .from({ persons: collection })
    .where(({ persons }) => gt(persons.age, 30))
    .select(({ persons }) => ({
      id: persons.id,
      name: persons.name,
      age: persons.age,
    })))

// Then use it later
const {data} =  useLiveQuery(query)

No need to call defineQuery but since we've moved the query outside the useLiveQuery hook and into a query variable such that it can be reused, we now have to explicitly type the q argument. This is only a concern for people using TypeScript, and i don't think it's a big deal that you'll have to type the argument as q: QueryBuilder. Internally, we can support this by extending the from and join methods with an additional overload for functions. So if the arguments that is passed is a function, we need to call it with a new query builder.

Regarding subqueries i would propose this:

// Inside a query we get a reference
const userIsAdult = ({ user: Ref<User> }) => gt(user.age, 18)

export const userNameUpper = ({ user: Ref<User> }) => ({
  name: upper(user.name),
})

const users = useLiveQuery((q) => 
  q
    .from({ user: usersCollection }) // usersCollection is a Collection<User>
    .where(userIsAdult)
    .select(userNameUpper)
)

This is the simplest approach since you don't have to call any additional functions like defineForRow. Those functions were introduced purely for typing purposes and we can solve the typing issues by correctly typing the argument of the subqueries (in this example using { user: Ref<User> }).

Essentially, my proposal boils down to using Javascript's built-in abstraction mechanisms rather than introducing our own (defineQuery, defineForRow, etc.). As for the types, we can type the (sub)queries directly.

@DawidWraga
Copy link
Contributor

I was thinking the exact same thing as Kevin.

@samwillis
Copy link
Collaborator Author

Yep, we chatted it over and are going to go with the simpler option. It's then just docs and renaming a few of the internal types 👍

Closing this

@samwillis samwillis closed this Jul 1, 2025
@samwillis samwillis deleted the query2-defineQuery branch July 22, 2025 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants