WIP: predicate pushdown to sync

[kevin-dp]

No description provided.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 617988c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[samwillis]

@kevin-dp I rebased this branch on main and fixed the conflicts where we have refactored the Collection class.
(I have a backup of the old branch)

[samwillis]

This is close, we just need to implement config options.

We had discussed two new modes for collections:

1. lazy-permanent: never load the initial snapshot, only load the incremental snapshots as needed.
2. lazy-initial: the intention of this was to allow the user to load snapshots white the whole collection synced in the background. So it's marked as "ready" immediately, and the uses the incremental snapshots until its fully loaded.

We then have the current mode where the collection loads the full snapshot and becomes "ready" when done.

These new modes map to Electric collections well, but maybe less well to Query collections.

I don't like this naming, and I also think we should consider splitting the config into multiple dimensions rather than combining them.

I think the dimensions are:

1. sync an initial snapshot or not
2. allow incremental sync via snapshots (this is essentially implied by the presence of a onLoadMore callback, but that currently always exists on an electric collection and so can't be disabled)
3. mark a collection as "ready" immediately, or wait for the initial snapshot to complete

The planned options did not cover the "lazy load the initial snapshot, mark ready immediately, but there is no incremental snapshots" as an option. This is useful for situations where you want to render a main collection immediately, but a joined one can come in later. (we talked about this being a mode of a join, but maybe its a mode of a collection?)

We also need to decide what options apply to all collections, and what to specific collection types.

[kevin-dp]

I'm closing this PR because i split it into 2 separate PRs: #617 (for the ts/db changes) and #618 (for the changes to the Electric collection).