Fix MongoDB replication with mixed _id types

[rkistner]

The issue

This is a regression in v1.13.0.

In #163, we changed initial snapshot queries on MongoDB to chunk on _id, using {_id: {$gt: ...}} queries to filter out earlier chunks. The issue is that when the same collection uses different types for the _id field, the $gt operator only returns documents with the same type. That meant that when you have a large collection with for example both string and ObjectId _ids, only the documents with string _id would replicate in the initial snapshot.

This would be visible in the logs, with the collection finishing replicating with mismatching counts, such as:

Replicating "mydb"."mycollection" 12356/~23000

The fix

This changes the snapshot query to { $expr: { $gt: ['$_id', { $literal: ... }] } }. This is almost the same, but the $expr version respects the total ordering of BSON types (same as the sort stage), and doesn't filter out the other types. It still uses the _id index.

[changeset-bot]

🦋 Changeset detected

Latest commit: f6b1d0f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages

Name	Type
@powersync/service-module-mongodb	Patch
@powersync/service-core	Patch
@powersync/service-image	Patch
@powersync/service-schema	Patch
@powersync/service-core-tests	Patch
@powersync/service-module-core	Patch
@powersync/service-module-mongodb-storage	Patch
@powersync/service-module-mysql	Patch
@powersync/service-module-postgres-storage	Patch
@powersync/service-module-postgres	Patch
test-client	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Copilot]

Pull Request Overview

This pull request fixes MongoDB replication issues when collections contain mixed _id types by updating the snapshot query to use a $expr comparison. Key changes include:

• Adding a new test case for chunked snapshots with mixed _id types.
• Updating the snapshot query filter in MongoSnapshotQuery.ts to use $expr with $literal.
• Minor comment improvements and changeset updates.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
modules/module-mongodb/test/src/chunked_snapshot.test.ts	Added a new test case to exercise mixed _id type behavior.
modules/module-mongodb/src/replication/MongoSnapshotQuery.ts	Updated the filter query to properly handle mixed _id types.
modules/module-mongodb/src/replication/ChangeStream.ts	Added a clarifying comment on when iteration stops.
.changeset/shy-pugs-teach.md	Updated changeset message to document the fix.