MongoDB: Fix replication of undefined values

[rkistner]

When replicating Postgres rows, in some edge cases we have toasted (missing) values in rows. We use undefined in our internal "current_data" storage to indicate these values are missing, and wait until we have all values before processing the row in sync rules.

With MongoDB replication, we always get the full document - we don't run into the case above. However, we had a bug where undefined values were kept as-is when replicating, triggering that logic. This caused rows with undefined values to not be replicated, without any indication why.

Our tests appeared to check for this case, but didn't in reality. MongoDB deprecated the undefined value in BSON, making it very difficult to actually insert an undefined value in the database. Our tests actually only tested for null, and never for undefined. The tests now use a nasty workaround using mapReduce to insert an undefined value.

This only changes the behavior for top-level undefined values. Behavior for undefined in nested objects and arrays is unchanged:

// top-level: converted to null
{value: undefined} -> {value: null}
// array: converted to null
{array: [undefined] -> {array: "[null]"}
// nested document: treated as missing key
{nested: {value: undefined} -> {nested: "{}"}

The the Discord thread reporting this issue.

[changeset-bot]

🦋 Changeset detected

Latest commit: 513b51e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages

Name	Type
@powersync/service-module-mongodb	Patch
@powersync/service-core	Patch
@powersync/service-image	Patch
@powersync/service-module-mysql	Patch
@powersync/service-module-postgres	Patch
test-client	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[stevensJourney]

LGTM