Skip to content

Conversation

@hu-ahmed
Copy link
Contributor

In the current Normalized mapper, partial delete events (AttributeDeleted, FeatureDeleted, etc.) are dropped, and merge‑patch nulls are filtered out. This makes it impossible for downstream consumers to distinguish “field never existed” from “field was explicitly deleted.”

Solution
Introduce an opt‑in mapper option includeDeletedFields. When enabled:

Partial delete events are mapped and recorded in _deletedFields.
Merge‑patch nulls are detected before filtering and recorded in _deletedFields.
The default behavior is unchanged unless the option is enabled.
This preserves backward compatibility while allowing delete‑aware ingestion pipelines.

Copy link
Member

@thjaeckle thjaeckle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hu-ahmed - great addition when making use of the normalized mapper.

I just have one minor remark where the added boolean is not added in the toString() - apart from that the PR LGTM 👍

@hu-ahmed hu-ahmed force-pushed the extend-mapper-to-include-delete-fields branch from b2f1f13 to ff8b0d9 Compare January 27, 2026 15:32
@thjaeckle thjaeckle moved this to Waiting for Approval in Ditto Planning Jan 28, 2026
Copy link
Member

@thjaeckle thjaeckle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the enhancement @hu-ahmed

I just added a small change - we don't want to pass Optional as a parameter to methods - made that a @Nullable parameter instead.

@thjaeckle thjaeckle merged commit df071b8 into eclipse-ditto:master Jan 28, 2026
3 checks passed
@github-project-automation github-project-automation bot moved this from Waiting for Approval to Done in Ditto Planning Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants