add flink support FLIP-510#8452
Draft
irodriguezclaveria wants to merge 6 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support key-only deletes on primary-key table sink (FLIP-510)
Advertise the FLIP-510 keyOnlyDeletes capability on primary-key table sinks so the Flink planner can drop the upstream ChangelogNormalize node when the source produces deletes by key.
This is especially useful for CDC sources, where the UPDATE_AFTER already carries the full row: dropping ChangelogNormalize removes its state and overhead with no loss of correctness.
Gated behind a new table option sink.key-only-deletes.enabled (default false), so existing plans are unchanged unless explicitly opted in. The API only exists in Flink 2.1+, so the behavior is wired through ChangelogModeUtils in paimon-flink1-common (no-op) and paimon-flink2-common (sets the flag), keeping paimon-flink-common compiling against both Flink 1.x and 2.x.
This PR has been tested in a real environment and drops correctly the ChangelogNormalize.