Skip to content

[core] Support remove-record-on-delete for aggregation merge-engine#5360

Merged
yuzelin merged 3 commits intoapache:masterfrom
JackeyLee007:del-record-query
Mar 28, 2025
Merged

[core] Support remove-record-on-delete for aggregation merge-engine#5360
yuzelin merged 3 commits intoapache:masterfrom
JackeyLee007:del-record-query

Conversation

@JackeyLee007
Copy link
Copy Markdown
Contributor

@JackeyLee007 JackeyLee007 commented Mar 27, 2025

Purpose

Linked issue: close #xxx

We use aggregation in special case which needs removing the record when receiving a DELETE operation.

  1. We use --computed_column 'etl_create_time=now()' and field.etl_create_time.aggregate-function = first_non_null_value to
    record the first time when the record ingestd.
  2. It also needs removing the record when ingesting DELETE operation instead of just value retraction.

Tests

API and Format

Documentation

@JackeyLee007 JackeyLee007 changed the title [core] Support remove-record-on-delete for aggregation merge-enging and scaning delta/changelog files in stream mode [core] Support remove-record-on-delete for aggregation merge-engine Mar 27, 2025
@yuzelin yuzelin merged commit fa411a0 into apache:master Mar 28, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants