Skip to content

feat: Add StrictMetricsEvaluator#963

Merged
liurenjie1024 merged 21 commits intoapache:mainfrom
jonathanc-n:add-strict-metrics-evaluator
Feb 26, 2025
Merged

feat: Add StrictMetricsEvaluator#963
liurenjie1024 merged 21 commits intoapache:mainfrom
jonathanc-n:add-strict-metrics-evaluator

Conversation

@jonathanc-n
Copy link
Copy Markdown
Contributor

@jonathanc-n jonathanc-n commented Feb 12, 2025

part of #735. Added StrictMetricsEvaluator.

@jonathanc-n
Copy link
Copy Markdown
Contributor Author

Tests will be added tomorrow.

@jonathanc-n
Copy link
Copy Markdown
Contributor Author

Tests have been added, this implementation follows the pyiceberg implementation. cc @Fokko @liurenjie1024 @ZENOTME

Copy link
Copy Markdown
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@viirya Thanks for working on this, and also copying over the tests 🙌 I've left some comments

return ROWS_MUST_MATCH;
}

let mut evaluator = Self::new(data_file, schema);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still learning rust, why is this one mutable?

Copy link
Copy Markdown
Contributor Author

@jonathanc-n jonathanc-n Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bound predicate visitor takes in a mutable reference of the evaluator. in rust you have to declare as mutable as well as pass it in with mut to let the compiler know the function can modify

@jonathanc-n
Copy link
Copy Markdown
Contributor Author

@Fokko Thanks for the review. I got a bit mixed up with the logic with only nulls vs may contain null 😆. Should be good now

Copy link
Copy Markdown
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonathanc-n for this great pr, and all the tests! Just some minor suggestions, others looks great!

data_file: &'a DataFile,
}

impl<'a> StrictMetricsEvaluator<'a> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it has a lot of duplication with InclusiveMetricsEvaluator, maybe we could unify them in future.

@jonathanc-n
Copy link
Copy Markdown
Contributor Author

@liurenjie1024 @Fokko Thank you for the reviews, problems should be resolved.

Copy link
Copy Markdown
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonathanc-n for this pr!

@liurenjie1024 liurenjie1024 merged commit eb4c668 into apache:main Feb 26, 2025
17 checks passed
@jonathanc-n jonathanc-n deleted the add-strict-metrics-evaluator branch March 28, 2025 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants