Skip to content

Conversation

@kumarUjjawal
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

  • The goal is to replace the old statistics API with the new partition_statistics

What changes are included in this PR?

  • implemented the partition_statistics API for SortMergeJoinExec
  • Removed the special case that returned unknown statistics for specific partitions.
  • Added test_partition_statistics to verify both aggregate and specific partition statistics work correctly

Are these changes tested?

  • Addes new tests

Are there any user-facing changes?

@github-actions github-actions bot added the physical-plan Changes to the physical-plan crate label Dec 30, 2025
@kumarUjjawal
Copy link
Contributor Author

cc @xudong963

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kumarUjjawal I feel the test might be expanded

For aggregate statistics (partition = None): Should return meaningful statistics computed from both inputs

For partition-specific statistics (partition = Some(0)): Currently returns unknown statistics, but ideally should return statistics specific to that partition

Currently it just checks column num?

@kumarUjjawal
Copy link
Contributor Author

kumarUjjawal commented Dec 31, 2025

Thanks @kumarUjjawal I feel the test might be expanded

For aggregate statistics (partition = None): Should return meaningful statistics computed from both inputs

For partition-specific statistics (partition = Some(0)): Currently returns unknown statistics, but ideally should return statistics specific to that partition

Currently it just checks column num?

Good point. Thank you for the feedback.

@kumarUjjawal kumarUjjawal requested a review from comphead January 4, 2026 07:05
Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kumarUjjawal

@comphead comphead added this pull request to the merge queue Jan 4, 2026
Merged via the queue into apache:main with commit e0b4e8d Jan 4, 2026
32 checks passed
@xudong963
Copy link
Member

@kumarUjjawal @comphead Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants