Skip to content

Conversation

Chen-Yuan-Lai
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

Related #16324 #16617

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

No

@github-actions github-actions bot added the core Core DataFusion crate label Jul 30, 2025
@Chen-Yuan-Lai Chen-Yuan-Lai force-pushed the migrate_core_test_to_insta_part_3 branch from 16959ba to 5cb1e98 Compare August 3, 2025 17:11
@Chen-Yuan-Lai Chen-Yuan-Lai force-pushed the migrate_core_test_to_insta_part_3 branch from 5cb1e98 to 0e07645 Compare August 4, 2025 09:42
@Chen-Yuan-Lai Chen-Yuan-Lai marked this pull request as ready for review August 4, 2025 14:40
@Chen-Yuan-Lai
Copy link
Contributor Author

@alamb @blaginin, sorry for the long delay, I think the PR is ready to be reviewed :)

@blaginin blaginin self-requested a review August 10, 2025 10:35
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this PR @Chen-Yuan-Lai -- unfortuately I am not likely going to be able to find the contiguous time needed to review a PR this large.

I wonder if you would be willing to help make it easier to review, by splitting it up into smaller PRs ?

Also I think the structure of some of these tests is pretty gnarly to begin with (e.g. macros with many parameters). Maybe we can work out a better pattern as part of this work if we are going to rewrite the whole thing anyways

Maybe we can try to port a few tests in one of the files to use iinsta to make sure we are good with the pattern before applying the pattern to the entire thing

For example, a more explicit Test style struct might make the options being tested more efficient.

I know this is a big ask, but I think getting these tests into better shape would really help the project

/// when the flag `prefer_existing_sort` is `true` for bounded cases.
/// * `$PLAN`: The plan to optimize.
/// * `$SOURCE_UNBOUNDED`: Whether the given plan contains an unbounded source.
macro_rules! assert_optimized_in_all_boundedness_situations {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yeah, I remember this macro -- this is crazy

@Chen-Yuan-Lai
Copy link
Contributor Author

Maybe we can try to port a few tests in one of the files to use iinsta to make sure we are good with the pattern before applying the pattern to the entire thing

For example, a more explicit Test style struct might make the options being tested more efficient.

I know this is a big ask, but I think getting these tests into better shape would really help the project

Sure! I will try it soon

@alamb
Copy link
Contributor

alamb commented Aug 14, 2025

Thank you so much @Chen-Yuan-Lai

@alamb
Copy link
Contributor

alamb commented Sep 29, 2025

Any idea if you will be able to split this PR into smaller pieces @Chen-Yuan-Lai ?

I can try to get an AI agent to do it too if you prefer

@Chen-Yuan-Lai
Copy link
Contributor Author

@alamb Sure, feel free to use an AI agent to split this PR! Sorry for the long delay - I've been short on time. Thanks for offering to help with this!

# Conflicts:
#	datafusion/core/tests/physical_optimizer/enforce_distribution.rs
@blaginin
Copy link
Contributor

Resolved one minor conflict

@blaginin
Copy link
Contributor

I can try to get an AI agent to do it too if you prefer

happy to do the split too!

@alamb
Copy link
Contributor

alamb commented Oct 1, 2025

I can try to get an AI agent to do it too if you prefer

happy to do the split too!

that would be great too -- thank you !

@alamb
Copy link
Contributor

alamb commented Oct 3, 2025

I ported part of the sort_enforcement tests here:

# Conflicts:
#	datafusion/core/tests/physical_optimizer/enforce_sorting.rs
#	datafusion/core/tests/physical_optimizer/partition_statistics.rs
@blaginin
Copy link
Contributor

blaginin commented Oct 6, 2025

i'll chunk this into more prs

@alamb
Copy link
Contributor

alamb commented Oct 6, 2025

i'll chunk this into more prs

😍

@alamb
Copy link
Contributor

alamb commented Oct 6, 2025

I'll prioritize getting them reviewed

# Conflicts:
#	datafusion/core/tests/physical_optimizer/replace_with_order_preserving_variants.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants