Skip to content

Conversation

renato2099
Copy link
Contributor

Which issue does this PR close?

Closes #1234

Rationale for this change

Remove duplicated column when using window functions.

What changes are included in this PR?

Aliasing the window expression at the datafusion-python api such that over in datafusion's dataframe/mod.rs the current behavior is still honored.

Are there any user-facing changes?

no

@renato2099
Copy link
Contributor Author

renato2099 commented Sep 14, 2025

Hi @timsaucer @kosiew , do you think you could please review this small PR?

@kosiew
Copy link
Contributor

kosiew commented Sep 16, 2025

hi @renato2099

The changes in this PR replaced DataFrame.with_column with a select-based implementation that filters out the target column and pushes the aliased expression at the end of the projection list. This reorders the DataFrame columns (a, b, c, e, d), leading test_named_struct to fail because it expects the replaced column d to remain in its original position (a, b, c, d, e).

Can you amend the code so that the tests pass?

@timsaucer
Copy link
Member

Thank you for the PR. This is a good work around, but I think we should fix it properly in our upstream repo instead.

@timsaucer
Copy link
Member

Would you be willing to look at this in our upstream repository and fix it there? apache/datafusion#17630

@renato2099
Copy link
Contributor Author

I think we should fix it properly in our upstream repo instead.

yeah I was wondering about this myself 😅 but I wasn't sure where the problem was really coming from, so I went for this

Would you be willing to look at this in our upstream repository and fix it there? apache/datafusion#17630

for sure! let me take a stab at this

@renato2099 renato2099 closed this Sep 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Display of Lag With with_column uses autogenerated name and provided name

3 participants