ENH: usecols takes input order for read_csv implementation review #61967
Conversation
… complicated usecols order
eicchen
changed the title
| False, | ||
| ": bool\n " | ||
| "Whether usecols parameter will use order of input when " | ||
| "making a DataFrame. \n This feature will be default in pandas 3.0" |
There was a problem hiding this comment.
i think if this option is being introduced in 3.0 it won't be enforced until 4.0
There was a problem hiding this comment.
Sounds good, is that the only concern with the implementation? If so I'll go ahead and apply it to other functions and update the docs.
I can also add modifying the flag to 4.0 milestones too, idk if there is a timeline for it just yet but figured better to add it while it's fresh
There was a problem hiding this comment.
i think we said this should default to "warn"?
There was a problem hiding this comment.
Yeah, have it pop up and warn users that this was going to be a future change coming in 4.0. Unless you're talking about something else?
| @pytest.mark.parametrize("usecols", [(3, 0, 2), ("d", "a", "c")]) | ||
| @pytest.mark.parametrize("usecols_use_order", (True, False)) | ||
| def test_usecols_order(all_parsers, usecols, usecols_use_order): | ||
| # TODOE add portion in doc for 3.0 transition |
There was a problem hiding this comment.
| # TODOE add portion in doc for 3.0 transition | |
| # TODO: add portion in doc for 3.0 transition |
|
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
3 participants
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.This is just the implementation for using usecols order for read_csv that I wanted to have people look at before moving to apply it to other places like read_excel and read_clipboard. If it all looks good, I'll go back and add all necessary documentation about future deprecation along with a popup when using usecols. This is mainly just for checking that the implementation doesn't have any glaring issues.
I ran the entire test suite just to be safe and it all looks good. The only thing the errored were some datetime tests that had nothing to do with the changes that I could find.
Oh it is also worth noting that pyarrow already uses the usecols order by default so that's probably worth adding to the documentation regardless