-
Notifications
You must be signed in to change notification settings - Fork 167
Description
Deprecation warnings in Pandas CSV Reader (pandas_extensions.py) due to deprecated pandas arguments.
Current behavior
Running the Hamilton test suite on the latest main branch generates multiple FutureWarning messages from pandas.read_csv, triggered by deprecated keyword arguments in PandasCSVReader:
keep_date_colverbosedelim_whitespace
These warnings originate from:
hamilton/plugins/pandas_extensions.py inside PandasCSVReader._get_loading_kwargs().
Stack Traces
Example warnings:
FutureWarning: The 'keep_date_col' keyword in pd.read_csv is deprecated and will be removed in a future version.
FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use sep='\s+' instead.
FutureWarning: The 'verbose' keyword in pd.read_csv is deprecated and will be removed in a future version.
Steps to replicate behavior
- Run:
pytest -q - Observe pandas
FutureWarningoutput from CSV-related tests
Library & System Information
- Hamilton: latest
main - pandas: 2.2.x
- Python: 3.11
- macOS
Expected behavior
Hamilton should avoid passing deprecated arguments to pandas.read_csv, preventing warnings and ensuring compatibility with future pandas versions.
Specifically:
- Remove deprecated args (
keep_date_col,verbose) - Replace
delim_whitespace=Truewith recommendedsep=r"\s+" - Ensure test suite runs with zero deprecation warnings from pandas
Additional context
Fixing this will:
- Clean up test output
- Improve compatibility with pandas ≥ 2.0
- Prevent potential future runtime errors when pandas removes these parameters