Skip to content

Deprecation warnings in Pandas CSV Reader #1417

@iprithv

Description

@iprithv

Deprecation warnings in Pandas CSV Reader (pandas_extensions.py) due to deprecated pandas arguments.

Current behavior

Running the Hamilton test suite on the latest main branch generates multiple FutureWarning messages from pandas.read_csv, triggered by deprecated keyword arguments in PandasCSVReader:

  • keep_date_col
  • verbose
  • delim_whitespace

These warnings originate from:
hamilton/plugins/pandas_extensions.py inside PandasCSVReader._get_loading_kwargs().

Stack Traces

Example warnings:

FutureWarning: The 'keep_date_col' keyword in pd.read_csv is deprecated and will be removed in a future version.
FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use sep='\s+' instead.
FutureWarning: The 'verbose' keyword in pd.read_csv is deprecated and will be removed in a future version.

Steps to replicate behavior

  1. Run: pytest -q
  2. Observe pandas FutureWarning output from CSV-related tests

Library & System Information

  • Hamilton: latest main
  • pandas: 2.2.x
  • Python: 3.11
  • macOS

Expected behavior

Hamilton should avoid passing deprecated arguments to pandas.read_csv, preventing warnings and ensuring compatibility with future pandas versions.

Specifically:

  • Remove deprecated args (keep_date_col, verbose)
  • Replace delim_whitespace=True with recommended sep=r"\s+"
  • Ensure test suite runs with zero deprecation warnings from pandas

Additional context

Fixing this will:

  • Clean up test output
  • Improve compatibility with pandas ≥ 2.0
  • Prevent potential future runtime errors when pandas removes these parameters

Metadata

Metadata

Assignees

Labels

i/opythonPull requests that update python code

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions