Skip to content

TST: Add missing skips for unavailable pyarrow #57397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

TST: Add missing skips for unavailable pyarrow #57397

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2724548
TST: Add missing skips for unavailable pyarrow
QuLogic Feb 13, 2024
5af4e21
Merge branch 'main' into test-skip-pyarrow
mroeschke Oct 31, 2024
7f83935
Merge branch 'main' into test-skip-pyarrow
mroeschke Oct 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pandas/core/arrays/arrow/accessors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def _is_valid_pyarrow_dtype(self, pyarrow_dtype) -> bool:

def _validate(self, data) -> None:
dtype = data.dtype
if not isinstance(dtype, ArrowDtype):
if not pa_version_under10p1 and not isinstance(dtype, ArrowDtype):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm only unsure about this one; ArrowDtype is conditionally imported in this file, but I don't know if this should be if pa_version_under10p1 or not isinstance9dtype, ArrowDtype):?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this this was OK as is?

  • If the type wasn't an ArrowDtype this should raise as the wrong type
  • If the type was ArrowDtype, a user already needed the correct version of pyarrow as it's validated in the ArrowDtype.__init__

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, it's not OK as is. ArrowDtype is conditionally imported, so its use here needs to be guarded. Otherwise you get a test failure:

____________________ TestSeriesMisc.test_inspect_getmembers ____________________
[gw0] linux -- Python 3.12.2 /usr/bin/python3
self = <pandas.tests.series.test_api.TestSeriesMisc object at 0xb66af1c8>
    def test_inspect_getmembers(self):
        # GH38782
        pytest.importorskip("jinja2")
        ser = Series(dtype=object)
        msg = "Series._data is deprecated"
        with tm.assert_produces_warning(
            DeprecationWarning, match=msg, check_stacklevel=False
        ):
>           inspect.getmembers(ser)
.../pandas/tests/series/test_api.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.12/inspect.py:607: in getmembers
    return _getmembers(object, predicate, getattr)
/usr/lib/python3.12/inspect.py:585: in _getmembers
    value = getter(object, key)
.../pandas/core/accessor.py:224: in __get__
    accessor_obj = self._accessor(obj)
.../pandas/core/arrays/arrow/accessors.py:73: in __init__
    super().__init__(
.../pandas/core/arrays/arrow/accessors.py:41: in __init__
    self._validate(data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <pandas.core.arrays.arrow.accessors.ListAccessor object at 0x9924ca38>
data = Series([], dtype: object)
    def _validate(self, data):
        dtype = data.dtype
>       if not isinstance(dtype, ArrowDtype):
E       NameError: name 'ArrowDtype' is not defined
.../pandas/core/arrays/arrow/accessors.py:49: NameError

What I'm unsure of is whether this should raise if (the import condition failed or the type is wrong), or if (the import condition is true and the type is wrong).

# Raise AttributeError so that inspect can handle non-struct Series.
raise AttributeError(self._validation_msg.format(dtype=dtype))

Expand Down
4 changes: 3 additions & 1 deletion pandas/tests/io/formats/style/test_bar.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
import numpy as np
import pytest

from pandas.compat._optional import VERSIONS

from pandas import (
NA,
DataFrame,
Expand Down Expand Up @@ -347,7 +349,7 @@ def test_styler_bar_with_NA_values():


def test_style_bar_with_pyarrow_NA_values():
pytest.importorskip("pyarrow")
pytest.importorskip("pyarrow", VERSIONS["pyarrow"])
data = """name,age,test1,test2,teacher
Adam,15,95.0,80,Ashby
Bob,16,81.0,82,Ashby
Expand Down
4 changes: 4 additions & 0 deletions pandas/tests/io/parser/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ def pyarrow_parser_only(request):
"""
Fixture all of the CSV parsers using the Pyarrow engine.
"""
pytest.importorskip("pyarrow", VERSIONS["pyarrow"])
return request.param()


Expand Down Expand Up @@ -216,6 +217,9 @@ def all_parsers_all_precisions(request):
Fixture for all allowable combinations of parser
and float precision
"""
parser = request.param[0]
if parser.engine == "pyarrow":
pytest.importorskip("pyarrow", VERSIONS["pyarrow"])
return request.param


Expand Down
Loading