Skip to content

deal with pandas new nan_is_na mode #3160

@MarcoGorelli

Description

@MarcoGorelli

In pandas 3.0 there will be a nan_is_na mode: pandas-dev/pandas#62040

If it's True (default), then nan will be indistuinguishable from na. If False, they will be distinguished, and isna will only report in NA values

We need to make a decision here about what to do for pandas-like

I think what would be most natural would be to just keep doing what we're doing:

  • for nan, return whatever s != s outputs
  • for na, return whatever s.isna() outputs

Note that a consequence of this is that when nan_is_na is True, then s_nw.is_nan() would differ from s_nw.is_null():

In [1]: import pandas as pd

In [2]: pd.set_option("mode.nan_is_na", True)

In [3]: s = pd.Series([-1, 1, None], dtype='Int64') ** .5

In [4]: s
Out[4]: 
0    <NA>
1     1.0
2    <NA>
dtype: Float64

In [5]: s.isna()
Out[5]: 
0     True
1    False
2     True
dtype: bool

In [6]: s != s
Out[6]: 
0     <NA>
1    False
2     <NA>
dtype: boolean

but I think that's OK, if pandas wants to mask NaN values to be NA ones, then that's their decision

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions