Skip to content

DEPR: invalid-for-dtype fill_value in unstack #62289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 8, 2025
Merged

DEPR: invalid-for-dtype fill_value in unstack #62289

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -641,6 +641,7 @@ Other Deprecations
- Deprecated strings ``w``, ``d``, ``MIN``, ``MS``, ``US`` and ``NS`` denoting units in :class:`Timedelta` in favour of ``W``, ``D``, ``min``, ``ms``, ``us`` and ``ns`` (:issue:`59051`)
- Deprecated the ``arg`` parameter of ``Series.map``; pass the added ``func`` argument instead. (:issue:`61260`)
- Deprecated using ``epoch`` date format in :meth:`DataFrame.to_json` and :meth:`Series.to_json`, use ``iso`` instead. (:issue:`57063`)
- Deprecated allowing ``fill_value`` that cannot be held in the original dtype (excepting NA values for integer and bool dtypes) in :meth:`Series.unstack` and :meth:`DataFrame.unstack` (:issue:`12189`, :issue:`53868`)

.. ---------------------------------------------------------------------------
.. _whatsnew_300.prior_deprecations:
Expand Down
28 changes: 26 additions & 2 deletions pandas/core/reshape/reshape.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,10 @@
from pandas._config.config import get_option

import pandas._libs.reshape as libreshape
from pandas.errors import PerformanceWarning
from pandas.errors import (
Pandas4Warning,
PerformanceWarning,
)
from pandas.util._decorators import cache_readonly
from pandas.util._exceptions import find_stack_level

Expand All @@ -28,7 +31,10 @@
needs_i8_conversion,
)
from pandas.core.dtypes.dtypes import ExtensionDtype
from pandas.core.dtypes.missing import notna
from pandas.core.dtypes.missing import (
isna,
notna,
)

import pandas.core.algorithms as algos
from pandas.core.algorithms import (
Expand Down Expand Up @@ -298,7 +304,25 @@ def get_new_values(self, values, fill_value=None):
new_values[:] = fill_value
else:
if not mask_all:
old_dtype = dtype
dtype, fill_value = maybe_promote(dtype, fill_value)
if old_dtype != dtype:
if old_dtype.kind not in "iub":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why we only need to warn for these numeric types.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for int/bool dtypes, we can introduce NaNs which entail upcasting. For all other dtypes we should be able to unstack without having to upcast. same idea as not warning on int/bool in PDEP6

warnings.warn(
# GH#12189, GH#53868
"Using a fill_value that cannot be held in the existing "
"dtype is deprecated and will raise in a future version.",
Pandas4Warning,
stacklevel=find_stack_level(),
)
elif not isna(fill_value):
warnings.warn(
# GH#12189, GH#53868
"Using a fill_value that cannot be held in the existing "
"dtype is deprecated and will raise in a future version.",
Pandas4Warning,
stacklevel=find_stack_level(),
)
new_values = np.empty(result_shape, dtype=dtype)
if not mask_all:
new_values.fill(fill_value)
Expand Down
16 changes: 12 additions & 4 deletions pandas/tests/frame/test_stack_unstack.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,12 @@ def test_unstack_fill(self, future_stack):
)
tm.assert_frame_equal(result, expected)

# From a series with incorrect data type for fill_value
result = data.unstack(fill_value=0.5)
msg = (
"Using a fill_value that cannot be held in the existing dtype is deprecated"
)
with tm.assert_produces_warning(Pandas4Warning, match=msg):
# From a series with incorrect data type for fill_value
result = data.unstack(fill_value=0.5)
expected = DataFrame(
{"a": [1, 0.5, 5], "b": [2, 4, 0.5]}, index=["x", "y", "z"], dtype=float
)
Expand Down Expand Up @@ -161,8 +165,12 @@ def test_unstack_fill_frame(self):
expected["B"] = expected["B"].astype(np.float64)
tm.assert_frame_equal(result, expected)

# From a dataframe with incorrect data type for fill_value
result = df.unstack(fill_value=0.5)
msg = (
"Using a fill_value that cannot be held in the existing dtype is deprecated"
)
with tm.assert_produces_warning(Pandas4Warning, match=msg):
# From a dataframe with incorrect data type for fill_value
result = df.unstack(fill_value=0.5)

rows = [[1, 3, 2, 4], [0.5, 5, 0.5, 6], [7, 0.5, 8, 0.5]]
expected = DataFrame(rows, index=list("xyz"), dtype=float)
Expand Down
Loading