Skip to content

Conversation

kernelism
Copy link

This PR fixes a bug where Series.describe() fails on certain ExtensionArray dtypes such as pint[kg], due to attempting to cast the result to Float64Dtype. This is because some of the produced statistics are not castable to float, which raises errors like DimensionalityError.

We now avoid forcing a Float64Dtype return dtype when the EA’s scalar values cannot be safely cast. Instead:

If the EA produces outputs with mixed dtypes, the result is returned with dtype=None.

@jbrockmendel This is a new PR with the comments from 61760 addressed. Apologies for the duplication.

@jbrockmendel
Copy link
Member

Looks like this breaks some other tests

@simonjayhawkins simonjayhawkins added Bug ExtensionArray Extending pandas with custom dtypes or arrays. labels Jul 22, 2025
@andrewgsavage
Copy link

I think the EA needs an attribute to tell describe what the dtype should be to avoid using object

@jbrockmendel
Copy link
Member

I think the EA needs an attribute to tell describe what the dtype should be to avoid using object

i suspect this may be a use case for the just-introduced EA._cast_pointwise_result

- Bug in :meth:`Index.sort_values` when passing a key function that turns values into tuples, e.g. ``key=natsort.natsort_key``, would raise ``TypeError`` (:issue:`56081`)
- Bug in :meth:`MultiIndex.fillna` error message was referring to ``isna`` instead of ``fillna`` (:issue:`60974`)
- Bug in :meth:`Series.describe` where median percentile was always included when the ``percentiles`` argument was passed (:issue:`60550`).
- Bug in :meth:`Series.describe` where statistics with multiple dtypes for ExtensionArrays were coerced to ``float64`` which raised a ``DimensionalityError``` (:issue:`61707`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DimensionalityError is specific to pint i think, so not relevant to most users

@mroeschke
Copy link
Member

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

@mroeschke mroeschke closed this Oct 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug ExtensionArray Extending pandas with custom dtypes or arrays.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: .describe() doesn't work for EAs

5 participants