PERF: future_stack=True with non-MulitIndex columns

[rhshadrach]

• closes PERF: df.unstack() is 500 times slower since pandas>=2.1  #58391 (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

df = pd.DataFrame(np.random.random(size=(10000, 100)))
%timeit df.unstack()
17.2 ms ± 159 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <--- PR
205 ms ± 3.37 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)     <--- main

[rhshadrach]

@jbrockmendel - the failing tests are:

FAILED pandas/tests/extension/test_numpy.py::TestNumpyExtensionArray::test_stack[float-True-columns0] - AssertionError: (<class 'numpy.dtypes.Float64DType'>, dtype('float64')) vs (<class 'pandas.core.dtypes.dtypes.NumpyEADtype'>, NumpyEADtype('float64'))
FAILED pandas/tests/extension/test_numpy.py::TestNumpyExtensionArray::test_stack[object-True-columns0] - AssertionError: (<class 'numpy.dtypes.ObjectDType'>, dtype('O')) vs (<class 'pandas.core.dtypes.dtypes.NumpyEADtype'>, NumpyEADtype('object'))

This happens because of some monkey-patching

https://github.com/pandas-dev/pandas/blame/3b48b17e52f3f3837b9ba8551c932f44633b5ff8/pandas/tests/extension/test_numpy.py#L72-L73

It's not clear to me if the code should handle this case, or if this behavior is okay and the test should be modified. Was wondering if you had any experience dealing with this.

[rhshadrach]

Not sure if there is a more canonical way to get the dtype in this case.

[jbrockmendel]

The behavior is fine, the test should be patched

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

[mroeschke]

Thanks @rhshadrach