BUG: Fix Index.equals between object and string

[sanggon6107]

• closes BUG: Can only compare identically-labeled Series objects (string vs. object) #61099
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/v3.0.0.rst file if fixing a bug or adding a new feature.

Description of the code change on Index.equals

On the main branch, Index.equals casts self to object only when self.dtype.na_value is np.nan. The comparison actually succeeds when self.dtype.na_value is np.nan as below.

>>> import pandas as pd
>>> import numpy as np

>>> s1 = pd.Series([1, 2, 3], index=['a', 'b', 'c'])
>>> s2 = pd.Series([4, 5, 6], index=['a', 'b', 'c'])
>>> s2.index = s2.index.astype(pd.StringDtype(storage="pyarrow", na_value=np.nan))

>>> print(s1 < s2)
a    True
b    True
c    True
dtype: bool

However, since doc stated that dtype is not compared, self should be casted regardless of self.dtype.na_value so that self could be compared with other dtypes as desired.

Description of the code change on test_mixed_col_index_dtype

using_infer_string has been removed since I think that result should be string regardless of using_infer_string. This is becaus of the code change made on Index.equals - since Index.equals consider df1.columns is equal to df2.colums, Index.intersection returns self(which is string). You could see the result becomes object(which is the dtype of df2) in case of result = df2 + df1. On the main branch, on the other hand, Index.intersection returns object because Index.equals returns False, and then both self and other are cast to object by _find_common_type_compat. (see L3287 at pandas/core/indexes/base.py)

pandas/pandas/core/indexes/base.py

Lines 3286 to 3290 in 25e6462

	elif self.dtype != other.dtype:
	dtype = self._find_common_type_compat(other)
	this = self.astype(dtype, copy=False)
	other = other.astype(dtype, copy=False)
	return this.intersection(other, sort=sort)

• I created this pull request since @MayurKishorKumar doesn't seem to work on this issue anymore, but please let me know if there is going to be further actions on the previous PR and I am supposed to close this one.

[rhshadrach]

Thanks for the PR!

[rhshadrach]

Can you instead use the fixture string_dtype_no_object throughout these tests.

[sanggon6107]

Thanks @rhshadrach , changed the code.

[rhshadrach]

This condition was added in #56106, I think the na_value part was added just to be conservative.

[sanggon6107]

Thanks! Yes, maybe the bug was only confirmed for pyarrow_numpy at that moment.

[rhshadrach]

Can you also check s_str > s_obj.

[sanggon6107]

Thanks for the feedback! added s_str > s_obj.

[rhshadrach]

lgtm

[rhshadrach]

Thanks @sanggon6107!