BUG: Inconsistent behavior of `MultiIndex.union` depending on duplicates and names

[torfsen]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

index1_without_name = pd.Index([1, 2])
index1_with_name = pd.Index([1, 2], name="x")
index2_without_duplicates = pd.Index([2, 3])
index2_with_duplicates = pd.Index([2, 3, 2])

multi_index1_without_name = pd.MultiIndex.from_tuples([(1, "a"), (2, "b")])
multi_index1_with_name = pd.MultiIndex.from_tuples([(1, "a"), (2, "b")], names=["x", "y"])
multi_index2_without_duplicates = pd.MultiIndex.from_tuples([(2, "b"), (3, "c")])
multi_index2_with_duplicates = pd.MultiIndex.from_tuples([(2, "b"), (3, "c"), (2, "b")])

# These work
print(index1_without_name.union(index2_without_duplicates))
print(index1_without_name.union(index2_with_duplicates))
print(index1_with_name.union(index2_without_duplicates))
print(index1_with_name.union(index2_with_duplicates))
print(multi_index1_without_name.union(multi_index2_without_duplicates))
print(multi_index1_without_name.union(multi_index2_with_duplicates))
print(multi_index1_with_name.union(multi_index2_without_duplicates))

# This one raises
print(multi_index1_with_name.union(multi_index2_with_duplicates))

Issue Description

For 2 MultiIndex instances i1 and i2, i1.union(i2)behaves inconsistently depending on whetheri1has names and whetheri2` has duplicates:

• If i1 has no names or i2 has no duplicates then i1.union(i2) works as expected
• If i1 has names and i2 has duplicates then i1.union(i2) raises ValueError: cannot join with no overlapping index names

In addition, if i1 and i2 are plain Index instances, then the case that is problematic for MultiIndex (names and duplicates) works as expected.

Expected Behavior

I expect no exception to be raised. The result should contain the duplicate values of the second MultiIndex as duplicates, just as in the other cases for consistency (although personally this did surprise me, but that's a different topic).

Installed Versions

INSTALLED VERSIONS

commit : c888af6
python : 3.12.7
python-bits : 64
OS : Linux
OS-release : 6.14.0-27-generic
Version : #27~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 22 17:38:49 UTC 2
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.3.1
numpy : 1.26.4
pytz : 2024.2
dateutil : 2.9.0.post0
pip : 25.1.1
Cython : None
sphinx : None
IPython : 8.32.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
blosc : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : 3.1.4
lxml.etree : None
matplotlib : None
numba : None
numexpr : 2.10.2
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : 2.9.10
pymysql : None
pyarrow : 18.0.0
pyreadstat : None
pytest : 8.3.4
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : 3.10.1
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2024.2
qtpy : None
pyqt5 : None