BUG: concatenation is allowing duplicate columns when axis=1

[Sakethsreeram7]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df1=pd.DataFrame({'a':1,'b':2},index=[0])
df2=pd.DataFrame({'a':1,'b':2},index=[0])
df_new=pd.concat([df1,df2],axis=1)

Issue Description

When concatenation is done by axis=1 the duplicate columns are added up without raising an error then this will be very confusing as these are not identified as different

Expected Behavior

If the columns are the same before concatenating then in new df these should be some unique identifier
let's say: a(1), b(1) like this

Installed Versions

Replace this line with the output of pd.show_versions()

[kevkle]

i would like to contribute if thats ok for you

The question is whether an error should be raised or the columns should be renamed ?

[Sakethsreeram7]

columns should be renamed , Thanks!

[quantresearch1]

Hi, in terms of renaming scheme, I assume we would the usual _x and _y default suffixes that we see in merges ?

[asishm]

This already exists with the verify_integrity param. pd.concat([df1, df2], axis=1, verify_integrity=True) raises with ValueError: Indexes have overlapping values: Index(['a', 'b'], dtype='object')

[kevkle]

But isn't it good if the columns are updated by themselves and not just raise an error

[kevkle]

Or that it should be fixed with another parameter directly ?

[yuanx749]

Seems join is more suitable in this case, or you can DataFrame.add_suffix before concat.

[rhshadrach]

Thanks for the report! As mentioned above there are already suitable arguments / methods if one wants to raise or suffix the columns. I'm negative on adding more complexity to concat as these alternatives already exist. Closing.