BUG: df.where(ea) coerces to object, rather than nullable dtype like ser.where(ea) does

[MarcoGorelli]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

# DataFrame case
df = pd.Series([1, 2, 3], dtype='int64').to_frame()
print(df.where(df>1, pd.Series([pd.NA, pd.NA, pd.NA], dtype='Int64'), axis=0).dtypes)

# Series case
ser = pd.Series([1, 2, 3], dtype='int64')
print(ser.where(ser>1, pd.Series([pd.NA, pd.NA, pd.NA], dtype='Int64'), axis=0).dtype)

Issue Description

Outputs are:

DataFrame case

object

Series case:

Int64

NOTE: this was noted as 'todo' in https://github.com/pandas-dev/pandas/pull/38742/files, but I don't think there exists an issue about it

Expected Behavior

In the DataFrame case, to get

Int64

Installed Versions

INSTALLED VERSIONS

commit : f2a91a0
python : 3.8.13.final.0
python-bits : 64
OS : Linux
OS-release : 5.10.102.1-microsoft-standard-WSL2
Version : #1 SMP Wed Mar 2 00:30:59 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 1.6.0.dev0+315.gf2a91a0ed8.dirty
numpy : 1.23.3
pytz : 2022.2.1
dateutil : 2.8.2
setuptools : 59.8.0
pip : 22.2.2
Cython : 0.29.32
pytest : 7.1.3
hypothesis : 6.54.6
sphinx : 4.5.0
blosc : None
feather : None
xlsxwriter : 3.0.3
lxml.etree : 4.9.1
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : 2.9.3
jinja2 : 3.0.3
IPython : 8.5.0
pandas_datareader: 0.10.0
bs4 : 4.11.1
bottleneck : 1.3.5
brotli :
fastparquet : 0.8.3
fsspec : 2021.11.0
gcsfs : 2021.11.0
matplotlib : 3.6.0
numba : 0.56.2
numexpr : 2.8.3
odfpy : None
openpyxl : 3.0.10
pandas_gbq : 0.17.8
pyarrow : 9.0.0
pyreadstat : 1.1.9
pyxlsb : 1.0.9
s3fs : 2021.11.0
scipy : 1.9.1
snappy :
sqlalchemy : 1.4.41
tables : 3.7.0
tabulate : 0.8.10
xarray : 2022.9.0
xlrd : 2.0.1
xlwt : 1.3.0
zstandard : 0.18.0
tzdata : None
None

[MarcoGorelli]

Even if the dtype could be preserved in other, this issue would be blocked by #49056

[jbrockmendel]

The DataFrame case now gives Int64 dtype. Could use a test (first check to see if one already exists).

[vijmeister]

I see a test here: https://github.com/pandas-dev/pandas/blob/main/pandas/tests/frame/indexing/test_where.py

def test_where_producing_ea_cond_for_np_dtype():
    # GH#44014
    df = DataFrame({"a": Series([1, pd.NA, 2], dtype="Int64"), "b": [1, 2, 3]})
    result = df.where(lambda x: x.apply(lambda y: y > 1, axis=1))
    expected = DataFrame(
        {"a": Series([pd.NA, pd.NA, 2], dtype="Int64"), "b": [np.nan, 2, 3]}
    )
    tm.assert_frame_equal(result, expected) 

[Jay-Patankar]

@MarcoGorelli is this issue still open, would like to work on this

[jbrockmendel]

test_where_producing_ea_cond_for_np_dtype is similar to the OP example but not quite the same. The df starts as Int64 whereas the OP starts with lower-case int64.

@Jay-Patankar go for it

[nejail]

@MarcoGorelli @jbrockmendel Is this issue still isn't solved? if so is it alright I can work on it?