BUG: .replace() won't work on Series if a NaN is found on new string dtype

[ErichMarx]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

# Use new string type string[pyarrow_numpy]
pd.options.future.infer_string = True

df = pd.Series(["VALUE1", None, "VALUE3"], name="COLUMN1")
replace_dict = {"VALUE1": "REPLACED1", "VALUE3": "REPLACED3"}
df = df.replace(replace_dict)

Issue Description

The replace method is expected to replace all values in the df Series that match the keys in replace_dict with their corresponding values. However, the replacement stops once a NaN ("string", pd.options.future.infer_string = True) value is encountered, and the subsequent values are not replaced.

If the column is of type object, that works fine. (Check expected behaviour code)

Expected Behavior

Value VALUE3 should be replaced by REPLACED3.

import pandas as pd

# String dtype will be object
pd.options.future.infer_string = False

df = pd.Series(["VALUE1", None, "VALUE3"], name="COLUMN1")
replace_dict = {"VALUE1": "REPLACED1", "VALUE3": "REPLACED3"}
df = df.replace(replace_dict)

Installed Versions

INSTALLED VERSIONS

commit : 0691c5c
python : 3.12.6
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : AMD64 Family 25 Model 80 Stepping 0, AuthenticAMD
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252

pandas : 2.2.3
numpy : 2.0.2
pytz : 2024.2
dateutil : 2.9.0.post0
pip : None
Cython : None
sphinx : None
IPython : 8.28.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
blosc : None
bottleneck : 1.4.2
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : None
lxml.etree : None
matplotlib : None
numba : 0.60.0
numexpr : 2.10.1
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : None
pymysql : None
pyarrow : 17.0.0
pyreadstat : 1.2.7
pytest : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : 2.0.36
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2024.2
qtpy : None
pyqt5 : None

[ErichMarx]

Hi @lfffkh,

I just tried 2.2.2 and 2.2.3, both still with the same output. (reinstalling on a fresh venv)
I did notice your dtype is str, and not string like mine.
Can you confirm if your series dtype is also string[pyarrow_numpy]?

[lfffkh]

Hi, I tried to reproduce the code you provided, but in my output, “VALUE3” was correctly replaced with "REPLACED3".

[lfffkh]

Hi @lfffkh,

I just tried 2.2.2 and 2.2.3, both still with the same output. (reinstalling on a fresh venv) I did notice your dtype is str, and not string like mine. Can you confirm if your series dtype is also string[pyarrow_numpy]?

Sorry, after reviewing the code, I see that the dtype is actually str.

[asishm]

Thanks for the report. @ErichMarx you have the third checkbox checked indicating, that you've checked on the main branch. Can you please confirm because I can't reproduce the error.

[ErichMarx]

Hi @asishm,
That indeed does not happen on the main branch. I had mistakenly checked that box.
The output I get on the main branch pandas is the same as @lfffkh showed above.
Thanks for checking this out, I will close the issue.