[SPARK-55296][PS] Support CoW mode with pandas 3

[ueshin]

What changes were proposed in this pull request?

Support CoW (Copy-on-Write) mode with pandas 3.

Why are the changes needed?

Pandas 3 is doing copy-on-write for everything.

For example:

>>> pdf = pd.DataFrame(
...     [[1, 2], [4, 5], [7, 8]],
...     index=["cobra", "viper", "sidewinder"],
...     columns=["max_speed", "shield"],
... )
>>>
>>> pser1 = pdf.max_speed
>>> pser2 = pdf.shield
>>>
>>> pdf.loc[["viper", "sidewinder"], ["max_speed", "shield"]] = 10

• pandas 2

>>> pdf
            max_speed  shield
cobra               1       2
viper              10      10
sidewinder         10      10
>>> pser1
cobra          1
viper         10
sidewinder    10
Name: max_speed, dtype: int64
>>> pser2
cobra          2
viper         10
sidewinder    10
Name: shield, dtype: int64

• pandas 3

>>> pdf
            max_speed  shield
cobra               1       2
viper              10      10
sidewinder         10      10
>>> pser1
cobra         1
viper         4
sidewinder    7
Name: max_speed, dtype: int64
>>> pser2
cobra         2
viper         5
sidewinder    8
Name: shield, dtype: int64

Or for Series:

>>> pdf = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]}, index=["cobra", "viper", "sidewinder"])
>>>
>>> pser = pdf.x
>>> psery = pdf.y
>>>
>>> pser.loc[pser % 2 == 1] = -pser

• pandas 2

>>> pdf
            x  y
cobra      -1  4
viper       2  5
sidewinder -3  6
>>> pser
cobra        -1
viper         2
sidewinder   -3
Name: x, dtype: int64
>>> psery
cobra         4
viper         5
sidewinder    6
Name: y, dtype: int64

• pandas 3

>>> pdf
            x  y
cobra       1  4
viper       2  5
sidewinder  3  6
>>> pser
cobra        -1
viper         2
sidewinder   -3
Name: x, dtype: int64
>>> psery
cobra         4
viper         5
sidewinder    6
Name: y, dtype: int64

Does this PR introduce any user-facing change?

Yes, it will behave more like pandas 3.

How was this patch tested?

Updated the related tests to make it clear, but basically the existing tests should pass.

Was this patch authored or co-authored using generative AI tooling?

Codex (GPT-5.3-Codex)

[ueshin]

cc @gaogaotiantian @HyukjinKwon @zhengruifeng

[zhengruifeng]

Awesome

[gaogaotiantian]

I didn't realize that this fix could be so clean. Great work! I guess this fix by itself should clear up many test failures.

[ueshin]

Thanks! merging to master.