Commit fb0c4b8
[SPARK-53506][PS] Disallow
### What changes were proposed in this pull request?
Disallow `%` between Decimal and float under ANSI
pandas:
```py
>>> pdf['decimal'] % 0.1
Traceback (most recent call last):
...
TypeError: unsupported operand type(s) for %: 'decimal.Decimal' and 'float'
```
pandas on spark before:
```py
>>> psdf['decimal'] % 0.1
0 0.1
1 0.1
2 0.1
Name: decimal, dtype: float64
```
pandas on spark after:
```py
>>> psdf['decimal'] % 0.1
Traceback (most recent call last):
...
TypeError: Modulo can not be applied to given types.
```
### Why are the changes needed?
Part of https://issues.apache.org/jira/browse/SPARK-53389
### Does this PR introduce _any_ user-facing change?
No, the feature is not released yet
### How was this patch tested?
Unit tests
Commands below passed:
```py
1097 SPARK_ANSI_SQL_MODE=true ./python/run-tests --python-executables=python3.11 --testnames "pyspark.pandas.tests.data_type_ops.test_num_mul_div NumMulDivTests.test_mod"
1098 SPARK_ANSI_SQL_MODE=false ./python/run-tests --python-executables=python3.11 --testnames "pyspark.pandas.tests.data_type_ops.test_num_mul_div NumMulDivTests.test_mod"
```
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #52255 from xinrong-meng/mod.
Authored-by: Xinrong Meng <xinrong@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>% between Decimal and float under ANSI1 parent b08ac30 commit fb0c4b8
File tree
2 files changed
+43
-0
lines changed- python/pyspark/pandas
- data_type_ops
- tests/data_type_ops
2 files changed
+43
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
469 | 469 | | |
470 | 470 | | |
471 | 471 | | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
472 | 479 | | |
473 | 480 | | |
474 | 481 | | |
| |||
572 | 579 | | |
573 | 580 | | |
574 | 581 | | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
575 | 589 | | |
576 | 590 | | |
577 | 591 | | |
| |||
Lines changed: 29 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
114 | 143 | | |
115 | 144 | | |
116 | 145 | | |
| |||
0 commit comments