Commit 9875380
committed
[SPARK-51850] Fix
### What changes were proposed in this pull request?
This PR aims to fix `DataFrame.execute` to reset previously received Arrow batch data.
### Why are the changes needed?
To prevent wrong data duplication when repeating execution on the same `DataFrame`.
### Does this PR introduce _any_ user-facing change?
This, this is a bug fix.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #72 from dongjoon-hyun/SPARK-51850.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>DataFrame.execute to reset previously received Arrow batch data1 parent cb08c76 commit 9875380
File tree
2 files changed
+12
-0
lines changed- Sources/SparkConnect
- Tests/SparkConnectTests
2 files changed
+12
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
151 | 151 | | |
152 | 152 | | |
153 | 153 | | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
154 | 157 | | |
155 | 158 | | |
156 | 159 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
311 | 311 | | |
312 | 312 | | |
313 | 313 | | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
314 | 323 | | |
315 | 324 | | |
316 | 325 | | |
| |||
0 commit comments