Commit 10c7614
[SPARK-52055][PYTHON][CONNECT] Suppress the exception in ExecutePlanResponseReattachableIterator.__del__
### What changes were proposed in this pull request?
This PR proposes to suppress the exception in `ExecutePlanResponseReattachableIterator.__del__`.
### Why are the changes needed?
It tries its best effort to close it. When it fails, we should just ignore it.
Now when you run `./bin/pyspark --remote local` and `exit(0)` directly without doing anything, it shows a warning as below:
```
Exception ignored in: <function ExecutePlanResponseReattachableIterator.__del__ at 0x12229f7e0>
Traceback (most recent call last):
File "/.../python/pyspark/sql/connect/client/reattach.py", line 347, in __del__
return self.close()
^^^^^^^^^^^^
File "/.../python/pyspark/sql/connect/client/reattach.py", line 343, in close
self._release_all()
File "/.../python/pyspark/sql/connect/client/reattach.py", line 245, in _release_all
thread_pool.submit(target)
File "/opt/miniconda3/envs/python3.11/lib/python3.11/concurrent/futures/thread.py", line 169, in submit
raise RuntimeError('cannot schedule new futures after '
RuntimeError: cannot schedule new futures after interpreter shutdown
```
### Does this PR introduce _any_ user-facing change?
The warning will not be shown.
### How was this patch tested?
Manually via `./bin/pyspark --remote local`.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes apache#50847 from HyukjinKwon/SPARK-52055.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>1 parent 20e94a5 commit 10c7614
1 file changed
+4
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
344 | 344 | | |
345 | 345 | | |
346 | 346 | | |
347 | | - | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
0 commit comments