Commit d8a7a5e
fix(hdfs): Nullify hdfsFile_ before checking success status (facebookincubator#14666)
Summary:
driver_->CloseFile can fail if the task is interrupted. In that case, hdfsFile_ is not reset to nullptr. Later, when the HdfsWriteFile destructor runs, it invokes close() again, resulting in a double close. This leads to a fatal JNI error:
```java
25/08/25 08:12:06 INFO [dispatcher-Executor] Executor: Executor is trying to kill task 43.2 in stage 11.0 (TID 617), reason: another attempt succeeded
HdfsWriteFile.cpp:57, Function:close, Expression: success == 0 (-1 vs. 0) Failed to close hdfs file: IOException: Failed to shutdown streamer, Source: RUNTIME, ErrorCode: INVALID_STATE
```
Eventually, the JVM crashes with a segmentation fault during a JNI call to FSDataOutputStream.close():
```java
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fa2ec7e0c48, pid=174755, tid=0x00007fa220262700
#
# JRE version: OpenJDK Runtime Environment (8.0_252-b09) (build 1.8.0_252-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.252-b09 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0x68fc48] jni_invoke_nonstatic(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thread*)+0x48
#
```
Stack trace excerpt showing the duplicate close() call path:
```c++
facebookincubator#8 invokeMethodOnJclass(..., "org/apache/hadoop/fs/FSDataOutputStream", "close", "()V")
facebookincubator#10 hdfsCloseFile(...)
facebookincubator#11 facebook::velox::HdfsWriteFile::close()
facebookincubator#12 facebook::velox::HdfsWriteFile::~HdfsWriteFile()
facebookincubator#14 facebook::velox::dwio::common::WriteFileSink::~WriteFileSink()
facebookincubator#15 facebook::velox::parquet::Writer::abort()
```
Pull Request resolved: facebookincubator#14666
Reviewed By: gggrace14
Differential Revision: D82880508
Pulled By: xiaoxmeng
fbshipit-source-id: c49467744b7784c3373faf61e8a46058f5243caf1 parent a029fcf commit d8a7a5e
File tree
2 files changed
+35
-1
lines changed- velox/connectors/hive/storage_adapters/hdfs
- tests
2 files changed
+35
-1
lines changedLines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
57 | 60 | | |
58 | 61 | | |
59 | 62 | | |
60 | 63 | | |
61 | 64 | | |
62 | | - | |
63 | 65 | | |
64 | 66 | | |
65 | 67 | | |
| |||
Lines changed: 32 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
525 | 526 | | |
526 | 527 | | |
527 | 528 | | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
0 commit comments