Skip to content

Commit 417ddd0

Browse files
Load: fix memory leak when failed to send chunk data in first phase (apache#15518)
This commit addresses a memory leak issue in the load process by ensuring that memory usage is reduced even when the dispatch of a piece node fails. The key changes include: - Storing the result of the dispatch call in a variable (isDispatchSuccess) before reducing memory usage. - Deducting the memory usage prior to checking the dispatch result to avoid leaks. - Returning false immediately after the reduction when the dispatch fails.
1 parent cb19f37 commit 417ddd0

File tree

1 file changed

+9
-5
lines changed

1 file changed

+9
-5
lines changed

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/scheduler/load/LoadTsFileScheduler.java

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -707,12 +707,8 @@ private boolean addOrSendChunkData(ChunkData chunkData) throws LoadFileException
707707
if (pieceNode.getDataSize() == 0) { // total data size has been reduced to 0
708708
break;
709709
}
710-
if (!scheduler.dispatchOnePieceNode(pieceNode, replicaSet)) {
711-
return false;
712-
}
710+
final boolean isDispatchSuccess = scheduler.dispatchOnePieceNode(pieceNode, replicaSet);
713711

714-
dataSize -= pieceNode.getDataSize();
715-
block.reduceMemoryUsage(pieceNode.getDataSize());
716712
regionId2ReplicaSetAndNode.replace(
717713
sortedRegionId,
718714
new Pair<>(
@@ -722,6 +718,14 @@ private boolean addOrSendChunkData(ChunkData chunkData) throws LoadFileException
722718
singleTsFileNode
723719
.getTsFileResource()
724720
.getTsFile()))); // can not just remove, because of deletion
721+
dataSize -= pieceNode.getDataSize();
722+
block.reduceMemoryUsage(pieceNode.getDataSize());
723+
724+
if (!isDispatchSuccess) {
725+
// Currently there is no retry, so return directly
726+
return false;
727+
}
728+
725729
if (isMemoryEnough()) {
726730
break;
727731
}

0 commit comments

Comments
 (0)