fix: Fix memory leak caused by asynchronous prefetch #14722
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -446,7 +446,7 @@ class CoalescedLoad { | |
| return state_; | ||
| } | ||
|
|
||
| virtual void cancel() { | ||
| setEndState(State::kCancelled); | ||
| } | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -101,6 +101,17 @@ class DirectCoalescedLoad : public cache::CoalescedLoad { | |
| return size; | ||
| } | ||
|
|
||
| void cancel() override { | ||
| folly::SemiFuture<bool> waitFuture(false); | ||
| if (state() == State::kLoading && !loadOrFuture(&waitFuture)) { | ||
| waitFuture.wait(); | ||
|
Contributor
There was a problem hiding this comment. such wait can easily lead to deadlock in Prestissimo use case @kewang1024 @Yuhta There was a problem hiding this comment. Can you share more details? Let me check if Gluten has the issue or not.
Contributor
|
||
| } | ||
| for (auto& request : requests_) { | ||
| pool_->freeNonContiguous(request.data); | ||
| } | ||
| CoalescedLoad::cancel(); | ||
|
Contributor
There was a problem hiding this comment. Could you help confirm if it is better to call cancel at first?
Author
There was a problem hiding this comment. It is incorrect to call the cancel function first, because doing so will immediately set state_ to cancelled, and as a result, the loadOrFuture function will not perform any actual waiting. |
||
| } | ||
|
|
||
| private: | ||
| const std::shared_ptr<IoStatistics> ioStats_; | ||
| const std::shared_ptr<filesystems::File::IoStats> fsStats_; | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
cancel()function is invoked by~DirectBufferedInput(), which ultimately depends on the destructor of TableScan, as TableScan owns thestd::unique_ptr<connector::DataSource> dataSource_. This dependency relationship is illustrated in this discussion. As a result, TableScan must be destroyed before the memory manager, but the current approach in this PR does not explicitly guarantee that order.To ensure that the memory held by the load is always properly released, PR #8205 triggers close() through the TableScan::close() call chain to address the similar issue.
Would you like to share your opinions on the fixes? cc: @xiaoxmeng @FelixYBW Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The solution is either free the resources manually from top to bottom or free it in object destruction. Either way works.