Skip to content

Conversation

@drempapis
Copy link
Contributor

In the 122707 test-failure we get the error

Expected: an empty collection
         but: <[LEAK: resource was not cleaned up before it was garbage-collected.
    Recent access records: 
     #1:
        in [elasticsearch[node_s1][search][T#2]][testSearchWithRandomDisconnects {seed=[9C4F49EC3D6BE222:4BFA3E4DF9657D16]}]
        org.elasticsearch.action.ActionListener.respondAndRelease(ActionListener.java:388)
        org.elasticsearch.action.ActionRunnable$3.accept(ActionRunnable.java:79)
        org.elasticsearch.action.ActionRunnable$3.accept(ActionRunnable.java:76)
        org.elasticsearch.action.ActionRunnable$4.doRun(ActionRunnable.java:101)
        org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:34)
        org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1044)
        org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
        java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095)
        java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)
        java.base/java.lang.Thread.run(Thread.java:1447)
    #2:
        in [elasticsearch[node_s1][search][T#2]][testSearchWithRandomDisconnects {seed=[9C4F49EC3D6BE222:4BFA3E4DF9657D16]}]
        org.elasticsearch.action.search.ArraySearchPhaseResults.consumeResult(ArraySearchPhaseResults.java:47)
        org.elasticsearch.action.search.QueryPhaseResultConsumer.consumeResult(QueryPhaseResultConsumer.java:159)
        org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardResult(AbstractSearchAsyncAction.java:503)
        org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.onShardResult(SearchQueryThenFetchAsyncAction.java:180)
        org.elasticsearch.action.search.AbstractSearchAsyncAction$1.innerOnResponse(AbstractSearchAsyncAction.

This draft PR, addresses(?) the resource leak reported by the leak tracker, which indicates that a search-phase-related resource (likely ArraySearchPhaseResults or a similar buffer) is not being properly released before garbage collection.

To mitigate this, the existing listener is wrapped by ActionListener.runAfter, ensuring that queryResultConsumer::close is always invoked once the search operation completes — whether it succeeds or fails:

final ActionListener<SearchResponse> releasingListener = ActionListener.runAfter(listener, queryResultConsumer::close);

This hopefully ensures that the queryResultConsumer is reliably closed after the original listener is notified, preventing the resource leak and aligning with the expected lifecycle management.

This has not been extensively tested yet, but initial results suggest it resolves the issue in the SearchWithRandomDisconnectsIT test case.

cc @benchaplin (you are currently reviewing tests that disable batched execution)

@drempapis drempapis added >test Issues or PRs that are addressing/adding tests Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch :Search Foundations/Search Catch all for Search Foundations v9.2.0 labels Aug 13, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-foundations (Team:Search Foundations)

Copy link
Contributor

@benchaplin benchaplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am struggling to picture what went wrong here, however I ran the test with your fix a few times and still experienced failures. I will think about this one more...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch >test Issues or PRs that are addressing/adding tests v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants