Skip to content

Conversation

@BiteTheDDDDt
Copy link
Contributor

This pull request introduces a delayed deletion mechanism for QueryContext objects in the fragment manager to ensure that runtime filter merging works correctly after a query reaches its end-of-stream (EOS). The main changes involve adding a new map to retain query contexts temporarily, modifying the lifecycle management of query contexts, and introducing utility methods to support this behavior.

Query Context Lifecycle Management:

  • Added a new member _query_ctx_map_delay_delete to FragmentMgr to keep QueryContext objects alive for a short period after query EOS, ensuring that the runtime filter coordinator can complete its work.
  • Modified FragmentMgr::remove_query_context to erase entries from _query_ctx_map_delay_delete as part of the cleanup process.
  • Updated the QueryContext destructor to call remove_query_context only if the fragment manager is available, preventing issues during unit tests.

Runtime Filter Merge Coordination:

  • In FragmentMgr::_get_or_create_query_ctx, now inserts the QueryContext into _query_ctx_map_delay_delete if the runtime filter merge handler is not empty, ensuring the context is retained for merging.
  • Added an empty() method to RuntimeFilterMergeControllerEntity to check if the filter map is empty, which is used to determine if delayed deletion is needed.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

This pull request introduces a delayed deletion mechanism for
`QueryContext` objects in the fragment manager to ensure that runtime
filter merging works correctly after a query reaches its end-of-stream
(EOS). The main changes involve adding a new map to retain query
contexts temporarily, modifying the lifecycle management of query
contexts, and introducing utility methods to support this behavior.

**Query Context Lifecycle Management:**

* Added a new member `_query_ctx_map_delay_delete` to `FragmentMgr` to
keep `QueryContext` objects alive for a short period after query EOS,
ensuring that the runtime filter coordinator can complete its work.
* Modified `FragmentMgr::remove_query_context` to erase entries from
`_query_ctx_map_delay_delete` as part of the cleanup process.
* Updated the `QueryContext` destructor to call `remove_query_context`
only if the fragment manager is available, preventing issues during unit
tests.

**Runtime Filter Merge Coordination:**

* In `FragmentMgr::_get_or_create_query_ctx`, now inserts the
`QueryContext` into `_query_ctx_map_delay_delete` if the runtime filter
merge handler is not empty, ensuring the context is retained for
merging.
* Added an `empty()` method to `RuntimeFilterMergeControllerEntity` to
check if the filter map is empty, which is used to determine if delayed
deletion is needed.
@BiteTheDDDDt BiteTheDDDDt requested a review from yiguolei as a code owner January 7, 2026 09:51
@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@Thearas
Copy link
Contributor

Thearas commented Jan 7, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Now query contexts will not be removed from the FragmentMgr eagerly which may lead to a very large map to hold all query contexts.
@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@BiteTheDDDDt BiteTheDDDDt changed the title [Chore](query) add _query_ctx_map_delay_delete (#59262) [Chore](query) add _query_ctx_map_delay_delete (#59262) (#56911) Jan 8, 2026
@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 22.22% (4/18) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.34% (18701/35063)
Line Coverage 39.07% (172860/442437)
Region Coverage 33.79% (133737/395839)
Branch Coverage 34.76% (57845/166410)

@yiguolei
Copy link
Contributor

skip buildall

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jan 12, 2026
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit aae5957 into apache:branch-4.0 Jan 12, 2026
28 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants