Skip to content

Conversation

@mbutrovich
Copy link
Contributor

@mbutrovich mbutrovich commented Dec 18, 2025

Which issue does this PR close?

N/A.

Rationale for this change

Performance profiling reveals significant execution time spent in tokio::runtime::Runtime::enter() overhead. The polling loop in executePlan was entering and exiting the Tokio runtime on every iteration, causing significant performance degradation.

What changes are included in this PR?

Refactored the polling loop in Java_org_apache_comet_Native_executePlan to enter the Tokio runtime once for the entire loop instead of on each iteration:

  • Moved get_runtime().block_on() to wrap the entire loop instead of calling it inside the loop
  • Used tokio::task::block_in_place() around JNI calls in the Poll::Pending branch to ensure thread safety (JNI env is thread-local and cannot safely migrate between Tokio worker threads)
  • Added TODO comment documenting potential future optimization if JNI thread safety can be guaranteed without block_in_place

How are these changes tested?

Existing tests.

Copy link
Contributor

@parthchandra parthchandra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. pending ci.

@mbutrovich mbutrovich changed the title perf: refactor executePlan to try to avoid constantly entering tokio runtime. perf: refactor executePlan to try to avoid constantly entering Tokio runtime Dec 18, 2025
@codecov-commenter
Copy link

codecov-commenter commented Dec 18, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.61%. Comparing base (f09f8af) to head (8b83a08).
⚠️ Report is 777 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2938      +/-   ##
============================================
+ Coverage     56.12%   59.61%   +3.48%     
- Complexity      976     1379     +403     
============================================
  Files           119      167      +48     
  Lines         11743    15430    +3687     
  Branches       2251     2550     +299     
============================================
+ Hits           6591     9198    +2607     
- Misses         4012     4946     +934     
- Partials       1140     1286     +146     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@andygrove andygrove merged commit 618a440 into apache:main Dec 19, 2025
168 of 169 checks passed
@mbutrovich mbutrovich deleted the poll_refactor branch December 19, 2025 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants