[ML] Refactor inference request executor to leverage scheduled execution #126858
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jonathan-buttner
added
>bug
:ml
|
Hi @jonathan-buttner, I've created a changelog YAML for you. |
…r/elasticsearch into ml-refactor-request-exec
| assertTrue(service.isTerminated()); | ||
| } | ||
|
|
||
| public void testSleep_ThrowingInterruptedException_TerminatesService() throws Exception { |
There was a problem hiding this comment.
We're no longer using a "sleeper" so we don't need this test anymore.
| while (isShutdown() == false) { | ||
| handleTasks(); | ||
| } | ||
| } catch (InterruptedException e) { |
There was a problem hiding this comment.
Since we're not sleeping and we're not using a long lived thread we don't need to catch the interrupted exception.
|
Pinging @elastic/ml-core (Team:ML) |
dan-rubinstein
approved these changes
Apr 16, 2025
jonathan-buttner
added a commit
to jonathan-buttner/elasticsearch
that referenced
this pull request
Apr 16, 2025
…ion (elastic#126858) * Using threadpool schedule and fixing tests * Update docs/changelog/126858.yaml * Clean up * change log
💔 Backport failed
You can use sqren/backport to manually backport by running |
jonathan-buttner
added a commit
to jonathan-buttner/elasticsearch
that referenced
this pull request
Apr 16, 2025
…ion (elastic#126858) * Using threadpool schedule and fixing tests * Update docs/changelog/126858.yaml * Clean up * change log (cherry picked from commit 7a0f63c) # Conflicts: # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/HttpRequestSenderTests.java
jonathan-buttner
added a commit
to jonathan-buttner/elasticsearch
that referenced
this pull request
Apr 16, 2025
…ion (elastic#126858) * Using threadpool schedule and fixing tests * Update docs/changelog/126858.yaml * Clean up * change log (cherry picked from commit 7a0f63c) # Conflicts: # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/HttpRequestSenderTests.java
jonathan-buttner
added a commit
to jonathan-buttner/elasticsearch
that referenced
this pull request
Apr 16, 2025
…ion (elastic#126858) * Using threadpool schedule and fixing tests * Update docs/changelog/126858.yaml * Clean up * change log (cherry picked from commit 7a0f63c) # Conflicts: # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/HttpRequestSenderTests.java
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
elasticsearchmachine
pushed a commit
that referenced
this pull request
Apr 16, 2025
…ion (#126858) (#126948) * Using threadpool schedule and fixing tests * Update docs/changelog/126858.yaml * Clean up * change log (cherry picked from commit 7a0f63c) # Conflicts: # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/HttpRequestSenderTests.java
elasticsearchmachine
pushed a commit
that referenced
this pull request
Apr 16, 2025
…ion (#126858) (#126950) * Using threadpool schedule and fixing tests * Update docs/changelog/126858.yaml * Clean up * change log (cherry picked from commit 7a0f63c) # Conflicts: # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/HttpRequestSenderTests.java
elasticsearchmachine
pushed a commit
that referenced
this pull request
Apr 17, 2025
…ion (#126858) (#126949) * Using threadpool schedule and fixing tests * Update docs/changelog/126858.yaml * Clean up * change log (cherry picked from commit 7a0f63c) # Conflicts: # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/HttpRequestSenderTests.java
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #126853
This PR refactors the
RequestExecutorServiceto useThreadPool.scheduleinstead of having a long lived thread that sleeps.Testing
The
inference_utilitythread should no longer report an always active thread:Retrieving hot threads shouldn't show the utility thread all the time now