[ML] Prevent the trained model deployment memory estimation from double-counting allocations. #131990
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 971cfb9.
valeriy42
added
>bug
:ml
|
Pinging @elastic/ml-core (Team:ML) |
|
Hi @valeriy42, I've created a changelog YAML for you. |
This was referenced Jul 28, 2025
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 28, 2025
…le-counting allocations. (elastic#131990) This reverts commit 971cfb9. The refactoring in 971cfb9 introduced a bug that could potentially lead to double-counting of the number of allocations in the trained model memory estimation.
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 28, 2025
…le-counting allocations. (elastic#131990) This reverts commit 971cfb9. The refactoring in 971cfb9 introduced a bug that could potentially lead to double-counting of the number of allocations in the trained model memory estimation.
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 28, 2025
…le-counting allocations. (elastic#131990) This reverts commit 971cfb9. The refactoring in 971cfb9 introduced a bug that could potentially lead to double-counting of the number of allocations in the trained model memory estimation.
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 28, 2025
…le-counting allocations. (elastic#131990) This reverts commit 971cfb9. The refactoring in 971cfb9 introduced a bug that could potentially lead to double-counting of the number of allocations in the trained model memory estimation.
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Jul 28, 2025
…-tracking * upstream/main: Fix MergeWithLowDiskSpaceIT testRelocationWhileForceMerging (elastic#131806) [ML] Prevent the trained model deployment memory estimation from double-counting allocations. (elastic#131990) ES|QL Assert current thread during query planning and execution (elastic#131807) Add ElasticsearchIndexDeletionPolicy and EngineConfig policy wrapper (elastic#130442) [TEST] Adds tests for ESTestCase randomSubset methods (elastic#131745) Simplify esql session (elastic#131925) Simplify EsqlExecution info serialization (elastic#131823) Add utility to check for project global block (elastic#131927) [DOCS] Update ES|QL applies to's (elastic#131805) Handle structured log messages (elastic#131027) Mute org.elasticsearch.test.rest.yaml.RcsCcsCommonYamlTestSuiteIT test {p0=search/600_flattened_ignore_above/flattened ignore_above multi-value field} elastic#131967 Mute org.elasticsearch.xpack.remotecluster.CrossClusterEsqlRCS2EnrichUnavailableRemotesIT testEsqlEnrichWithSkipUnavailable elastic#131965 Mute org.elasticsearch.xpack.restart.FullClusterRestartIT testWatcherWithApiKey {cluster=UPGRADED} elastic#131964 [ES|QL] Fix aggregate_metric_double sorting and mv_expand issues (elastic#131658) Reduce logging levels for meter usage tests (elastic#131935)
afoucret
pushed a commit
to afoucret/elasticsearch
that referenced
this pull request
Jul 28, 2025
…le-counting allocations. (elastic#131990) This reverts commit 971cfb9. The refactoring in 971cfb9 introduced a bug that could potentially lead to double-counting of the number of allocations in the trained model memory estimation.
valeriy42
added a commit
that referenced
this pull request
Jul 31, 2025
A follow-up to #131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in #131990.
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 31, 2025
…ic#132170) A follow-up to elastic#131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in elastic#131990.
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 31, 2025
…ic#132170) A follow-up to elastic#131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in elastic#131990.
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 31, 2025
…ic#132170) A follow-up to elastic#131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in elastic#131990. (cherry picked from commit 80c47f3)
valeriy42
added a commit
to valeriy42/elasticsearch
that referenced
this pull request
Jul 31, 2025
…ic#132170) A follow-up to elastic#131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in elastic#131990. (cherry picked from commit 80c47f3)
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jul 31, 2025
…) (#132271) A follow-up to #131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in #131990.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jul 31, 2025
…) (#132270) A follow-up to #131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in #131990.
smalyshev
pushed a commit
to smalyshev/elasticsearch
that referenced
this pull request
Jul 31, 2025
…ic#132170) A follow-up to elastic#131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in elastic#131990.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Aug 1, 2025
…#132170) (#132275) * [ML] Make AssignmentPlan to consider only assigned allocations (#132170) A follow-up to #131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in #131990. (cherry picked from commit 80c47f3) * Fix backport errors * Fix unit test
elasticsearchmachine
pushed a commit
that referenced
this pull request
Sep 8, 2025
…#132170) (#132274) * [ML] Make AssignmentPlan to consider only assigned allocations (#132170) A follow-up to #131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in #131990. (cherry picked from commit 80c47f3) * Fix backport errors * Fix unit test
sarog
pushed a commit
to portsbuild/elasticsearch
that referenced
this pull request
Sep 11, 2025
…elastic#132170) (elastic#132274) * [ML] Make AssignmentPlan to consider only assigned allocations (elastic#132170) A follow-up to elastic#131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in elastic#131990. (cherry picked from commit 80c47f3) * Fix backport errors * Fix unit test
sarog
pushed a commit
to portsbuild/elasticsearch
that referenced
this pull request
Sep 19, 2025
…elastic#132170) (elastic#132274) * [ML] Make AssignmentPlan to consider only assigned allocations (elastic#132170) A follow-up to elastic#131990. This PR ensures that only assigned allocations and not current allocations are used in the memory requirements calculation in AssignmentPlan. This change led to the simplification of the code in ZoneAwareAssignmentPlanner and TrainedModelRebalancer. This PR also improves readability by adding comments, code documentation, renaming variables, and making the flow of if statements more straightforward. Marking is a non-issue since the bug was already documented in elastic#131990. (cherry picked from commit 80c47f3) * Fix backport errors * Fix unit test
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This reverts commit 971cfb9.
The refactoring in 971cfb9 introduced a bug that could potentially lead to double-counting of the number of allocations in the trained model memory estimation.