Skip to content

Conversation

@BrianRothermich
Copy link
Contributor

@BrianRothermich BrianRothermich commented May 29, 2025

Relates to an effort to combine the merge schedulers from stateless and stateful. The stateless merge scheduler has MergeMetrics that we want in both stateless and stateful. This PR copies over the merge metrics from the stateless merge scheduler into the combined merge scheduler.

Relates ES-9687

@elasticsearchmachine elasticsearchmachine added v9.1.0 serverless-linked Added by automation, don't add manually labels May 29, 2025
@BrianRothermich BrianRothermich force-pushed the brothermich/ES-9687--metrics branch 2 times, most recently from 61f0a99 to 79917cd Compare May 31, 2025 14:36
@BrianRothermich BrianRothermich marked this pull request as ready for review June 9, 2025 23:21
@BrianRothermich BrianRothermich requested a review from a team as a code owner June 9, 2025 23:21
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Jun 9, 2025
@BrianRothermich BrianRothermich added >non-issue :Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. and removed needs:triage Requires assignment of a team area label labels Jun 9, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed-indexing (Team:Distributed Indexing)

@elasticsearchmachine elasticsearchmachine added the Team:Distributed Indexing Meta label for Distributed Indexing team label Jun 9, 2025
@BrianRothermich BrianRothermich requested review from albertzaharovits and removed request for a team June 9, 2025 23:23
Copy link
Contributor

@albertzaharovits albertzaharovits left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Left 2 small comments.

throw new IllegalStateException("The merge task is already started or aborted");
}
mergeTracking.mergeStarted(onGoingMerge);
mergeMetrics.moveQueuedMergeBytesToRunning(onGoingMerge, mergeMemoryEstimateBytes);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should also be invoked on the "abort" merge code path.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Updated

threadPoolMergeExecutorService,
merge -> 0
merge -> 0,
MergeMetrics.NOOP
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please make these tests (in this class) take a mock(MergeMetrics.class) and assert the invocations on it (e.g. when merge tasks are run/aborted), in order to have some test coverage for the metrics code too.

Copy link
Contributor Author

@BrianRothermich BrianRothermich Jun 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated some tests to verify the metrics reporting in 629d70a

@albertzaharovits albertzaharovits self-requested a review June 19, 2025 13:57
Copy link
Contributor

@albertzaharovits albertzaharovits left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, good job!

@BrianRothermich BrianRothermich merged commit 0f39ff5 into elastic:main Jun 23, 2025
32 checks passed
@BrianRothermich BrianRothermich deleted the brothermich/ES-9687--metrics branch June 23, 2025 23:42
mridula-s109 pushed a commit to mridula-s109/elasticsearch that referenced this pull request Jun 25, 2025
Relates to an effort to combine the merge schedulers from stateless and stateful. The stateless merge scheduler has MergeMetrics that we want in both stateless and stateful. This PR copies over the merge metrics from the stateless merge scheduler into the combined merge scheduler.

Relates ES-9687
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. >non-issue serverless-linked Added by automation, don't add manually Team:Distributed Indexing Meta label for Distributed Indexing team v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants