Skip to content

Conversation

@wankai123
Copy link
Member

@wankai123 wankai123 commented Aug 14, 2025

  • If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
  • Update the CHANGES log.

Optimize metrics aggregate/persistent worker: seperate OAL and MAL workers and consume pools, and make the consumer thread sleep when no data in.

…` workers and consume pools, and make the consumer thread sleep when no data in.
@wankai123 wankai123 added this to the 10.3.0 milestone Aug 14, 2025
@wankai123 wankai123 requested review from Copilot and wu-sheng August 14, 2025 00:58
@wankai123 wankai123 added backend OAP backend related. enhancement Enhancement on performance or codes labels Aug 14, 2025
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes the metrics aggregation and persistence processing by separating OAL (Observability Analysis Language) and MAL (Meter Analysis Language) workers into dedicated thread pools with different configurations. The key optimization introduces a notification-based consumption pattern for MAL workers to reduce unnecessary resource usage when no data is available.

  • Separates OAL and MAL workers into dedicated worker classes with optimized configurations
  • Introduces notification-based consumer thread sleeping for MAL workers to reduce CPU usage
  • Updates the data carrier consumer pool to support notifiable consumption patterns

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
ConsumerPoolFactoryTest.java Updates test to include new notifiable parameter in BulkConsumePool.Creator
MultipleChannelsConsumer.java Adds notification-based consumption logic with sleep when no data
BulkConsumePool.java Adds notifiable parameter and consumer notification capability
MetricsStreamProcessor.java Implements factory pattern to create OAL/MAL specific workers
MetricsPersistentMinWorker.java Converts to abstract base class with configurable pool parameters
MetricsPersistentMinOALWorker.java OAL-specific persistent worker with optimized settings
MetricsPersistentMinMALWorker.java MAL-specific persistent worker with notification support
MetricsAggregateWorker.java Converts to abstract base class with configurable pool parameters
MetricsAggregateOALWorker.java OAL-specific aggregate worker with high-throughput settings
MetricsAggregateMALWorker.java MAL-specific aggregate worker with notification support
changes.md Documents the optimization changes

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

wu-sheng
wu-sheng previously approved these changes Aug 14, 2025
@wankai123 wankai123 merged commit fe26250 into apache:master Aug 14, 2025
343 of 345 checks passed
@wankai123 wankai123 deleted the queue-pool branch August 14, 2025 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend OAP backend related. enhancement Enhancement on performance or codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants