Adding Subscription Metrics to Metric Recorder and Onboarding M365 to Auth Metrics from Metrics Recorder #6363

chrisale000 · 2025-12-20T00:13:30Z

Description

This PR introduces comprehensive metrics tracking for Microsoft Office 365 API operations in the Data Prepper M365 source plugin by extending the VendorAPIMetricsRecorder to support both start subscription APIs and authentication APIs, maintaining centralized vendor API metrics while adding M365-specific operation tracking.

Changes

New Metrics in VendorAPIMetricsRecorder

Start Subscription Metrics: Extended VendorAPIMetricsRecorder to support Office 365 start subscription operations
- Tracks start subscription operations (success/failure/latency/call counts)
- Provides multiple overloaded methods for latency recording (Supplier, Runnable, Duration)
- Maintains centralized vendor API metrics while supporting M365-specific operations

Modified Components

Office365RestClient: Updated to use VendorAPIMetricsRecorder for subscription metrics
- Integrated VendorAPIMetricsRecorder for start subscription operations
- Wrapped start subscription logic with latency tracking
- Removed generic apiCallsCounter in favor of specific start subscription metrics
Office365Configuration (formerly Office365RestClientConfiguration): Updated dependency injection
- Updated Office365RestClient bean to inject VendorAPIMetricsRecorder for subscription metrics
- Added VendorAPIMetricsRecorder bean injection for Office365AuthenticationProvider
- Renamed from Office365RestClientConfiguration to reflect broader configuration scope

Test Coverage

VendorAPIMetricsRecorderTest: Updated/added unit tests
- Tests subscription metric recording methods
- Validates success/failure tracking for start subscription operations
- Tests latency recording with multiple overloads
- Includes edge cases and error scenarios
Office365RestClientTest: Updated integration tests
- Added tests for metrics recording during start subscription operations
- Validates failure metric recording
- Tests AF20024 error handling with metrics
Office365AuthenticationProviderTest: Updated unit tests
- Added tests for VendorAPIMetricsRecorder injection
- Validates authentication metrics recording
- Tests success and failure scenarios for authentication metrics

Metrics Added

Start Subscription Metrics(via VendorAPIMetricsRecorder):
- startSubscriptionRequestsSuccess: Count of successful start subscription operations
- startSubscriptionRequestsFailed: Count of failed start subscription operations
- startSubscriptionRequestLatency: Timer for start subscription operation duration
- startSubscriptionApiCalls: Count of individual API calls during start subscription operations
Authentication Metrics (via VendorAPIMetricsRecorder):
- authenticationRequestsSuccess: Count of successful authentication requests
- authenticationRequestsFailed: Count of failed authentication requests
- authenticationRequestLatency: Timer for authentication request duration

Testing:

Spun up local M365 pipeline and confirmed that all metrics (subscription and authentication) were generated successfully.

Check List

New functionality includes testing.
New functionality has a documentation issue. Please link to it in this PR.
- New functionality has javadoc added
Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

vecheka · 2025-12-22T18:21:33Z

.../java/org/opensearch/dataprepper/plugins/source/microsoft_office365/Office365RestClient.java

-            // Start subscriptions for each content type
-            headers.setContentLength(0);
+                // Start subscriptions for each content type
+                headers.setContentLength(0);


nit: indentation.

The indentation was previously off, this fixes the incorrect indentation.

I think in the future you can run this command to fix indentation

./gradlew :data-prepper-plugins:saas-source-plugins:microsoft-office365-source:spotlessApply

Awesome, thank you for this command. Just ran it and looks like no changes were made so the new indentation is correct!

wjyao0316 · 2025-12-22T18:55:06Z

...nsearch/dataprepper/plugins/source/microsoft_office365/metrics/Office365MetricsRecorder.java

+ * NOTE: Any new Office 365 specific metrics should be implemented here to maintain proper
+ * separation of vendor-specific functionality and keep Office 365 metrics centralized.
+ */
+public class Office365MetricsRecorder {


It would be hard to maintain metric allowlist if we start to introduce per service metric.

We should try to make this metric generic so can be extended for similar cases in the future.

I would suggest add StartSubscription recorder to the vendorAPIMetric instead of having its own one.

graytaylor0 · 2026-01-06T22:25:25Z

.../java/org/opensearch/dataprepper/plugins/source/microsoft_office365/Office365RestClient.java

-            // Start subscriptions for each content type
-            headers.setContentLength(0);
+                // Start subscriptions for each content type
+                headers.setContentLength(0);


I think in the future you can run this command to fix indentation

./gradlew :data-prepper-plugins:saas-source-plugins:microsoft-office365-source:spotlessApply

… Auth Metrics from Metrics Recorder Signed-off-by: Alexander Christensen <[email protected]>

… Auth Metrics from Metrics Recorder (opensearch-project#6363) Signed-off-by: Alexander Christensen <[email protected]> Signed-off-by: Nathan Wand <[email protected]>

chrisale000 requested review from KarstenSchnitter, chenqi0805, dinujoh, dlvenable, engechas, graytaylor0, kkondaka, oeyh, san81, sb2k16 and srikanthjg as code owners December 20, 2025 00:13

chrisale000 force-pushed the m365-metrics branch 2 times, most recently from aabd6e8 to 8af181c Compare December 22, 2025 02:42

chrisale000 changed the title ~~Adding M365 Specific Metrics Recorder~~ Adding M365 Specific Metrics Recorder and Emitting Auth Metrics Dec 22, 2025

chrisale000 force-pushed the m365-metrics branch 2 times, most recently from 86ec81f to b6c0d6e Compare December 22, 2025 03:01

vecheka reviewed Dec 22, 2025

View reviewed changes

wjyao0316 reviewed Dec 22, 2025

View reviewed changes

chrisale000 force-pushed the m365-metrics branch 4 times, most recently from 1e0c7ba to c2324ae Compare January 6, 2026 18:21

chrisale000 changed the title ~~Adding M365 Specific Metrics Recorder and Emitting Auth Metrics~~ Adding Subscription Metrics to Metric Recorder and Onboarding M365 to Auth Metrics from Metrics Recorder Jan 6, 2026

graytaylor0 approved these changes Jan 6, 2026

View reviewed changes

wandna-amazon approved these changes Jan 6, 2026

View reviewed changes

chrisale000 force-pushed the m365-metrics branch from c2324ae to d9ef33f Compare January 7, 2026 17:53

san81 approved these changes Jan 7, 2026

View reviewed changes

chrisale000 force-pushed the m365-metrics branch from d9ef33f to 5527039 Compare January 7, 2026 18:27

Adding Subscription Metrics to Metric Recorder and Onboarding M365 to…

ad4bb22

… Auth Metrics from Metrics Recorder Signed-off-by: Alexander Christensen <[email protected]>

chrisale000 force-pushed the m365-metrics branch from 5527039 to ad4bb22 Compare January 7, 2026 21:53

san81 merged commit 02e9ef1 into opensearch-project:main Jan 7, 2026
48 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Adding Subscription Metrics to Metric Recorder and Onboarding M365 to Auth Metrics from Metrics Recorder #6363

Adding Subscription Metrics to Metric Recorder and Onboarding M365 to Auth Metrics from Metrics Recorder #6363

Uh oh!

chrisale000 commented Dec 20, 2025 •

edited

Loading

Uh oh!

vecheka Dec 22, 2025

Uh oh!

chrisale000 Jan 6, 2026

Uh oh!

graytaylor0 Jan 6, 2026

Uh oh!

chrisale000 Jan 6, 2026

Uh oh!

wjyao0316 Dec 22, 2025

Uh oh!

graytaylor0 Jan 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Adding Subscription Metrics to Metric Recorder and Onboarding M365 to Auth Metrics from Metrics Recorder #6363

Adding Subscription Metrics to Metric Recorder and Onboarding M365 to Auth Metrics from Metrics Recorder #6363

Uh oh!

Conversation

chrisale000 commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

New Metrics in VendorAPIMetricsRecorder

Modified Components

Test Coverage

Metrics Added

Testing:

Check List

Uh oh!

vecheka Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

chrisale000 Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

graytaylor0 Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

chrisale000 Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

wjyao0316 Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

graytaylor0 Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

chrisale000 commented Dec 20, 2025 •

edited

Loading