Skip to content
Closed
Changes from 2 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
3f3186a
Add MeasurementProcessor specification to Metrics SDK
Blinkuu Dec 3, 2024
a8f5de3
Update TODO
Blinkuu Dec 3, 2024
1ce9e4d
Update specification/metrics/sdk.md
Blinkuu Dec 3, 2024
4b0a58d
Update specification/metrics/sdk.md
pellared Dec 3, 2024
449d2fb
Update specification/metrics/sdk.md
pellared Dec 3, 2024
60adbd3
Remove Shutdown and ForceFlush from MeasurementProcessor spec
Blinkuu Dec 6, 2024
9d60b12
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Dec 6, 2024
17f650c
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Dec 19, 2024
4cbc0ec
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jan 2, 2025
01bcc0a
feat: define built-in measurement processor
Blinkuu Jan 2, 2025
d28e993
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jan 7, 2025
90d331d
chore: update the wording
Blinkuu Jan 7, 2025
ee72e3f
chore: rename simple processor to default processor
Blinkuu Jan 7, 2025
225000a
feat: specify how we reference the next processor in the chain
Blinkuu Jan 7, 2025
8fbc341
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jan 22, 2025
f57fe00
Update specification of the `next` parameter and remove `NoopProcessor`
Blinkuu Jan 22, 2025
66c6926
Update specification/metrics/sdk.md
Blinkuu Jan 23, 2025
e585028
Update CHANGELOG.md
Blinkuu Jan 23, 2025
83a81a5
Update metrics/sdk.md
Blinkuu Jan 23, 2025
dbaffa1
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jan 23, 2025
f4c26c1
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jan 29, 2025
ebcbda8
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Feb 7, 2025
b97a231
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Feb 13, 2025
e6eec36
Ensure that the pipeline ends with DefaultProcessor
Blinkuu Feb 13, 2025
20f2977
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Feb 14, 2025
e4fbe83
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Feb 19, 2025
e706183
Apply suggestions from code review
pellared Feb 19, 2025
3ef9483
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Feb 20, 2025
00ce75f
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Feb 28, 2025
c4c9a65
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Mar 13, 2025
58baa69
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Mar 21, 2025
fc8b59d
Remove redundant CHANGELOG.md entry after conflict resolution
Blinkuu Mar 21, 2025
5c76f60
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Mar 27, 2025
45fc676
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Apr 4, 2025
edad787
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
pellared Apr 12, 2025
e872b5d
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu May 14, 2025
cb769a7
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu May 22, 2025
b4710a3
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu May 30, 2025
7f22b07
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jun 17, 2025
42781a3
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jun 17, 2025
958cb86
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jun 24, 2025
0f5e3cd
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jul 8, 2025
4242cfa
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jul 8, 2025
c8b1e76
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jul 9, 2025
756de83
Update description on DefaultProcessor
Blinkuu Jul 10, 2025
11d423b
Update sdk.md
pellared Jul 10, 2025
9cedc06
Update CHANGELOG.md
pellared Jul 10, 2025
3887c5d
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jul 15, 2025
c1a7627
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jul 19, 2025
94c43a5
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
Blinkuu Jul 21, 2025
d7254ec
Merge branch 'main' into add-measurement-processor-to-metrics-sdk-spec
cijothomas Jul 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 76 additions & 2 deletions specification/metrics/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,11 @@ linkTitle: SDK
* [Instrument advisory parameters](#instrument-advisory-parameters)
* [Instrument enabled](#instrument-enabled)
- [Attribute limits](#attribute-limits)
- [MeasurementProcessor](#measurementprocessor)
* [MeasurementProcessor operations](#measurementprocessor-operations)
+ [OnMeasure](#onmeasure)
+ [Shutdown](#shutdown-1)
+ [ForceFlush](#forceflush-1)
- [Exemplar](#exemplar)
* [ExemplarFilter](#exemplarfilter)
+ [AlwaysOn](#alwayson)
Expand All @@ -64,9 +69,9 @@ linkTitle: SDK
- [MetricReader](#metricreader)
* [MetricReader operations](#metricreader-operations)
+ [Collect](#collect)
+ [Shutdown](#shutdown-1)
+ [Shutdown](#shutdown-2)
* [Periodic exporting MetricReader](#periodic-exporting-metricreader)
+ [ForceFlush](#forceflush-1)
+ [ForceFlush](#forceflush-2)
- [MetricExporter](#metricexporter)
* [Push Metric Exporter](#push-metric-exporter)
+ [Interface Definition](#interface-definition)
Expand Down Expand Up @@ -986,6 +991,75 @@ Attributes which belong to Metrics are exempt from the
time. Attribute truncation or deletion could affect identity of metric time
series and the topic requires further analysis.

## MeasurementProcessor

`MeasurementProcessor` is an interface which allows hooks when a `Measurement` is recorded by an `Instrument`.

`MeasurementProcessors` can be registered directly on SDK `MeterProvider` and they are invoked in the same order as they were registered.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it would be good to specify how these are meant to be registered, similar to what we have for span processors. Can they be provided when the MeterProvider is instantiated (similar to span processors)? Does it need to be possible to dynamically register/unregister them (similar to callbacks)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My suggestion is to use the same wording as we use for Provider configuration in general.
i.e It MUST be possible to add measurement_processor as part of MeterProvider creation.
Implementations MAY allow modifications after MeterProvider is created. If they do, then all existing meter/instruments should reflect the effect of this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the exact same wording we use for LogRecordProcessor and SpanProcessor.

I think it should be consistent, it's the same concept.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should specify how MeterProviders are configured in the MeterProvider spec, not in the MeasurementProcessors spec. See this section of TracerProvider: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#configuration that @cijothomas quoted above. Make sure anything related to MeasurementProcessor is marked as Development status.


SDK MUST allow users to implement and configure custom processors.

The following diagram shows `MeasurementProcessor`'s relationship to other components in the SDK:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include Views in this diagram. Importantly, is View configuration applied before the Measurement processor, or after, or both? We clearly want it to be applied before the aggregation portion of views is applied, but it would be nice if it was applied after the View's attribute filter so processors don't need to consider attributes that aren't going to be kept.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Views are meant to be applied after Measurement Processors. They operate on a higher level of abstraction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized cardinality limits should probably also be included, but we can wait until the discussion below is resolved.


```plaintext
+------------------+
| MeterProvider | +----------------------+ +-----------------+
| Meter A | Measurements... | | Metrics... | |
| Instrument X |-----------------> MeasurementProcessor +------------> In-memory state |
| Instrument Y + | | | |
| Meter B | +----------------------+ +-----------------+
| Instrument Z |
| ... | +----------------------+ +-----------------+
| ... | Measurements... | | Metrics... | |
| ... |-----------------> MeasurementProcessor +------------> In-memory state |
| ... | | | | |
| ... | +----------------------+ +-----------------+
+------------------+
```

### MeasurementProcessor operations

#### OnMeasure

`OnMeasure` is called when a `Measurement` is recorded. This method is called synchronously on the thread that emitted the `Measurement`, therefore it SHOULD NOT block or throw exceptions.

**Parameters:**

* `measurement` - a [Measurement](./api.md#measurement) that was recorded
* `context` - the resolved `Context` (the explicitly passed `Context` or the current `Contex`)

**Returns:** Void

For a `MeasurementProcessor` registered directly on SDK `MeterProvider`, the `measurement` mutations MUST be visible in next registered processors.

A `MeasuremenetProcessor` may freely modify `measurement` for the duration of the `OnMeasure` call.

#### Shutdown

Shuts down the processor. Called when the SDK is shut down. This is an opportunity for the processor to do any cleanup required.

`Shutdown` SHOULD be called only once for each`MeasurementProcessor` instance. After the call to `Shutdow`, subsequent calls to `OnMeasure` are not allowed. SDKs SHOULD ignore these calls gracefully, if possible.

`Shutdown` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out.

`Shutdown` MUST include the effects of `ForceFlush`.

`Shutdown` SHOULD complete or abort within some timeout. `Shutdown` can be implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry SDK authors can decide if they want to make the shutdown timeout configurable.

#### ForceFlush

This is a hint to ensure that any tasks associated with `Measurements` for which the `MeasurementProcessor` had already received events prior to the call to `ForceFlush` SHOULD be completed as soon as possible, preferably before returning from this method.

<!-- TODO: Should we mingle with the Exporter concept here? For metrics, the only thing we care is that Measuremenets are processed before aggregation happens -->

In particular, if any `MeasurementProcessor` has any associated exporter, it SHOULD try to call the exporter's `Export` with all `Measurements` for which this was not already done and then invoke `ForceFlush` on it. If a timeout is specified (see below), the `MeasurementProcessor` MUST prioritize honoring the timeout over finishing all calls. It MAY skip or abort some or all `Export` or `ForceFlush` calls it has made to achieve this goal.

`ForceFlush` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out.

`ForceFlush` SHOULD only be called in cases where it is absolutely necessary, such as when using some FaaS providers that may suspend the process after an invocation, but before the `MeasurementProcessor` exports the emitted `Measuremenets`.

`ForceFlush` SHOULD complete or abort within some timeout. `ForceFlush` can be implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry SDK authors can decide if they want to make the flush timeout configurable.

## Exemplar

**Status**: [Stable](../document-status.md)
Expand Down