Skip to content

Commit b8ea807

Browse files
authored
[sdk-metrics] ExemplarReservoir dedicated diagnostic and custom ExemplarReservoir support (#5558)
1 parent 49d70e0 commit b8ea807

File tree

13 files changed

+178
-16
lines changed

13 files changed

+178
-16
lines changed

OpenTelemetry.sln

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -312,6 +312,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "experimental-apis", "experi
312312
docs\diagnostics\experimental-apis\OTEL1001.md = docs\diagnostics\experimental-apis\OTEL1001.md
313313
docs\diagnostics\experimental-apis\OTEL1002.md = docs\diagnostics\experimental-apis\OTEL1002.md
314314
docs\diagnostics\experimental-apis\OTEL1003.md = docs\diagnostics\experimental-apis\OTEL1003.md
315+
docs\diagnostics\experimental-apis\OTEL1004.md = docs\diagnostics\experimental-apis\OTEL1004.md
315316
docs\diagnostics\experimental-apis\README.md = docs\diagnostics\experimental-apis\README.md
316317
EndProjectSection
317318
EndProject

build/Common.props

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
<Nullable>enable</Nullable>
1111
<ImplicitUsings>enable</ImplicitUsings>
1212
<!-- Suppress warnings for repo code using experimental features -->
13-
<NoWarn>$(NoWarn);OTEL1000;OTEL1001;OTEL1002;OTEL1003</NoWarn>
13+
<NoWarn>$(NoWarn);OTEL1000;OTEL1001;OTEL1002;OTEL1003;OTEL1004</NoWarn>
1414
<!--temporarily disable. See 3958-->
1515
<!--<AnalysisLevel>latest-All</AnalysisLevel>-->
1616
</PropertyGroup>

docs/diagnostics/experimental-apis/OTEL1002.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,12 @@
44

55
This is an Experimental API diagnostic covering the following APIs:
66

7-
* `AlwaysOnExemplarFilter`
8-
* `AlwaysOffExemplarFilter`
97
* `Exemplar`
10-
* `ExemplarFilter`
8+
* `ExemplarFilterType`
119
* `MeterProviderBuilder.SetExemplarFilter` extension method
12-
* `TraceBasedExemplarFilter`
10+
* `ReadOnlyExemplarCollection`
11+
* `ReadOnlyFilteredTagCollection`
12+
* `MetricPoint.TryGetExemplars`
1313

1414
Experimental APIs may be changed or removed in the future.
1515

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# OpenTelemetry .NET Diagnostic: OTEL1004
2+
3+
## Overview
4+
5+
This is an Experimental API diagnostic covering the following APIs:
6+
7+
* `ExemplarReservoir`
8+
* `FixedSizeExemplarReservoir`
9+
* `ExemplarMeasurement<T>`
10+
* `MetricStreamConfiguration.ExemplarReservoirFactory.get`
11+
* `MetricStreamConfiguration.ExemplarReservoirFactory.set`
12+
13+
Experimental APIs may be changed or removed in the future.
14+
15+
## Details
16+
17+
The OpenTelemetry Specification defines an [ExemplarReservoir
18+
API](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#exemplarreservoir)
19+
and a mechanism for configuring `ExemplarReservoir` via the [View
20+
API](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#stream-configuration)
21+
in the Metrics SDK.
22+
23+
From the specification:
24+
25+
> The SDK MUST provide a mechanism for SDK users to provide their own
26+
> ExemplarReservoir implementation. This extension MUST be configurable on a
27+
> metric View, although individual reservoirs MUST still be instantiated per
28+
> metric-timeseries...
29+
30+
We are exposing these APIs experimentally for the following reasons:
31+
32+
* `FixedSizeExemplarReservoir` is not part of the spec. It is meant to help
33+
custom reservoir authors and takes care of correctly creating & updating
34+
`struct Exemplar`s (managing tag filtering when views are used), handles
35+
`Exemplar` collection, and ensures all operations are safe to be called
36+
concurrency (spec requirement). We want to see if this is helpful and meets
37+
the needs of users.
38+
39+
* There is currently no way to use
40+
`MetricStreamConfiguration.ExemplarReservoirFactory` to switch a metric to a
41+
different built-in reservoir (`AlignedHistogramBucketExemplarReservoir` or
42+
`SimpleFixedSizeExemplarReservoir`). This is something supported by the spec
43+
but we want to understand the use cases and needs before exposing these types.
44+
Also it seems the default reservoirs may change.
45+
46+
* There is currently no way to get access to the bucket index inside a reservoir
47+
when a measurement is recorded against a histogram with explicit bounds. The
48+
spec says the reservoir should calculate this given the
49+
definition/configuration of the bounds but the SDK has already done this
50+
computation. It seems unncessarily complicated to expose the configuration and
51+
wasteful to do the work twice. We want to understand the types of algorithms
52+
which users will want to implement before exposing something.
53+
54+
**TL;DR** We want to gather feedback on the usability of the API and for the
55+
need(s) in general for custom reservoirs before exposing a stable API.
56+
57+
<!--
58+
## Provide feedback
59+
60+
Please provide feedback on [this issue](TODO) if you need stable support for
61+
custom `ExemplarReservoir`s.
62+
-->

docs/diagnostics/experimental-apis/README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,12 @@ Description: MetricStreamConfiguration CardinalityLimit Support
3939

4040
Details: [OTEL1003](./OTEL1003.md)
4141

42+
### OTEL1004
43+
44+
Description: ExemplarReservoir Support
45+
46+
Details: [OTEL1004](./OTEL1004.md)
47+
4248
## Inactive
4349

4450
Experimental APIs which have been released stable or removed:

src/OpenTelemetry/.publicApi/Experimental/PublicAPI.Unshipped.txt

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,12 @@ OpenTelemetry.Metrics.ExemplarMeasurement<T>.ExemplarMeasurement() -> void
2525
OpenTelemetry.Metrics.ExemplarMeasurement<T>.Tags.get -> System.ReadOnlySpan<System.Collections.Generic.KeyValuePair<string!, object?>>
2626
OpenTelemetry.Metrics.ExemplarMeasurement<T>.Value.get -> T
2727
OpenTelemetry.Metrics.ExemplarReservoir
28-
OpenTelemetry.Metrics.ExemplarReservoir.ExemplarReservoir() -> void
2928
OpenTelemetry.Metrics.ExemplarReservoir.ResetOnCollect.get -> bool
29+
OpenTelemetry.Metrics.FixedSizeExemplarReservoir
30+
OpenTelemetry.Metrics.FixedSizeExemplarReservoir.Capacity.get -> int
31+
OpenTelemetry.Metrics.FixedSizeExemplarReservoir.FixedSizeExemplarReservoir(int capacity) -> void
32+
OpenTelemetry.Metrics.FixedSizeExemplarReservoir.UpdateExemplar(int exemplarIndex, in OpenTelemetry.Metrics.ExemplarMeasurement<double> measurement) -> void
33+
OpenTelemetry.Metrics.FixedSizeExemplarReservoir.UpdateExemplar(int exemplarIndex, in OpenTelemetry.Metrics.ExemplarMeasurement<long> measurement) -> void
3034
OpenTelemetry.Metrics.MetricPoint.TryGetExemplars(out OpenTelemetry.Metrics.ReadOnlyExemplarCollection exemplars) -> bool
3135
OpenTelemetry.Metrics.MetricStreamConfiguration.CardinalityLimit.get -> int?
3236
OpenTelemetry.Metrics.MetricStreamConfiguration.CardinalityLimit.set -> void
@@ -46,6 +50,7 @@ OpenTelemetry.ReadOnlyFilteredTagCollection.Enumerator.Enumerator() -> void
4650
OpenTelemetry.ReadOnlyFilteredTagCollection.Enumerator.MoveNext() -> bool
4751
OpenTelemetry.ReadOnlyFilteredTagCollection.GetEnumerator() -> OpenTelemetry.ReadOnlyFilteredTagCollection.Enumerator
4852
OpenTelemetry.ReadOnlyFilteredTagCollection.ReadOnlyFilteredTagCollection() -> void
53+
override sealed OpenTelemetry.Metrics.FixedSizeExemplarReservoir.Collect() -> OpenTelemetry.Metrics.ReadOnlyExemplarCollection
4954
static OpenTelemetry.Logs.LoggerProviderBuilderExtensions.AddProcessor(this OpenTelemetry.Logs.LoggerProviderBuilder! loggerProviderBuilder, OpenTelemetry.BaseProcessor<OpenTelemetry.Logs.LogRecord!>! processor) -> OpenTelemetry.Logs.LoggerProviderBuilder!
5055
static OpenTelemetry.Logs.LoggerProviderBuilderExtensions.AddProcessor(this OpenTelemetry.Logs.LoggerProviderBuilder! loggerProviderBuilder, System.Func<System.IServiceProvider!, OpenTelemetry.BaseProcessor<OpenTelemetry.Logs.LogRecord!>!>! implementationFactory) -> OpenTelemetry.Logs.LoggerProviderBuilder!
5156
static OpenTelemetry.Logs.LoggerProviderBuilderExtensions.AddProcessor<T>(this OpenTelemetry.Logs.LoggerProviderBuilder! loggerProviderBuilder) -> OpenTelemetry.Logs.LoggerProviderBuilder!
@@ -63,3 +68,4 @@ static OpenTelemetry.Sdk.CreateLoggerProviderBuilder() -> OpenTelemetry.Logs.Log
6368
static Microsoft.Extensions.Logging.OpenTelemetryLoggingExtensions.UseOpenTelemetry(this Microsoft.Extensions.Logging.ILoggingBuilder! builder) -> Microsoft.Extensions.Logging.ILoggingBuilder!
6469
static Microsoft.Extensions.Logging.OpenTelemetryLoggingExtensions.UseOpenTelemetry(this Microsoft.Extensions.Logging.ILoggingBuilder! builder, System.Action<OpenTelemetry.Logs.LoggerProviderBuilder!>! configure) -> Microsoft.Extensions.Logging.ILoggingBuilder!
6570
static Microsoft.Extensions.Logging.OpenTelemetryLoggingExtensions.UseOpenTelemetry(this Microsoft.Extensions.Logging.ILoggingBuilder! builder, System.Action<OpenTelemetry.Logs.LoggerProviderBuilder!>? configureBuilder, System.Action<OpenTelemetry.Logs.OpenTelemetryLoggerOptions!>? configureOptions) -> Microsoft.Extensions.Logging.ILoggingBuilder!
71+
virtual OpenTelemetry.Metrics.FixedSizeExemplarReservoir.OnCollected() -> void

src/OpenTelemetry/CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@
1212
which could have led to a measurement being dropped.
1313
([#5546](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5546))
1414

15+
* **Experimental (pre-release builds only):** Exposed
16+
`FixedSizeExemplarReservoir` as a public API to support custom implementations
17+
of `ExemplarReservoir` which may be configured using the
18+
`ExemplarReservoirFactory` property on the View API.
19+
([#5558](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5558))
20+
1521
## 1.8.1
1622

1723
Released 2024-Apr-17

src/OpenTelemetry/Metrics/Exemplar/ExemplarMeasurement.cs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,10 @@ namespace OpenTelemetry.Metrics;
1212
/// <summary>
1313
/// Represents an Exemplar measurement.
1414
/// </summary>
15-
/// <remarks><inheritdoc cref="Exemplar" path="/remarks/para[@experimental-warning='true']"/></remarks>
15+
/// <remarks><inheritdoc cref="ExemplarReservoir" path="/remarks/para[@experimental-warning='true']"/></remarks>
1616
/// <typeparam name="T">Measurement type.</typeparam>
1717
#if NET8_0_OR_GREATER
18-
[Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)]
18+
[Experimental(DiagnosticDefinitions.ExemplarReservoirExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)]
1919
#endif
2020
public
2121
#else

src/OpenTelemetry/Metrics/Exemplar/ExemplarReservoir.cs

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,19 +13,26 @@ namespace OpenTelemetry.Metrics;
1313
/// ExemplarReservoir base implementation and contract.
1414
/// </summary>
1515
/// <remarks>
16-
/// <inheritdoc cref="Exemplar" path="/remarks/para[@experimental-warning='true']"/>
16+
/// <para experimental-warning="true"><b>WARNING</b>: This is an experimental API which might change or be removed in the future. Use at your own risk.</para>
1717
/// Specification: <see
1818
/// href="https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#exemplarreservoir"/>.
1919
/// </remarks>
2020
#if NET8_0_OR_GREATER
21-
[Experimental(DiagnosticDefinitions.ExemplarExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)]
21+
[Experimental(DiagnosticDefinitions.ExemplarReservoirExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)]
2222
#endif
2323
public
2424
#else
2525
internal
2626
#endif
2727
abstract class ExemplarReservoir
2828
{
29+
// Note: This constructor is internal because we don't allow custom
30+
// ExemplarReservoir implementations to be based directly on the base class
31+
// only FixedSizeExemplarReservoir.
32+
internal ExemplarReservoir()
33+
{
34+
}
35+
2936
/// <summary>
3037
/// Gets a value indicating whether or not the <see
3138
/// cref="ExemplarReservoir"/> should reset its state when performing

src/OpenTelemetry/Metrics/Exemplar/FixedSizeExemplarReservoir.cs

Lines changed: 71 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,56 @@
11
// Copyright The OpenTelemetry Authors
22
// SPDX-License-Identifier: Apache-2.0
33

4+
#if EXPOSE_EXPERIMENTAL_FEATURES && NET8_0_OR_GREATER
5+
using System.Diagnostics.CodeAnalysis;
6+
#endif
47
using OpenTelemetry.Internal;
58

69
namespace OpenTelemetry.Metrics;
710

8-
internal abstract class FixedSizeExemplarReservoir : ExemplarReservoir
11+
#if EXPOSE_EXPERIMENTAL_FEATURES
12+
/// <summary>
13+
/// An <see cref="ExemplarReservoir"/> implementation which contains a fixed
14+
/// number of <see cref="Exemplar"/>s.
15+
/// </summary>
16+
/// <remarks><inheritdoc cref="ExemplarReservoir" path="/remarks/para[@experimental-warning='true']"/></remarks>
17+
#if NET8_0_OR_GREATER
18+
[Experimental(DiagnosticDefinitions.ExemplarReservoirExperimentalApi, UrlFormat = DiagnosticDefinitions.ExperimentalApiUrlFormat)]
19+
#endif
20+
public
21+
#else
22+
internal
23+
#endif
24+
abstract class FixedSizeExemplarReservoir : ExemplarReservoir
925
{
1026
private readonly Exemplar[] runningExemplars;
1127
private readonly Exemplar[] snapshotExemplars;
1228

29+
/// <summary>
30+
/// Initializes a new instance of the <see cref="FixedSizeExemplarReservoir"/> class.
31+
/// </summary>
32+
/// <param name="capacity">The capacity (number of <see cref="Exemplar"/>s)
33+
/// to be contained in the reservoir.</param>
34+
#pragma warning disable RS0022 // Constructor make noninheritable base class inheritable
1335
protected FixedSizeExemplarReservoir(int capacity)
36+
#pragma warning restore RS0022 // Constructor make noninheritable base class inheritable
1437
{
38+
// Note: RS0022 is suppressed because we do want to allow custom
39+
// ExemplarReservoir implementations to be created by deriving from
40+
// FixedSizeExemplarReservoir.
41+
1542
Guard.ThrowIfOutOfRange(capacity, min: 1);
1643

1744
this.runningExemplars = new Exemplar[capacity];
1845
this.snapshotExemplars = new Exemplar[capacity];
1946
this.Capacity = capacity;
2047
}
2148

22-
internal int Capacity { get; }
49+
/// <summary>
50+
/// Gets the capacity (number of <see cref="Exemplar"/>s) contained in the
51+
/// reservoir.
52+
/// </summary>
53+
public int Capacity { get; }
2354

2455
/// <summary>
2556
/// Collects all the exemplars accumulated by the Reservoir.
@@ -56,12 +87,48 @@ internal sealed override void Initialize(AggregatorStore aggregatorStore)
5687
base.Initialize(aggregatorStore);
5788
}
5889

90+
internal void UpdateExemplar<T>(
91+
int exemplarIndex,
92+
in ExemplarMeasurement<T> measurement)
93+
where T : struct
94+
{
95+
this.runningExemplars[exemplarIndex].Update(in measurement);
96+
}
97+
98+
/// <summary>
99+
/// Fired when <see cref="Collect"/> has finished before returning a <see cref="ReadOnlyExemplarCollection"/>.
100+
/// </summary>
101+
/// <remarks>
102+
/// Note: This method is typically used to reset the state of reservoirs and
103+
/// is called regardless of the value of <see
104+
/// cref="ExemplarReservoir.ResetOnCollect"/>.
105+
/// </remarks>
59106
protected virtual void OnCollected()
60107
{
61108
}
62109

63-
protected void UpdateExemplar<T>(int exemplarIndex, in ExemplarMeasurement<T> measurement)
64-
where T : struct
110+
/// <summary>
111+
/// Updates the <see cref="Exemplar"/> stored in the reservoir at the
112+
/// specified index with an <see cref="ExemplarMeasurement{T}"/>.
113+
/// </summary>
114+
/// <param name="exemplarIndex">Index of the <see cref="Exemplar"/> to update.</param>
115+
/// <param name="measurement"><see cref="ExemplarMeasurement{T}"/>.</param>
116+
protected void UpdateExemplar(
117+
int exemplarIndex,
118+
in ExemplarMeasurement<long> measurement)
119+
{
120+
this.runningExemplars[exemplarIndex].Update(in measurement);
121+
}
122+
123+
/// <summary>
124+
/// Updates the <see cref="Exemplar"/> stored in the reservoir at the
125+
/// specified index with an <see cref="ExemplarMeasurement{T}"/>.
126+
/// </summary>
127+
/// <param name="exemplarIndex">Index of the <see cref="Exemplar"/> to update.</param>
128+
/// <param name="measurement"><see cref="ExemplarMeasurement{T}"/>.</param>
129+
protected void UpdateExemplar(
130+
int exemplarIndex,
131+
in ExemplarMeasurement<double> measurement)
65132
{
66133
this.runningExemplars[exemplarIndex].Update(in measurement);
67134
}

0 commit comments

Comments
 (0)