From d7bf628123a915a304afe131dd2fff37e3494242 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Tue, 19 Nov 2024 11:17:13 -0800 Subject: [PATCH 1/6] Add InstrumentAdvice details to instrumentation doc. --- .../diagnostics/metrics-instrumentation.md | 67 ++++++++++++++++++- 1 file changed, 66 insertions(+), 1 deletion(-) diff --git a/docs/core/diagnostics/metrics-instrumentation.md b/docs/core/diagnostics/metrics-instrumentation.md index f739e46be0ebc..cc820a625560f 100644 --- a/docs/core/diagnostics/metrics-instrumentation.md +++ b/docs/core/diagnostics/metrics-instrumentation.md @@ -2,7 +2,7 @@ title: Creating Metrics description: How to add new metrics to a .NET library or application ms.topic: tutorial -ms.date: 08/27/2024 +ms.date: 11/19/2024 --- # Creating metrics @@ -224,6 +224,11 @@ Types of instruments currently available: to record these measurements during the collection tool's update interval: 1,5,2,3,10,9,7,4,6,8. A collection tool might report that the 50th, 90th, and 95th percentiles of these measurements are 5, 9, and 9 respectively. + > [!NOTE] + > For details about how to set the recommended bucket boundaries when creating + > a Histogram instrument see: [Using Advice to customize Histogram + > instruments](#using-advice-to-customize-histogram-instruments). + ### Best practices when selecting an instrument type - For counting things, or any other value that solely increases over time, use Counter or ObservableCounter. Choose between Counter and ObservableCounter depending on which @@ -542,6 +547,66 @@ Name Current Value > [!NOTE] > OpenTelemetry refers to tags as 'attributes'. These are two different names for the same functionality. +## Using Advice to customize Histogram instruments + +Consumers of Histogram instruments can choose different aggregation strategies. +The default mode when using OpenTelemetry is [Explicit Bucket Histogram +Aggregation](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#explicit-bucket-histogram-aggregation). + +When using explicit bucket aggregation the measurements recorded through a +Histogram will be grouped into buckets which track the sum and count of values +to that bucket. + +The default bucket configuration is described in the OpenTelemetry +Specification: `[ 0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, +7500, 10000 ]`. + +The default values may not lead to the best granularity for every Histogram. For +example, sub-second request durations would all fall into the `0` bucket. + +To solve this problem the `9.0.0` version of the +`System.Diagnostics.DiagnosticSource` package introduced the +() API. + +The `InstrumentAdvice` API may be used by instrumentation authors to specify the +set of recommended default bucket boundaries for a given Histogram. Consumers +can then choose to use those values when configuring aggregation. + +The following code shows an example using the `InstrumentAdvice` API to set +recommended default buckets. + +```csharp +using System; +using System.Diagnostics.Metrics; +using System.Threading; + +class Program +{ + static Meter s_meter = new Meter("HatCo.Store"); + static readonly IReadOnlyList s_shortSecondsBucketBoundaries = [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10]; + static Histogram s_orderProcessingTime = s_meter.CreateHistogram( + name: "hatco.store.order_processing_time", + unit: "s", + description: "Duration order processing.", + advice: new InstrumentAdvice { HistogramBucketBoundaries = s_shortSecondsBucketBoundaries }); + + static Random s_rand = new Random(); + + static void Main(string[] args) + { + Console.WriteLine("Press any key to exit"); + while (!Console.KeyAvailable) + { + // Pretend our store has one transaction each 100ms + Thread.Sleep(100); + + // Pretend that we measured how long it took to do the transaction (for example we could time it with Stopwatch) + s_orderProcessingTime.Record(s_rand.Next(5, 15) / 1000.0); + } + } +} +``` + ## Test custom metrics Its possible to test any custom metrics you add using . This type makes it easy to record the measurements from specific instruments and assert the values were correct. From ebf9bef290175c6a3f981305a66180ea0a56a941 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Wed, 20 Nov 2024 13:25:46 -0800 Subject: [PATCH 2/6] Code review. --- .../diagnostics/metrics-instrumentation.md | 50 +++++++++++++------ 1 file changed, 35 insertions(+), 15 deletions(-) diff --git a/docs/core/diagnostics/metrics-instrumentation.md b/docs/core/diagnostics/metrics-instrumentation.md index cc820a625560f..198560705592d 100644 --- a/docs/core/diagnostics/metrics-instrumentation.md +++ b/docs/core/diagnostics/metrics-instrumentation.md @@ -549,16 +549,20 @@ Name Current Value ## Using Advice to customize Histogram instruments -Consumers of Histogram instruments can choose different aggregation strategies. -The default mode when using OpenTelemetry is [Explicit Bucket Histogram -Aggregation](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#explicit-bucket-histogram-aggregation). - -When using explicit bucket aggregation the measurements recorded through a -Histogram will be grouped into buckets which track the sum and count of values -to that bucket. - -The default bucket configuration is described in the OpenTelemetry -Specification: `[ 0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, +When using Histograms, it is the responsibility of the tool or library +collecting the data to decide how best to represent the distribution of values +that were recorded. A common strategy (and the [default mode when using +OpenTelemetry](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#explicit-bucket-histogram-aggregation)) +is to divide up the range of possible values into sub-ranges called buckets and +report how many recorded values were in each bucket. For example a tool might +divide numbers into three buckets, those less than 1, those between 1-10, and +those greater than 10. If your app recorded the values 0.5, 6, 0.1, 12 then +there would be two data points the first bucket, one in the second, and one in +the 3rd. + +The tool or library collecting the Histogram data is responsible for defining +the buckets it will use. The default bucket configuration when using +OpenTelemetry is: `[ 0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000 ]`. The default values may not lead to the best granularity for every Histogram. For @@ -569,8 +573,17 @@ To solve this problem the `9.0.0` version of the () API. The `InstrumentAdvice` API may be used by instrumentation authors to specify the -set of recommended default bucket boundaries for a given Histogram. Consumers -can then choose to use those values when configuring aggregation. +set of recommended default bucket boundaries for a given Histogram. The tool or +library collecting the Histogram data can then choose to use those values when +configuring aggregation. This is supported in the OpenTelemetry .NET SDK as of +version `1.10.0`. + +> [!IMPORTANT] +> In general more buckets will lead to more precise data for a given Histogram +> but each bucket requires memory to store the aggregated details. It is +> important to understand this tradeoff between precision and memory consumption +> when choosing the number of buckets to recommend via the `InstrumentAdvice` +> API. The following code shows an example using the `InstrumentAdvice` API to set recommended default buckets. @@ -583,12 +596,11 @@ using System.Threading; class Program { static Meter s_meter = new Meter("HatCo.Store"); - static readonly IReadOnlyList s_shortSecondsBucketBoundaries = [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10]; static Histogram s_orderProcessingTime = s_meter.CreateHistogram( name: "hatco.store.order_processing_time", unit: "s", - description: "Duration order processing.", - advice: new InstrumentAdvice { HistogramBucketBoundaries = s_shortSecondsBucketBoundaries }); + description: "Order processing duration", + advice: new InstrumentAdvice { HistogramBucketBoundaries = [0.01, 0.05, 0.1, 0.5, 1, 5] }); static Random s_rand = new Random(); @@ -607,6 +619,14 @@ class Program } ``` +### Additional information + +For more details about explicit bucket Histograms in OpenTelemetry see: + +* [Why Histograms?](https://opentelemetry.io/blog/2023/why-histograms/) + +* [Histograms vs Summaries](https://opentelemetry.io/blog/2023/histograms-vs-summaries/) + ## Test custom metrics Its possible to test any custom metrics you add using . This type makes it easy to record the measurements from specific instruments and assert the values were correct. From 385df5bbc6f5222755d69c863eae39742a6d2e9e Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Wed, 20 Nov 2024 14:45:21 -0800 Subject: [PATCH 3/6] Code review. --- docs/core/diagnostics/metrics-instrumentation.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/core/diagnostics/metrics-instrumentation.md b/docs/core/diagnostics/metrics-instrumentation.md index 198560705592d..717582ad3b516 100644 --- a/docs/core/diagnostics/metrics-instrumentation.md +++ b/docs/core/diagnostics/metrics-instrumentation.md @@ -580,10 +580,11 @@ version `1.10.0`. > [!IMPORTANT] > In general more buckets will lead to more precise data for a given Histogram -> but each bucket requires memory to store the aggregated details. It is -> important to understand this tradeoff between precision and memory consumption -> when choosing the number of buckets to recommend via the `InstrumentAdvice` -> API. +> but each bucket requires memory to store the aggregated details and there is +> CPU cost to find the correct bucket when processing a measurement. It is +> important to understand the tradeoffs between precision and CPU/memory +> consumption when choosing the number of buckets to recommend via the +> `InstrumentAdvice` API. The following code shows an example using the `InstrumentAdvice` API to set recommended default buckets. From 731940546750f07d300f9765f345e0fc77a9d121 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Thu, 21 Nov 2024 11:23:10 -0800 Subject: [PATCH 4/6] Code review. --- docs/core/diagnostics/metrics-instrumentation.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/core/diagnostics/metrics-instrumentation.md b/docs/core/diagnostics/metrics-instrumentation.md index 717582ad3b516..3a0cba18e9c3d 100644 --- a/docs/core/diagnostics/metrics-instrumentation.md +++ b/docs/core/diagnostics/metrics-instrumentation.md @@ -568,15 +568,22 @@ OpenTelemetry is: `[ 0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, The default values may not lead to the best granularity for every Histogram. For example, sub-second request durations would all fall into the `0` bucket. -To solve this problem the `9.0.0` version of the +The tool or library collecting the Histogram data may offer mechanism(s) to +allow users to customize the bucket configuration. For example, OpenTelemetry +defines a [View +API](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#view). +This however requires end user action and makes it the user's responsibility to +understand the data distribution well enough to choose correct buckets. + +To improve the experience the `9.0.0` version of the `System.Diagnostics.DiagnosticSource` package introduced the () API. The `InstrumentAdvice` API may be used by instrumentation authors to specify the set of recommended default bucket boundaries for a given Histogram. The tool or library collecting the Histogram data can then choose to use those values when -configuring aggregation. This is supported in the OpenTelemetry .NET SDK as of -version `1.10.0`. +configuring aggregation leading to a more seamless onboarding experience for +users. This is supported in the OpenTelemetry .NET SDK as of version `1.10.0`. > [!IMPORTANT] > In general more buckets will lead to more precise data for a given Histogram From b77c9d3dde200c97ebddbaf11579f70ee717a11b Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 22 Nov 2024 11:13:35 -0800 Subject: [PATCH 5/6] Lint. --- docs/core/diagnostics/metrics-instrumentation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/core/diagnostics/metrics-instrumentation.md b/docs/core/diagnostics/metrics-instrumentation.md index 3a0cba18e9c3d..9982ae8b6e18b 100644 --- a/docs/core/diagnostics/metrics-instrumentation.md +++ b/docs/core/diagnostics/metrics-instrumentation.md @@ -577,7 +577,7 @@ understand the data distribution well enough to choose correct buckets. To improve the experience the `9.0.0` version of the `System.Diagnostics.DiagnosticSource` package introduced the -() API. +() API. The `InstrumentAdvice` API may be used by instrumentation authors to specify the set of recommended default bucket boundaries for a given Histogram. The tool or From 9161132fe49fb94a83600a3945d4c859de9a1144 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 22 Nov 2024 11:45:14 -0800 Subject: [PATCH 6/6] Tweak. --- docs/core/diagnostics/metrics-instrumentation.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/core/diagnostics/metrics-instrumentation.md b/docs/core/diagnostics/metrics-instrumentation.md index 9982ae8b6e18b..219580e22e141 100644 --- a/docs/core/diagnostics/metrics-instrumentation.md +++ b/docs/core/diagnostics/metrics-instrumentation.md @@ -583,7 +583,8 @@ The `InstrumentAdvice` API may be used by instrumentation authors to specify the set of recommended default bucket boundaries for a given Histogram. The tool or library collecting the Histogram data can then choose to use those values when configuring aggregation leading to a more seamless onboarding experience for -users. This is supported in the OpenTelemetry .NET SDK as of version `1.10.0`. +users. This is supported in the [OpenTelemetry .NET SDK as of version +1.10.0](https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/metrics/customizing-the-sdk#configuring-the-aggregation-of-a-histogram). > [!IMPORTANT] > In general more buckets will lead to more precise data for a given Histogram