2727
2828</details >
2929
30+ ## Introduction
31+
32+ This document provides comprehensive guidance on leveraging OpenTelemetry
33+ metrics in Rust applications. Whether you're tracking request counts, monitoring
34+ response times, or analyzing resource utilization, this guide equips you with
35+ the knowledge to implement robust and efficient metrics collection.
36+
37+ It covers best practices, API usage patterns, memory management techniques, and
38+ advanced topics to help you design effective metrics solutions while steering
39+ clear of common challenges.
40+
3041## Best Practices
3142
3243// TODO: Add link to the examples, once they are modified to show best
@@ -36,6 +47,10 @@ practices.
3647
3748### Meter
3849
50+ [ Meter] ( https://docs.rs/opentelemetry/latest/opentelemetry/metrics/struct.Meter.html )
51+ provides the ability to create instruments for recording measurements or
52+ accepting callbacks to report measurements.
53+
3954:stop_sign : You should avoid creating duplicate
4055[ ` Meter ` ] ( https://docs.rs/opentelemetry/latest/opentelemetry/metrics/struct.Meter.html )
4156instances with the same name. ` Meter ` is fairly expensive and meant to be reused
@@ -73,8 +88,9 @@ instruments to their corresponding Rust SDK types.
7388
7489:heavy_check_mark : You should understand and pick the right instrument type.
7590
76- > [ !NOTE] Picking the right instrument type for your use case is crucial to
77- > ensure the correct semantics and performance. Check the [ Instrument
91+ > [ !NOTE]
92+ > Picking the right instrument type for your use case is crucial to ensure the
93+ correct semantics and performance. Check the [ Instrument
7894 Selection] ( https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/supplementary-guidelines.md#instrument-selection )
7995 section from the supplementary guidelines for more information.
8096
@@ -212,6 +228,8 @@ fn setup_metrics(meter: &opentelemetry::metrics::Meter) {
212228> [ !NOTE] The callbacks in the Observable instruments are invoked by the SDK
213229 during each export cycle.
214230
231+
232+
215233## MeterProvider Management
216234
217235Most use-cases require you to create ONLY one instance of MeterProvider. You
@@ -236,10 +254,10 @@ you create them. Follow these guidelines:
236254* ** Shutdown** : Explicitly call ` shutdown ` on the ` MeterProvider ` at the end of
237255 your application to ensure all metrics are properly flushed and exported.
238256
239- > [ !NOTE] If you did not use ` opentelemetry::global::set_meter_provider ` to set a
240- > clone of the ` MeterProvider ` as the global provider, then you should be aware
241- > that dropping the last instance of ` MeterProvider ` implicitly calls shutdown on
242- > the provider.
257+ > [ !NOTE] If you did not use ` opentelemetry::global::set_meter_provider ` to set
258+ > a clone of the ` MeterProvider ` as the global provider, then you should be
259+ > aware that dropping the last instance of ` MeterProvider ` implicitly calls
260+ > shutdown on the provider.
243261
244262:heavy_check_mark : Always call ` shutdown ` on the ` MeterProvider ` at the end of
245263your application to ensure proper cleanup.
@@ -367,7 +385,7 @@ Pre-aggregation offers several advantages:
367385
368386### Cardinality Limits
369387
370- The number of unique combinations of attributes for a given metric is referred
388+ The number of distinct combinations of attributes for a given metric is referred
371389to as the cardinality of that metric. Taking the [ fruit example] ( #example ) , if
372390we know that we can only have apple/lemon as the name, red/yellow/green as the
373391color, then we can say the cardinality is 6 (i.e 2 * 3). No matter how many
@@ -419,8 +437,8 @@ see much higher cardinality due to:
419437 infrastructure
4204383 . The possibility of reporting different key-value pair combinations in each
421439 export interval, as the cardinality limit only applies to the number of
422- unique attribute combinations tracked during a single export interval. (This
423- is only applicable to Delta temporality)
440+ distinct attribute combinations tracked during a single export interval.
441+ (This is only applicable to Delta temporality)
424442
425443Therefore, the actual cardinality in your metrics backend can be orders of
426444magnitude higher than what any single OTel SDK process handles in an export
@@ -429,9 +447,9 @@ cycle.
429447#### Cardinality Limits - Implications
430448
431449Cardinality limits are enforced for each export interval, meaning the metrics
432- aggregation system only allows up to the configured cardinality limit of unique
433- attribute combinations per metric. Understanding how this works in practice is
434- important:
450+ aggregation system only allows up to the configured cardinality limit of
451+ distinct attribute combinations per metric. Understanding how this works in
452+ practice is important:
435453
436454* ** Cardinality Capping** : When the limit is reached within an export interval,
437455 any new attribute combination is not individually tracked but instead folded
@@ -448,7 +466,7 @@ important:
448466 collection/export cycle. This means in each new interval, the SDK can track
449467 up to the cardinality limit of completely different attribute combinations.
450468 Over time, your metrics backend might see far more than the configured limit
451- of unique combinations from a single process.
469+ of distinct combinations from a single process.
452470
453471 * ** Cumulative Temporality** : Since the SDK maintains state across export
454472 intervals, once the cardinality limit is reached, new attribute combinations
@@ -457,7 +475,7 @@ important:
457475 for the lifetime of that metric instrument.
458476
459477* ** Impact on Monitoring** : While cardinality limits protect your system from
460- unbounded resource consumption, they do mean that high-cardinality dimensions
478+ unbounded resource consumption, they do mean that high-cardinality attributes
461479 may not be fully represented in your metrics. Since cardinality capping can
462480 cause metrics to be folded into the overflow bucket, it becomes impossible to
463481 predict which specific attribute combinations were affected across multiple
@@ -467,12 +485,13 @@ important:
467485 metrics in any backend system:
468486
469487 * ** Total Accuracy** : OpenTelemetry Metrics always ensures the total
470- aggregation (sum of metric values across all dimensions ) remains accurate,
488+ aggregation (sum of metric values across all attributes ) remains accurate,
471489 even when overflow occurs.
472490
473491 * ** Attribute-Based Query Limitations** : Any metric query based on specific
474- attributes could be misleading, as it's possible those dimensions were
475- folded into the overflow bucket due to cardinality capping.
492+ attributes could be misleading, as it's possible that measurements recorded
493+ with a superset of those attributes were folded into the overflow bucket due
494+ to cardinality capping.
476495
477496 * ** All Attributes Affected** : When overflow occurs, it's not just
478497 high-cardinality attributes that are affected. The entire attribute
@@ -481,9 +500,9 @@ important:
481500
482501#### Cardinality Limits - Example
483502
484- Extending our fruit sales tracking example, imagine we set a
485- cardinality limit of 3 and we're tracking sales with attributes for ` name ` ,
486- ` color ` , and ` store_location ` :
503+ Extending our fruit sales tracking example, imagine we set a cardinality limit
504+ of 3 and we're tracking sales with attributes for ` name ` , ` color ` , and
505+ ` store_location ` :
487506
488507During a busy sales period at time (T3, T4] , we record:
489508
@@ -506,7 +525,7 @@ The exported metrics would be:
506525 If we later query "How many red apples were sold?" the answer would be 10, not
507526 13, because the Midtown sales were folded into the overflow bucket. Similarly,
508527 queries about "How many items were sold in Midtown?" would return 0, not 3.
509- However, the total count across all dimensions (i.e How many total fruits were
528+ However, the total count across all attributes (i.e How many total fruits were
510529 sold in (T3, T4] would correctly give 26) would be accurate.
511530
512531 This limitation applies regardless of whether the attribute in question is
@@ -519,13 +538,6 @@ The exported metrics would be:
519538 words, attributes used to create ` Meter ` or ` Resource ` attributes are not
520539 subject to this cap.
521540
522- :heavy_check_mark : Use ` Meter ` attributes or ` Resource ` attributes as
523- appropriate, see [ modelling attributes] ( #modelling-metric-attributes ) for
524- details. In the above example, if a process only sells fruits from a
525- particular store, then store_location attribute should be modelled as a
526- Resource - this is not only efficient, but reduced the cardinality capping
527- possibility as well.
528-
529541// TODO: Document how to pick cardinality limit.
530542
531543### Memory Preallocation
@@ -542,7 +554,7 @@ is ineffective and can make metrics practically unusable. Moreover, it can
542554quickly lead to cardinality issues, resulting in metrics being capped.
543555
544556A better alternative is to use a concept in OpenTelemetry called
545- [ exemplars ] ( https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#exemplar ) .
557+ [ Exemplars ] ( https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#exemplar ) .
546558Exemplars provide a mechanism to correlate metrics with traces by sampling
547559specific measurements and attaching trace context to them.
548560
@@ -553,10 +565,10 @@ Currently, exemplars are not yet implemented in the OpenTelemetry Rust SDK.
553565When metrics are being collected, they normally get stored in a [ time series
554566database] ( https://en.wikipedia.org/wiki/Time_series_database ) . From storage and
555567consumption perspective, metrics can be multi-dimensional. Taking the [ fruit
556- example] ( #example ) , there are two dimensions - "name" and "color". For basic
557- scenarios, all the dimensions can be reported during the [ Metrics
568+ example] ( #example ) , there are two attributes - "name" and "color". For basic
569+ scenarios, all the attributes can be reported during the [ Metrics
558570API] ( #metrics-api ) invocation, however, for less trivial scenarios, the
559- dimensions can come from different sources:
571+ attributes can come from different sources:
560572
561573* [ Measurements] ( https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/api.md#measurement )
562574 reported via the [ Metrics API] ( #metrics-api ) .
@@ -567,12 +579,12 @@ dimensions can come from different sources:
567579* Additional attributes provided by the collector. For example, [ jobs and
568580 instances] ( https://prometheus.io/docs/concepts/jobs_instances/ ) in Prometheus.
569581
570- Here is the rule of thumb when modeling the dimensions :
582+ Here is the rule of thumb when modeling the attributes :
571583
572584* If the dimension is static throughout the process lifetime (e.g. the name of
573585 the machine, data center):
574586 * If the dimension applies to all metrics, model it as Resource, or even
575- better, let the collector add these dimensions if feasible (e.g. a collector
587+ better, let the collector add these attributes if feasible (e.g. a collector
576588 running in the same data center should know the name of the data center,
577589 rather than relying on / trusting each service instance to report the data
578590 center name).
0 commit comments