View-based metric levels #4828
Description
Depends on #4809. Forked from that PR. My intent is to demonstrate that "levels", or "detail presets" could be implemented as a layer on-top-of views.
Objective
Provide users with easy "presets" for controlling the level of detail of their metrics.
Design Constraints:
- Avoid adding a concept of levels directly to the metric API surface, to:
- Avoid imposing additional complexity on instrumentation authors.
- Avoid the "severity sprawl" that inevitably results in a large number of levels.
- Avoid the resulting inconsistency between how different libraries use each level.
- Accomplish this in a way that builds on views, rather than introduces a new, orthogonal API/SDK concept that is difficult to reason about in the presence of views.
- Be simple to implement, and optional for users to engage with.
Prior Art and Related Ideas
This is similar to #3205, but for metrics.
The collector has implemented "basic", "normal" (default), and "detailed" levels: https://opentelemetry.io/docs/collector/internal-telemetry/.
Related collector issues:
- Originally added in Add basic telemetry to collector census-instrumentation/opencensus-service#284 to control which attributes were added to telemetry
- [component] Add support for components specifying what instruments to enable depending on metric level opentelemetry-collector#11754
Proposal: Pre-defined Views
Introduce two new pre-defined Views in the metrics SDK: "Basic View", and "Detailed View". The Basic view configures metrics to produce the bare minimum of detail for metrics, and the Detailed View turns on additional details.
These pre-defined views would be exposed via declarative configuration:
meter_provider:
views:
- basic:Basic View
Note: This is hypothetical, and details will be ironed out later. We could land on a wide variety of presets.
The basic view:
- Disables all attributes except "error.type"
- Uses a reduced set of buckets on histograms.
- Disables min/max tracking.
- Uses a fixed-size reservoir of size 1.
Detailed View
The detailed view:
- Enables all OptIn instruments
- Enables all attributes.
- Uses exponential buckets
Rationale
"Level of detail" for metrics is primarily about controlling cardinality, but also can involve reducing the amount of data sent over the wire. To control level of detail, a user can:
- Control which metrics are enabled.
- Control which attributes are present on metrics.
- Control the aggregation used, including:
- The number buckets on histograms
- How many exemplars are stored.
- Whether min and max are tracked.
A naive application of logging levels to metrics would only control which metrics are enabled, but I think we can address more of the above while imposing less complexity on instrumentation authors.
Drawbacks
This proposal doesn't solve everything that levels could solve if they were part of the API. Since the API only distinguishes between default-on and default-off instruments and attributes, it limits the number of levels that are possible. For example, it isn't feasible to disable some metrics when moving from the default to the "basic" View. It also isn't possible to add additional "levels" between the default and "detailed" which enables progressively more-detailed metrics.
This proposal also doesn't make it easy to merge the "basic" or "detailed" view with user-defined views. If they were naively both added via declarative configuration, you would get duplicated metrics. What we probably want is to be able to override the "default view" with the "basic" or "detailed" view, and then apply user-defined views on-top-of those. That could be done with additional MeterProvider config (e.g. A DefaultView configuration option), and corresponding section in declarative config.
meter_provider:
default_view: basic
views:
# user-defined views here override the basic view.Tip: React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.