Skip to content

Review the language around standard attributes (and breaking changes) #4468

@lmolkova

Description

@lmolkova

The standard attribute definition SHOULD be used to represent attributes in data
modeling unless there is a strong justification to diverge. For example, the Log
Data Model has an extended [attributes](../logs/data-model.md#field-attributes)
definition allowing values of [type `Any`](../logs/data-model.md#type-any). This
reflects that LogRecord attributes are expected to model data produced from
external log APIs, which do not necessarily have the same value type
restrictions as the standard attribute definition.
Note: Extending the set of standard attribute value types is a breaking change.
This was decided after extensive debate, with arguments as follows:
* Limiting the types of attribute values to a set which has proved sufficient
during several years of OpenTelemetry's development is a useful guardrail for
design. In taking additional value types off the table, we narrow the solution
space and have more productive design conversations.
* Upon proposing to extend support for complex value types, we received significant
pushback. Limiting attribute value types to primitives and arrays of primitives
simplifies data consumers' efforts to create search indexes and perform statistical
analysis.
* To address concerns over restricting standard attributes to primitive types, it was
called out that complex types can be encoded as existing primitive types, such as
representing datetime as a string or 64 bit integer.

It's not clear how the arguments provided lead to the summary that extending standard attributes is a breaking change.
The discussion in the #3858 and the language used in arguments clearly tries to prevent the evolution and document the current decision, but there are no arguments that explain the breaking nature of such change.

If we want to stick to this decision in practice (i.e. as SDKs implement non-standard attributes), we need to:

  • remove the "breaking change" part
  • remove the The standard attribute definition SHOULD be used to represent attributes in data
    modeling unless there is a strong justification to diverge
    - there is no reason for Profiling to stick to standard attributes - there is no prior art in OTel that would be challenged by them
  • explain the limits of this decision such as whether it applies to:
    • OTel API which should not allow complex attributes on spans/metrics API - this does not seem like a good limitation - API should not be that opinionated - that's for SDKs and backends. Moreover, several SDKs violate it already.
    • OTel SDK - API permits complex attributes, but SDK doesn't (e.g. drops them by default) - that'd be the least problematic way to keep the limitation and would still allow the evolution without requiring API v2
    • Semantic conventions - API and SDK allow end users to do anything that makes sense for them in their backend, but OTel semconv won't define complex attributes for certain signals

Having said this, I'd like to re-litigate this decision on the following grounds:

Relitigating this decision does not go against the values - it can be done in non-breaking manner.


The proposal would be to

  • have the same attributes API on all signals
  • by default SDKs would drop (or serialize to JSON) complex attributes on metrics. Users were never able to populate them, so this won't break anything existing.
  • we may allow customizing this behavior and we'll pick a default (drop or serialize). We may even allow complex attributes on metrics as opt-in. This is SDK/exporter configuration.
  • we'd recommend backends that don't support complex attributes on some signal to use the same default - drop or serialize at exporting or ingestion time.
  • we'll restrict complex attribute usage on metrics in otel semconv via tooling/policies

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

Spec - Accepted

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions