Support complex attribute values on logs/events in semantic conventions

[lmolkova]

LogRecord attributes are special - they support complex attribute types unlike spans/metrics/resources that support standard attributes only (primitives and arrays of primitives). Note that the latter is the API limitation (not an OTLP one)

In semconv we assume that all attributes are cross-signal and therefore are limited to 'standard' attribute types.

The proposal is to lift this limitation and allow defining complex attributes in semantic conventions with the following limitations:

• they cannot be referenced on spans, metrics, or resources/entities. TBD: check with profiles
  • this can be solved via the semconv policy (given that attribute types are limited on the OTel API surface and not the protocol, we might not want to hardcode it in weaver)
• we should define if/how such attributes would interact with the standard attributes. E.g. are geo.location.lat=x and geo.location.lon=y attributes set on a span equivalent to geo.location={"lat": x, "lon": y} set on the log record?

This would help with the following problems:

• Logs/Events guidance on what to put to attributes vs body #1651 - we can have one property bag (attributes) to put all the event details into. Standard attributes can be used for correlation across all signals. Complex attributes could be reused across logs preserving semantic meaning (which we don't have today).
• it'd introduce reusable types discussed in Embed functionality v0.2 #1264

Related: open-telemetry/opentelemetry-specification#4334, open-telemetry/opentelemetry-specification#2888

[lmolkova]

Discussed at the spec call on 12/17

Some feedback/notes:

• Backends might leverage complex attributes if they start to pop up, but it does not seem like there is a burning need for them
• Prometheus would have a problem. Overall, complex attributes on metrics do not seem to be useful
• We can keep the door for complex attributes on spans open (and special-case metrics or support flattening for them)
• We can make the first step with complex attributes on logs (as a testing ground)

The first step:

• we should define anything that should be set across signals as a standard attribute
• we can additionally define a complex attribute that includes standard attributes.
• we need to define flattening rules for complex attributes - only things that can be flattened can be used for cross-signal correlation

Things to figure out:

• when should we define a complex attribute vs a set of standard attributes - see also Support maps and heterogeneous arrays as attribute values opentelemetry-specification#2888 (comment)

[felixbarny]

Just to add a use case for where complex attributes on spans would be useful: using the geoipprocessor on the source.address attribute. But I agree that starting with logs and expanding to spans later sounds like a good approach 👍

[lmolkova]

Side-thought on flattening rules - they give us an ability to apply attribute limits to individual leaf fields.

I.e.

{
   "foo1": {"key1": "this is a very long value ...  ", "key2": {...}}
   ....
   "foo142": {...}
}

could be subject to the:

• value length limit for foo1.key1, foo1.key2, etc
• number of unique attribute keys (foo1.key1, foo1.key2, ..., foo142.*)

We should also define reasonable limits on depth (so that key does not get too long)

If we go down this route, we should consider having additional configuration options for applying limits to complex attributes since
backends+distros that'd support complex attributes would want to avoid perf hit introduced by unnecessary validation on the SDK side and may want to apply some custom limits/validation instead

[adriangb]

I 100% agree that we should allow complex attributes everywhere and have a single definition of attributes.

[trask]

this is an interesting argument towards allowing complex attributes everywhere that I hadn't seen previously:

open-telemetry/opentelemetry-specification#376 (comment)

OTTL has a plethora of ways to assign map values to attributes for any signal. We allow this because pdata allows it.

[pellared]

Prometheus would have a problem. Overall, complex attributes on metrics do not seem to be useful

From open-telemetry/opentelemetry-specification#2888 (comment):

If, like arrays, maps are rarely used for metric label values, I don't think this will be a huge issue. We can JSON-encode them when they do appear and let users deal with those strings. But I don't see this being useful for metrics, and would personally be against using these in semantic conventions for metrics.

There is no blocking issue to add support for complex attributes for Prometheus. Complex attributes should simply be used in moderation (avoided) for metrics.