Empty string attributes and OTLP

[breedx-splk]

There appears to be a disconnect in how the spec wording and protobuf definitions treat empty string values in attributes.

Per the spec, these empty string Attribute values are important:

Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to processors / exporters.

This strongly suggests that empty string attribute values should be serialized as strings of zero length, and not simply as an "empty" Attribute.

The .proto definition, on the other hand, suggests something else within AnyValue (of which string is a type):

// The value is one of the listed fields. It is valid for all values to be unspecified
// in which case this AnyValue is considered to be "empty".

Because of this, many (all?) implementations do not specify a value at all when serializing an empty string, because it is assumed to be empty.

⭐ The problem, then, is that the Attribute's type information is lost.

A consumer cannot differentiate between an empty string and other empty types (array, list, object).

This issue was uncovered when attempting to deserialize protobuf data that contained empty string attribute values (from java contrib disk buffering code, linked below).

It can also be shown with json, in that the "empty" case omits the stringValue value, like this:

{
  "key": "empty",
  "value": {} // no type information!
}

instead of

{
  "key": "empty",
  "value": {
    "stringValue": "" // clearly a string
  }
}

Related:

• Allow empty strings to be written to protobuf opentelemetry-java#7656
• DiskBuffering crashes with empty string attributes opentelemetry-java-contrib#2257

[jsuereth]

The oddity here too - is that the specification seems to only imply type is preserved within the SDK but would be immediately lost when export over the protocol.

I think this is something we should sort out. E.g. @breedx-splk was working on a serialization within the SDK (e.g. a persistent buffer/queue) where if you naively use OTLP protos, then we actually loose information and can't abide by the specification.

[pellared]

A consumer cannot differentiate between an empty string and other empty types (array, list, object).

The question is whether actually any OTLP consumer finds it as an issue. Has nobody has reported it for years?

[LikeTheSalad]

A consumer cannot differentiate between an empty string and other empty types (array, list, object).

The question is whether actually any OTLP consumer finds it as an issue. Has nobody has reported it for years?

Just wanted to note here that the reason it was recently discovered is because of this use case, where we need to deserialize proto data, which is not an existing use case for the OTel Java SDK.

Also, I think it's important to mention that this error started to happen only after switching to the OTel Java core serialization mechanism, which is hand-rolled. Before that, the serialization mechanism used was Wire, a third-party lib to serialize/deserialize protos for Java, which didn't have this issue with empty strings. So, based on that, I think we can't discard the possibility that there might be something missing in the hand-rolled OTel Java proto serialization, or perhaps Wire uses some internal mechanisms to keep track of types. Still, it's probably a good thing to make these details clearer in the OTel spec too.

[LeadAssimilator]

A consumer cannot differentiate between an empty string and other empty types (array, list, object).

The question is whether actually any OTLP consumer finds it as an issue. Has nobody has reported it for years?

Past performance doesn't equal future results though. Just because others haven't noticed or cared enough to report it until now doesn't mean things aren't ambiguous in the spec and as a result possibly wrong in the java impl. Type information can actually be quite important to various tools when making storage decisions, querying and interpreting data for display.

I think some pertinent questions to ponder are:
Why was there a need to explicitly call out 0, empty strings and empty arrays as being meaningful?
Why wouldn't the types of attributes also be considered meaningful?
What exactly is the concept of "empty" as stated in the proto def file?
How or why should that aforementioned "empty" apply to the empty string "" which is semantically a distinct valid value of string? (i.e. since "" is a value of string, why shouldn't it always be written as "" when serialized like all other valid type values?)
How does the aforementioned "empty" apply to zero length arrays?
Can a given array be of mixed types?
Given the proto def of AnyValue is effectively a variant type in various languages, why shouldn't it follow similar definitions/patterns/conventions where both "" and zero-length arrays are stored preserving their types and are distinct from the empty/unknown types?
Does it actually make sense/is it the intent to actually use a lossy transform for the serialization format? (i.e. if loss is intended shouldn't that be as part of a separately well-defined pre-transform distinct from the serialization transform?)

I'm of the mind that since null itself is disallowed for attributes, then the concept of "empty" in the .proto for AnyValue has no real meaning and should be removed. As such, AnyValue types must then always be written with their type information which solves all of the current spec concerns in this issue.

[svrnm]

Tagged this as deciding:community-feedback since from the discussion it seems there is not yet a definitive decision on how to proceed, please update the tag accordingly when it is accepted.