[EndGoal] [DoNotMerge] Extend the set of attribute value types

oteps/4485-extending-attributes-to-support-complex-values.md

@@ -94,7 +94,7 @@ extending the standard attributes provides a more seamless and user-friendly API
 
 Currently, the SDK specification has a clause that says extending
 the set of standard attribute would be
-[considered a breaking change](/specification/common/README.md#standard-attribute).
+[considered a breaking change](/specification/common/README.md#attribute).
 
 We believe that removing this clause and extending standard
 attributes can be done gracefully across the OpenTelemetry ecosystem

oteps/entities/0256-entities-data-model.md

@@ -139,7 +139,7 @@ MAY change over the lifetime of the entity. MAY be empty. These
 attributes are not part of entity's identity.
 <p>
 Follows <a
-href="../../specification/logs/data-model.md#type-any">any</a>
+href="https://github.com/open-telemetry/opentelemetry-specification/blob/v1.44.0/specification/logs/data-model.md#type-any">any</a>
 value definition in the OpenTelemetry spec - it can be a scalar value,
 byte array, an array or map of values. Arbitrary deep nesting of values
 for arrays and maps is allowed.
@@ -682,7 +682,7 @@ There are a couple of reasons:
 ### Attribute Data Type
 
 The data model requires the Attributes field to use the extended
-[any](../../specification/logs/data-model.md#type-any)
+[any](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.44.0/specification/logs/data-model.md#type-any)
 attribute values, that allows more complex data types. This is different from the data
 type used by the Id field, which is more restricted in the shape.
 

spec-compliance-matrix.md

@@ -199,7 +199,6 @@ Disclaimer: this list of features is still a work in progress, please refer to t
 | LoggerProvider.Shutdown                      |          |     | +    |     | +      |      |        | +   |      | +   | -    |       |
 | LoggerProvider.ForceFlush                    |          |     | +    |     | +      |      |        | +   |      | +   | -    |       |
 | Logger.Emit(LogRecord)                       |          |     | +    |     | +      |      |        | +   |      | +   | -    |       |
-| Reuse Standard Attributes                    | X        | +   |      |     |        |      |        |     |      |     |      |       |
 | LogRecord.Set EventName                      |          | +   |      |     |        |      |        |     | +    | +   |      |       |
 | Logger.Enabled                               | X        | +   |      |     |        |      |        |     | +    | +   |      |       |
 | SimpleLogRecordProcessor                     |          |     | +    |     | +      |      |        | +   |      | +   |      |       |

specification/common/README.md

@@ -16,7 +16,7 @@ path_base_for_github_subdir:
 <!-- toc -->
 
 - [Attribute](#attribute)
-  * [Standard Attribute](#standard-attribute)
+  * [AnyValue](#anyvalue)
   * [Attribute Limits](#attribute-limits)
     + [Configurable Parameters](#configurable-parameters)
     + [Exempt Entities](#exempt-entities)
@@ -34,30 +34,7 @@ An `Attribute` is a key-value pair, which MUST have the following properties:
 
 - The attribute key MUST be a non-`null` and non-empty string.
   - Case sensitivity of keys is preserved. Keys that differ in casing are treated as distinct keys.
-- The attribute value is either:
-  - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer.
-  - An array of primitive type values. The array MUST be homogeneous,
-    i.e., it MUST NOT contain values of different types.
-
-For protocols that do not natively support non-string values, non-string values SHOULD be represented as JSON-encoded strings.  For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will be encoded as `1.5`, and an empty array of any type will be encoded as `[]`.
-
-Attribute values expressing a numerical value of zero, an empty string, or an
-empty array are considered meaningful and MUST be stored and passed on to
-processors / exporters.
-
-Attribute values of `null` are not valid and attempting to set a `null` value is
-undefined behavior.
-
-`null` values SHOULD NOT be allowed in arrays. However, if it is impossible to
-make sure that no `null` values are accepted
-(e.g. in languages that do not have appropriate compile-time type checking),
-`null` values within arrays MUST be preserved as-is (i.e., passed on to span
-processors / exporters as `null`). If exporters do not support exporting `null`
-values, they MAY replace those values by 0, `false`, or empty strings.
-This is required for map/dictionary structures represented as two arrays with
-indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`,
-both containing an array of strings to represent a mapping
-`header_keys[i] -> header_values[i]`).
+- The attribute value MUST be one of types defined in [AnyValue](#anyvalue).
 
 Attributes are equal when their keys and values are equal.
 
@@ -68,20 +45,36 @@ See [Requirement Level](https://github.com/open-telemetry/semantic-conventions/b
 See [this document](attribute-type-mapping.md) to find out how to map values obtained
 outside OpenTelemetry into OpenTelemetry attribute values.
 
-### Standard Attribute
+### AnyValue
+
+`AnyValue` is either:
 
-Attributes are used in various places throughout the OpenTelemetry data model.
-We designate the [previous attribute section](#attribute) as the standard
-attribute definition, in order to facilitate more intuitive and consistent API /
-SDK design.
+- a primitive type: string, boolean, double precision floating point (IEEE 754-1985), or signed 64 bit integer,
+- an homogeneous array of primitive type values. An homogeneous array MUST NOT contain values of different types.
+- a byte array.
+- an heteregonous array of [AnyValue](#anyvalue),
+- an [Attribute Collection](#attribute-collections),
+- an empty value (e.g. `null`).
 
-The standard attribute definition SHOULD be used to represent attributes in data
-modeling unless there is a strong justification to diverge. For example, the Log
-Data Model has an extended [attributes](../logs/data-model.md#field-attributes)
-definition allowing values of [type `Any`](../logs/data-model.md#type-any). This
-reflects that LogRecord attributes are expected to model data produced from
-external log APIs, which do not necessarily have the same value type
-restrictions as the standard attribute definition.
+For protocols that do not natively support non-string values, non-string values SHOULD be represented as JSON-encoded strings.  For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will be encoded as `1.5`, and an empty array of any type will be encoded as `[]`.
+
+`AnyValue` expressing an empty value, a numerical value of zero,
+an empty string, or an empty array are considered meaningful and MUST be stored
+and passed on to processors / exporters.
+
+`null` values SHOULD NOT be allowed in homogeneous arrays.
+However, if it is impossible to make sure that no `null` values are accepted
+(e.g. in languages that do not have appropriate compile-time type checking),
+`null` values within homogeneous arrays MUST be preserved as-is (i.e., passed on to span
+processors / exporters as `null`). If exporters do not support exporting `null`
+values, they MAY replace those values by 0, `false`, or empty strings.
+This is required for map/dictionary structures represented as two arrays with
+indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`,
+both containing an array of strings to represent a mapping
+`header_keys[i] -> header_values[i]`).
+
+Arbitrary deep nesting of values for heteregonous arrays and attribute collections
+is allowed (essentially allows to represent an equivalent of a JSON object).
 
 ### Attribute Limits
 
@@ -148,9 +141,12 @@ at this time, as discussed in
 [Spans](../trace/api.md#set-attributes), Span
 [Events](../trace/api.md#add-events), Span
 [Links](../trace/api.md#link) and
-[Log Records](../logs/data-model.md) may contain a collection of attributes. The
-keys in each such collection are unique, i.e. there MUST NOT exist more than one
-key-value pair with the same key. The enforcement of uniqueness may be performed
+[Log Records](../logs/data-model.md),
+and even [AnyValue](#anyvalue)
+may contain a collection of attributes.
+
+Implementation MUST by default ensure that the exported attribute collections
+contain only unique keys. The enforcement of uniqueness may be performed
 in a variety of ways as it best fits the limitations of the particular
 implementation.
 
@@ -173,6 +169,12 @@ that individual attribute value being exported using a streaming wire protocol.
 In such cases the enforcement of uniqueness will likely be the responsibility of
 the recipient of this data.
 
+Implementations MAY have an option to allow exporting attribute collections
+with duplicate keys (e.g. for better performance).
+If such option is provided, it MUST be documented that for many receivers,
+handling of maps with duplicate keys is unpredictable and it is the users'
+responsibility to ensure keys are not duplicate.
+
 Collection of attributes are equal when they contain the same attributes,
 irrespective of the order in which those elements appear
 (unordered collection equality).

specification/common/attribute-type-mapping.md

@@ -13,6 +13,7 @@ linkTitle: Mapping to AnyValue
 
 - [Converting to AnyValue](#converting-to-anyvalue)
   * [Primitive Values](#primitive-values)
+    + [Empty Value](#empty-value)
     + [Integer Values](#integer-values)
     + [Enumerations](#enumerations)
     + [Floating Point Values](#floating-point-values)
@@ -54,6 +55,12 @@ follow the rules described below.
 
 ### Primitive Values
 
+#### Empty Value
+
+Empty values MUST be converted to AnyValue with no
+[value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L28-L30)
+field being set.
+
 #### Integer Values
 
 Integer values which are within the range of 64 bit signed numbers

specification/entities/data-model.md

@@ -48,8 +48,8 @@ physical format and encoding of how entity data is recorded).
 | Field        | Type                                   | Description     |
 |--------------|----------------------------------------|-----------------|
 | Type         | string                                 | Defines the type of the entity. MUST not change during the lifetime of the entity. For example: "service" or "host". This field is required and MUST not be empty for valid entities. |
-| Id           | map<string, standard attribute value>  | Attributes that identify the entity.<p>MUST not change during the lifetime of the entity. The Id must contain at least one attribute.<p>Follows OpenTelemetry [Standard attribute definition](../common/README.md#standard-attribute). SHOULD follow OpenTelemetry [semantic conventions](https://github.com/open-telemetry/semantic-conventions) for attributes. |
-| Description  | map<string, any>                       | Descriptive (non-identifying) attributes of the entity.<p>MAY change over the lifetime of the entity. MAY be empty. These attributes are not part of entity's identity.<p>Follows [any](../logs/data-model.md#type-any) value definition in the OpenTelemetry spec. Arbitrary deep nesting of values for arrays and maps is allowed.<p>SHOULD follow OpenTelemetry [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md) for attributes. |
+| Id           | map<string, attribute value>  | Attributes that identify the entity.<p>MUST not change during the lifetime of the entity. The Id must contain at least one attribute.<p>Follows OpenTelemetry [attribute definition](../common/README.md#attribute). SHOULD follow OpenTelemetry [semantic conventions](https://github.com/open-telemetry/semantic-conventions) for attributes. |
+| Description  | map<string, attribute value>                       | Descriptive (non-identifying) attributes of the entity.<p>MAY change over the lifetime of the entity. MAY be empty. These attributes are not part of entity's identity.<p>Follows OpenTelemetry [attribute definition](../common/README.md#attribute). SHOULD follow OpenTelemetry [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md) for attributes. |
 
 ## Minimally Sufficient Identity
 

specification/logs/api.md

@@ -124,14 +124,6 @@ The API MUST accept the following parameters:
 - [Attributes](./data-model.md#field-attributes) (optional)
 - [Event Name](./data-model.md#field-eventname) (optional)
 
-**Status**: [Development](../document-status.md)
-
-The API SHOULD provide functionality for users to convert
-[Standard Attributes](../common/README.md#standard-attribute)
-so they can be used, or directly accept them, in the log signal.
-This allows the reuse of [Standard Attributes](../common/README.md#standard-attribute)
-across signals.
-
 ### Enabled
 
 To help users avoid performing computationally expensive operations when

specification/logs/data-model.md

@@ -15,10 +15,6 @@ weight: 2
 - [Design Notes](#design-notes)
   * [Requirements](#requirements)
   * [Events](#events)
-  * [Definitions Used in this Document](#definitions-used-in-this-document)
-    + [Type `any`](#type-any)
-    + [Type `map`](#type-mapstring-any)
-  * [Field Kinds](#field-kinds)
 - [Log and Event Record Definition](#log-and-event-record-definition)
   * [Field: `Timestamp`](#field-timestamp)
   * [Field: `ObservedTimestamp`](#field-observedtimestamp)
@@ -112,85 +108,6 @@ conventions defined for logs SHOULD be formatted as Events. Requirements and det
 Events are intended to be used by OpenTelemetry instrumentation. It is not a
 requirement that all LogRecords are formatted as Events.
 
-### Definitions Used in this Document
-
-In this document we refer to types `any` and `map<string, any>`, defined as
-follows.
-
-#### Type `any`
-
-Value of type `any` can be one of the following:
-
-- A scalar value: string, boolean, signed 64 bit integer, or double precision floating point (IEEE 754-1985)
-
-- A byte array,
-
-- An array (a list) of `any` values,
-
-- A `map<string, any>`,
-
-- [since 1.31.0] An empty value (e.g. `null`).
-
-#### Type `map<string, any>`
-
-Value of type `map<string, any>` is a map of string keys to `any` values. The
-keys in the map are unique (duplicate keys are not allowed).
-
-Arbitrary deep nesting of values for arrays and maps is allowed (essentially
-allows to represent an equivalent of a JSON object).
-
-The representation of the map is language-dependent.
-
-The implementation MUST by default ensure that the exported maps contain only unique keys.
-
-The implementation MAY have an option to allow exporting maps with duplicate keys
-(e.g. for better performance).
-If such option is provided, it MUST be documented that for many receivers,
-handling of maps with duplicate keys is unpredictable and it is the users'
-responsibility to ensure keys are not duplicate.
-
-### Field Kinds
-
-This Data Model defines a logical model for a log record (irrespective of the
-physical format and encoding of the record). Each record contains 2 kinds of
-fields:
-
-- Named top-level fields of specific type and meaning.
-
-- Fields stored as `map<string, any>`, which can contain arbitrary values of
-  different types. The keys and values for well-known fields follow semantic
-  conventions for key names and possible values that allow all parties that work
-  with the field to have the same interpretation of the data. See references to
-  semantic conventions for `Resource` and `Attributes` fields and examples in
-  [Appendix A](./data-model-appendix.md#appendix-a-example-mappings).
-
-The reasons for having these 2 kinds of fields are:
-
-- Ability to efficiently represent named top-level fields, which are almost
-  always present (e.g. when using encodings like Protocol Buffers where fields
-  are enumerated but not named on the wire).
-
-- Ability to enforce types of named fields, which is very useful for compiled
-  languages with type checks.
-
-- Flexibility to represent less frequent data as `map<string, any>`. This
-  includes well-known data that has standardized semantics as well as arbitrary
-  custom data that the application may want to include in the logs.
-
-When designing this data model we followed the following reasoning to make a
-decision about when to use a top-level named field:
-
-- The field needs to be either mandatory for all records or be frequently
-  present in well-known log and event formats (such as `Timestamp`) or is
-  expected to be often present in log records in upcoming logging systems (such
-  as `TraceId`).
-
-- The field’s semantics must be the same for all known log and event formats and
-  can be mapped directly and unambiguously to this data model.
-
-Both of the above conditions were required to give the field a place in the
-top-level structure of the record.
-
 ## Log and Event Record Definition
 
 [Appendix A](./data-model-appendix.md#appendix-a-example-mappings) contains many examples that show how
@@ -430,14 +347,15 @@ when it is used to represent an unspecified severity.
 
 ### Field: `Body`
 
-Type: [`any`](#type-any).
+Type: [AnyValue](../common/README.md#anyvalue).
 
 Description: A value containing the body of the log record. Can be for example
 a human-readable string message (including multi-line) describing the event in
 a free form or it can be a structured data composed of arrays and maps of other
-values. Body MUST support [`any` type](#type-any) to preserve the semantics of
-structured logs emitted by the applications. Can vary for each occurrence of the
-event coming from the same source. This field is optional.
+values. Body MUST support [AnyValue](../common/README.md#anyvalue) to preserve
+the semantics of structured logs emitted by the applications.
+Can vary for each occurrence of the event coming from the same source.
+This field is optional.
 
 ### Field: `Resource`
 
@@ -464,15 +382,12 @@ they all have the same value of `InstrumentationScope`. This field is optional.
 
 ### Field: `Attributes`
 
-Type: [`map<string, any>`](#type-mapstring-any).
+Type: [Attribute Collection](../common/README.md#attribute-collections).
 
 Description: Additional information about the specific event occurrence. Unlike
 the `Resource` field, which is fixed for a particular source, `Attributes` can
 vary for each occurrence of the event coming from the same source. Can contain
 information about the request context (other than [Trace Context Fields](#trace-context-fields)).
-The log attribute model MUST support [`any` type](#type-any),
-a superset of [standard Attribute](../common/README.md#attribute),
-to preserve the semantics of structured attributes emitted by the applications.
 This field is optional.
 
 #### Errors and Exceptions

specification/resource/data-model.md

@@ -24,8 +24,8 @@ running in a container on Kubernetes, which is associated to a Pod running on a
 Node that is a VM but also is in a namespace and possibly is part of a
 Deployment. Resource could have attributes to denote information about the
 Container, the Pod, the Node, the VM or the Deployment. All of these help
-identify what produced the telemetry. Note that there are certain "standard
-attributes" that have prescribed meanings.
+identify what produced the telemetry. Note that there are certain attributes
+that have prescribed meanings.
 
 A resource is composed of 0 or more [`Entities`](../entities/README.md) and 0
 or more attributes not associated with any entity.
@@ -35,7 +35,7 @@ The data model below defines a logical model for an Resource (irrespective of th
 | Field      | Type     | Description     |
 |------------|----------|-----------------|
 | Entities   | set\<Entity\> | Defines the set of Entities associated with this resource.<p>[Entity is defined here](../entities/data-model.md) |
-| Attributes | map\<string, standard attribute value\> | Additional Attributes that identify the resource.<p>MUST not change during the lifetime of the resource.<p>Follows OpenTelemetry [Standard attribute definition](../common/README.md#standard-attribute). |
+| Attributes | map\<string, attribute value\> | Additional Attributes that identify the resource.<p>MUST not change during the lifetime of the resource.<p>Follows OpenTelemetry [attribute definition](../common/README.md#attribute). |
 
 ## Identity