Add ECS conversion tables

solutions/observability/apm/resource-attributes.md

@@ -9,19 +9,23 @@ applies_to:
 
 # Resource attributes [apm-open-telemetry-resource-attributes]
 
-A resource attribute is a key/value pair containing information about the entity producing telemetry. Resource attributes are mapped to Elastic Common Schema (ECS) fields like `service.*`, `cloud.*`, `process.*`, etc. These fields describe the service and the environment that the service runs in.
+A resource attribute is a key-value pair containing information about the entity producing telemetry. Resource attributes are mapped to Elastic Common Schema (ECS) fields like `service.*`, `cloud.*`, `process.*`, and so on. These fields describe the service and the environment that the service runs in.
 
-The examples shown here set the Elastic (ECS) `service.environment` field for the resource, i.e. service, that is producing trace events. Note that Elastic maps the OpenTelemetry `deployment.environment` field to the ECS `service.environment` field on ingestion.
+The examples set the Elastic (ECS) `service.environment` field for the resource that's producing trace events. Elastic maps the OpenTelemetry `deployment.environment` field to the ECS `service.environment` field on ingestion.
 
-**OpenTelemetry agent**
+## Setting resource attributes
 
-Use the `OTEL_RESOURCE_ATTRIBUTES` environment variable to pass resource attributes at process invocation.
+You can set resource attributes through the environment variables or by editing the configuration of the resource processor of the Collector.
+
+### OpenTelemetry configuration
+
+Use the `OTEL_RESOURCE_ATTRIBUTES` environment variable to pass resource attributes at process invocation. For example:
 
 ```bash
 export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
 ```
 
-**OpenTelemetry collector**
+### Resource processor
 
 Use the [resource processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourceprocessor) to set or apply changes to resource attributes.
 
@@ -36,13 +40,116 @@ processors:
 ...
 ```
 
-::::{tip}
-Need to add event attributes instead? Use attributes—​not to be confused with resource attributes—​to add data to span, log, or metric events. Attributes can be added as a part of the OpenTelemetry instrumentation process or with the [attributes processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/attributesprocessor).
+## OTel resource attribute to ECS field mapping
+
+The following table summarizes the mapping between OpenTelemetry resource attributes and Elastic Common Schema (ECS) fields.
+
+| OTel Attribute                          | ECS Field                               |
+|-----------------------------------------|-----------------------------------------|
+| `http.method`                           | `http.method`                           |
+| `http.url`, `http.target`, `http.path`  | `url.original`                          |
+| `http.host`                             | `host.hostname`                         |
+| `http.scheme`                           | `url.scheme`                            |
+| `http.status_code`                      | `http.response.status_code`             |
+| `http.user_agent`                       | `user_agent.original`                   |
+| `net.peer.name`, `net.peer.ip`          | `source.domain`, `source.ip`            |
+| `net.peer.port`                         | `source.port`                           |
+| `net.host.name`                         | `host.hostname`                         |
+| `net.host.port`                         | `host.port`                             |
+| `db.system`                             | `span.db.type`                          |
+| `db.name`, `db.instance`                | `span.db.instance`                      |
+| `db.statement`                          | `span.db.statement`                     |
+| `rpc.system`                            | `rpc.system`                            |
+| `rpc.service`                           | `rpc.service`                           |
+| `rpc.method`                            | `rpc.method`                            |
+| `messaging.system`                      | `span.messaging.system`                 |
+| `messaging.destination`                 | `span.messaging.destination.name`       |
+| `messaging.operation`                   | `span.messaging.operation`              |
+| `service.name`                          | `service.name`                          |
+| `service.version`                       | `service.version`                       |
+| `service.instance.id`                   | `service.instance.id`                   |
+| `exception.type`                        | `error.type`                            |
+| `exception.message`                     | `error.message`                         |
+| `exception.stacktrace`                  | `error.stacktrace`                      |
+| `data_stream.dataset`                   | `data_stream.dataset`                   |
+| `data_stream.namespace`                 | `data_stream.namespace`                 |
+
+
+### Handling of unmapped attributes
+
+When an attribute doesn't have a direct ECS field mapping, the system stores it under the `labels` namespace and replaces dots with underscores in the attribute key to comply with field name limitations.
+
+For example, if an OpenTelemetry resource contains:
+
+```json
+{
+  "service.name": "user-service",
+  "deployment.environment": "production",
+  "otel.library.name": "my-lib",
+  "custom.attribute.with.dots": "value"
+}
+```
+
+Elastic APM stores the following:
+
+```json
+{
+  "service.name": "user-service",
+  "service.environment": "production",
+  "labels": {
+    "otel_library_name": "my-lib",
+    "custom_attribute_with_dots": "value"
+  }
+}
+```
+
+### Conditional attribute translation
 
-::::
+Some OpenTelemetry attributes are conditionally converted based on their value type.
 
-% Stateful only after this?
+The following table shows how OpenTelemetry resource attributes are converted.
+
+| OTel Attribute                                      | ECS Field                                 | Comment                                                                                                                                                                                                                                                |
+|----------------------------------------------------|-------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `http.scheme`, `http.host`, `http.target`, `http.url`, `http.path`, `http.query` | `event.Url`                               | Constructs the URL by combining the scheme, host, and target attributes. Uses `http.url` if provided.                                                                                                           |
+| `http.status_code`, `http.response.status_code`    | `http.Response.StatusCode`               | Converts the status code directly into the ECS field.                                                                                                                                                                                                 |
+| `http.method`, `http.request.method`               | `http.Request.Method`                    | Maps the HTTP method to the ECS field.                                                                                                                                                                                                                 |
+| `net.peer.ip`, `net.peer.port`, `net.peer.name`    | `event.Source`                            | Sets the source IP, port, and domain. Uses `net.peer.name` as the domain and `net.peer.ip` as the IP address.                                                                                                   |
+| `net.host.name`, `net.host.port`, `server.address`, `server.port` | `event.Destination`                       | Sets the destination IP, port, and domain. Uses `net.host.name` or `server.address` as the domain.                                                                                                             |
+| `messaging.system`, `messaging.destination`, `messaging.destination.name` | `event.Transaction.Message.QueueName`    | Populates the message queue name based on these attributes.                                                                                                                                                                                           |
+| `db.system`, `db.name`, `db.statement`, `db.user`  | `event.Span.Db`                           | Converts database-related attributes into the ECS database field. For example, `db.statement` is assigned directly to `Db.Statement`.                                                                           |
+| `rpc.system`, `rpc.service`, `rpc.method`          | `event.Service.Target`                    | Maps RPC attributes, such as the system and service, to the ECS service target fields.                                                                                                                         |
+| `telemetry.sdk.elastic_export_timestamp`           | Adjusts `event.Timestamp`                 | Adjusts the timestamp of spans and transactions based on the provided export timestamp.                                                                                                                         |
+| `sampler.type`, `sampler.param`                    | `Transaction.RepresentativeCount` or `Span.RepresentativeCount` | Uses the sampling type and parameter to calculate the representative count. For probabilistic sampling, the count is calculated as `1 / probability`.                                                           |
+| `data_stream.dataset`, `data_stream.namespace`     | `event.DataStream.Dataset`, `event.DataStream.Namespace` | Sanitizes the dataset and namespace values by replacing disallowed characters and truncating them to the maximum length.                                                                                         |
+| `exception.type`, `exception.message`, `exception.stacktrace`, `exception.escaped` | `Error.Exception`                         | Maps exception attributes to the ECS error exception fields (type, message, and stacktrace).                                                                                                                   |
+| `elastic.profiler_stack_trace_ids`                 | `Transaction.ProfilerStackTraceIds`       | Converts the profiler stack trace IDs to a list.                                                                                                                                                                                                      |
+| `http.user_agent`, `user_agent.original`           | `UserAgent.Original`                      | Maps the user agent string directly to the ECS field.                                                                                                                                                                                                 |
+| `network.connection.type`, `network.connection.subtype`, `network.carrier.*` | `Network.Connection`, `Network.Carrier`   | Maps the network connection type, subtype, and carrier details to the ECS fields.                                                                                                                                                                     |
+| `span.kind`                                        | `Transaction.Type` or `Span.Type`         | Determines the transaction or span type based on the span's kind. For example, `SpanKindConsumer` results in a transaction type of `messaging`.                                                                 |
+| `peer.service`                                     | `Destination.Service.Name`                | Maps the peer service name to the destination service name.                                                                                                                                                                                            |
+| `message_bus.destination`, `messaging.temp_destination` | `Destination.Service.Resource`            | Appends the message bus destination or queue name to the service resource.                                                                                                                                                                             |
+| `elastic.is_child`, `is_child`                     | `Span.ChildIds`                           | Determines if a span link is a child span and populates the `ChildIds` field accordingly.                                                                                                                                                              |
+
+Consider the following resource attributes:
+
+```json
+{
+  "http.status_code": 200,
+  "feature.enabled": true,
+  "http.request_headers": ["accept:json", "auth:token"]
+}
+```
 
-Elastic integrates with OpenTelemetry, allowing you to reuse your existing instrumentation to easily send observability data to the {{stack}}.
+The previous resource attributes are stored by Elastic APM as follows:
 
-For more information on how to combine Elastic and OpenTelemetry, see [OpenTelemetry integration](/solutions/observability/apm/use-opentelemetry-with-apm.md).
+```json
+{
+  "http.response.status_code": 200,
+  "labels": {
+    "feature_enabled": true,
+    "http_request_headers.0": "accept:json",
+    "http_request_headers.1": "auth:token"
+  }
+}
+```

solutions/observability/apm/spans.md

@@ -475,4 +475,49 @@ Support for span compression is available in the following agents and can be con
 | **Java agent** | [`span_compression_same_kind_max_duration`](apm-agent-java://reference/config-huge-traces.md#config-span-compression-same-kind-max-duration) | [`span_compression_exact_match_max_duration`](apm-agent-java://reference/config-huge-traces.md#config-span-compression-exact-match-max-duration) |
 | **.NET agent** | [`SpanCompressionSameKindMaxDuration`](apm-agent-dotnet://reference/config-core.md#config-span-compression-exact-match-max-duration) |
 | **Node.js agent** | [`spanCompressionSameKindMaxDuration`](apm-agent-nodejs://reference/configuration.md#span-compression-exact-match-max-duration) |
-| **Python agent** | [`span_compression_same_kind_max_duration`](apm-agent-python://reference/configuration.md#config-span-compression-exact-match-max_duration) |
+| **Python agent** | [`span_compression_same_kind_max_duration`](apm-agent-python://reference/configuration.md#config-span-compression-exact-match-max_duration) |
+
+## OpenTelemetry and Elastic APM spans
+
+OpenTelemetry spans are mapped to Elastic APM transactions and spans as follows:
+
+- Root spans, such as entry points, are mapped to APM transactions.
+- Child spans, such as internal operations and DB queries, are mapped to APM spans.
+
+The following table summarizes the mapping between OpenTelemetry span kinds and Elastic APM entities.
+
+| OpenTelemetry span kind | Mapped to APM | Example |
+|-------------------------|---------------|---------|
+| `SERVER` | Transaction | Incoming HTTP request (`GET /users/{id}`) |
+| `CONSUMER` | Transaction | Message queue consumer event |
+| `CLIENT` | Span | Outgoing database query (`SELECT * FROM users`) |
+| `PRODUCER` | Span | Sending a message to a queue |
+| `INTERNAL` | Span | Internal function execution |
+
+The following example shows OpenTelemetry spans:
+
+```json
+[
+  {
+    "traceId": "abcd1234",
+    "spanId": "root5678",
+    "parentId": null,
+    "name": "GET /users/{id}",
+    "kind": "SERVER"
+  },
+  {
+    "traceId": "abcd1234",
+    "spanId": "db1234",
+    "parentId": "root5678",
+    "name": "SELECT FROM users",
+    "kind": "CLIENT"
+  }
+]
+```
+
+The previous OTel spans are stored by Elastic APM as follows:
+
+```
+Transaction: GET /users/{id}
+ ├── Span: SELECT FROM users
+```