Skip to content

OTel Correlation Guides #30354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Aug 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
67dfe22
Initial draft.
brett0000FF Jul 2, 2025
ffb9f0b
Flesh out content into second draft.
brett0000FF Jul 8, 2025
c06a960
Merge branch 'master' of github.com:DataDog/documentation into brett.…
brett0000FF Jul 8, 2025
3ee6847
Tweak examples.
brett0000FF Jul 9, 2025
409da0e
Remove vague statement.
brett0000FF Jul 9, 2025
69dc93e
Create new Data Correlation parent page with UST.
brett0000FF Jul 14, 2025
9dabb1a
Update correlate logs and traces page.
brett0000FF Jul 14, 2025
d1251d3
Add separate pages for metrics and dbm.
brett0000FF Jul 14, 2025
75e126c
Add new pages to left-nav.
brett0000FF Jul 14, 2025
ffeaaac
Merge branch 'master' of github.com:DataDog/documentation into brett.…
brett0000FF Jul 14, 2025
e0dd27d
Clean up.
brett0000FF Jul 16, 2025
4d3d48d
Merge branch 'master' of github.com:DataDog/documentation into brett.…
brett0000FF Jul 16, 2025
dd49a7b
Add example to DBM page.
brett0000FF Jul 16, 2025
e974e04
Restructure into two distinct steps with a few log pipelines.
brett0000FF Jul 17, 2025
aac4af7
Remove repetition of correlate in nav.
brett0000FF Jul 17, 2025
25dd8f0
Merge branch 'master' into brett.blue/otel-correlation
brett0000FF Jul 21, 2025
780c393
Apply suggestions from code review
brett0000FF Jul 22, 2025
52bee79
Remove redundant alert.
brett0000FF Jul 22, 2025
5ebe932
Remove misleading description of attributes.
brett0000FF Jul 22, 2025
aefb382
Clarify how trace context is injected using bridges.
brett0000FF Jul 22, 2025
fe0405e
Remove level param from LoggingHandler example.
brett0000FF Jul 22, 2025
0e504fa
Revise 'Scrape logs from files' approach.
brett0000FF Jul 22, 2025
0f3e55a
Edits.
brett0000FF Jul 22, 2025
2f31c00
Apply final feedback and clean up.
brett0000FF Jul 22, 2025
a522cc8
Merge branch 'master' of github.com:DataDog/documentation into brett.…
brett0000FF Jul 22, 2025
b5de195
Fix spacing.
brett0000FF Jul 22, 2025
e9bae95
Address final nits.
brett0000FF Jul 28, 2025
f8cdc9f
Merge branch 'master' of github.com:DataDog/documentation into brett.…
brett0000FF Jul 28, 2025
8e7da6b
Remove DBM content; move to new branch.
brett0000FF Jul 31, 2025
3f1d1c7
Merge branch 'master' of github.com:DataDog/documentation into brett.…
brett0000FF Jul 31, 2025
f795a6a
Update content/en/opentelemetry/correlate/_index.md
brett0000FF Aug 7, 2025
9133880
Merge branch 'master' into brett.blue/otel-correlation
brett0000FF Aug 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions config/_default/menus/main.en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -780,16 +780,21 @@ menu:
identifier: otel_explore
parent: opentelemetry_top_level
weight: 7
- name: Correlate Logs and Traces
- name: Logs and Traces
url: /opentelemetry/correlate/logs_and_traces/
identifier: otel_logs
parent: otel_explore
weight: 701
- name: Correlate RUM and Traces
- name: Metrics and Traces
url: /opentelemetry/correlate/metrics_and_traces/
identifier: otel_metrics_traces
parent: otel_explore
weight: 702
- name: RUM and Traces
url: /opentelemetry/correlate/rum_and_traces/
identifier: otel_rum
parent: otel_explore
weight: 702
weight: 703
- name: Integrations
url: opentelemetry/integrations/
identifier: otel_integrations
Expand Down
91 changes: 84 additions & 7 deletions content/en/opentelemetry/correlate/_index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Correlate Data
title: Correlate OpenTelemetry Data
description: Learn how to correlate your OpenTelemetry traces, metrics, logs, and other telemetry in Datadog to get a unified view of your application's performance.
aliases:
- /opentelemetry/otel_logs/
further_reading:
Expand All @@ -10,13 +11,89 @@ further_reading:

## Overview

Link your telemetry data for full-stack observability:
Getting a unified view of your application's performance requires connecting its traces, metrics, logs, user interactions, and more. By correlating your OpenTelemetry data in Datadog, you can navigate between all related telemetry in a single view, allowing you to diagnose and resolve issues faster.

{{< whatsnext desc=" " >}}
{{< nextlink href="/opentelemetry/correlate/logs_and_traces/" >}}Connect Logs and Traces{{< /nextlink >}}
{{< nextlink href="/opentelemetry/correlate/rum_and_traces/" >}}Connect RUM and Traces{{< /nextlink >}}
{{< /whatsnext >}}

## Prerequisite: Unified service tagging

Datadog uses three standard tags to link telemetry together: `env`, `service`, and `version`.

To ensure your OpenTelemetry data is properly correlated, you must configure your application or system to use these tags by setting a standard set of OpenTelemetry resource attributes. Datadog automatically maps these attributes to the correct tags.

| OpenTelemetry Resource Attribute | Datadog Tag | Notes |
|----------------------------------|-------------|---------------------------------------------------------------------------------------------------------|
| `deployment.environment.name` | `env` | **Recommended**. Supported in Agent v7.58.0+ and Collector Exporter v0.110.0+. |
| `deployment.environment` | `env` | Use instead of `deployment.environment.name` if you are running an Agent version older than v7.58.0 or a Collector Exporter older than v0.110.0. |
| `service.name` | `service` | |
| `service.version` | `version` | |

You can set these attributes in your application's environment variables, SDK, or in the OpenTelemetry Collector.

{{< tabs >}}
{{% tab "Environment Variables" %}}

Set the `OTEL_RESOURCE_ATTRIBUTES` environment variable with your service's information:

```sh
export OTEL_SERVICE_NAME="my-service"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment.name=production,service.version=1.2.3"
```

{{% /tab %}}
{{% tab "SDK" %}}

Create a Resource with the required attributes and associate it with your TracerProvider in your application code.

Here's an example using the OpenTelemetry SDK for Python:

```python
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider

resource = Resource(attributes={
"service.name": "<SERVICE>",
"deployment.environment.name": "<ENV>",
"service.version": "<VERSION>"
})
tracer_provider = TracerProvider(resource=resource)
```

{{% /tab %}}
{{% tab "Collector" %}}

Use the `resource` processor in your Collector configuration to set the resource attributes on your telemetry data:

```yaml
processors:
resource:
attributes:
- key: service.name
value: "my-service"
action: upsert
- key: deployment.environment.name
value: "production"
action: upsert
- key: service.version
value: "1.2.3"
action: upsert
...
```

{{% /tab %}}
{{< /tabs >}}

## Correlate telemetry

After unified service tagging is configured, you can connect your various telemetry streams. Select a guide below for platform-specific instructions.

- [Correlate logs and traces][1]
- [Correlate metrics and traces][2]
- [Correlate RUM and traces][3]

## Further reading

{{< partial name="whats-next/whats-next.html" >}}
{{< partial name="whats-next/whats-next.html" >}}

[1]: /opentelemetry/correlate/logs_and_traces
[2]: /opentelemetry/correlate/metrics_and_traces
[3]: /opentelemetry/correlate/rum_and_traces
79 changes: 79 additions & 0 deletions content/en/opentelemetry/correlate/metrics_and_traces.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
title: Correlate OpenTelemetry Traces and Metrics
further_reading:
- link: "/opentelemetry/otel_tracing/"
tag: "Documentation"
text: "Send OpenTelemetry Traces to Datadog"
---

## Overview

Correlating traces with host metrics allows you to pivot from a slow request directly to the CPU and memory metrics of the host or container it ran on. This helps you determine if resource contention was the root cause of a performance issue.

Correlation between traces and metrics relies on the following resource attributes:

- `host.name`: For correlating with host metrics (CPU, memory, disk).
- `container.id`: For correlating with container metrics.

## Requirements

Before you begin, ensure you have configured [unified service tagging][1]. This is required for all data correlation in Datadog.

## Setup

To correlate traces and metrics, you must:

1. **Collect Host Metrics**: You must have the OpenTelemetry Collector configured to collect and send host metrics to Datadog.

2. **Ensure Consistent Tagging**: Your traces and metrics must share a consistent `host.name` (for hosts) or `container.id` (for containers) attribute for Datadog to link them.


### 1. Collect host metrics

To collect system-level metrics from your infrastructure, enable the `hostmetrics` receiver in your OpenTelemetry Collector configuration. This receiver gathers metrics like CPU, memory, disk, and network usage.

Add the `hostmetrics` receiver to the `receivers` section of your Collector configuration and enable it in your `metrics` pipeline:


```yaml
receivers:
hostmetrics:
collection_interval: 10s
scrapers:
cpu:
memory:
disk:
...

service:
pipelines:
metrics:
receivers: [hostmetrics, ...]
processors: [...]
exporters: [...]
```

For the complete, working configuration, including Kubernetes-specific setup, see the [Host Metrics][2] documentation.

### 2. Ensure consistent host and container tagging

For correlation to work, the `host.name` (or `container.id`) attribute on your traces must match the corresponding attribute on the metrics collected by the `hostmetrics` receiver.

## View correlated data in Datadog

After your application is sending traces and the Collector is sending host metrics, you can see the correlation in the APM Trace View.

1. Navigate to [**APM** > **Traces**][3].
2. Find and click on a trace from your instrumented service.
3. In the trace's flame graph, select a span that ran on the instrumented host.
4. In the details panel, click the **Infrastructure** tab. You should see the host metrics, like CPU and memory utilization, from the host that executed that part of the request.

This allows you to immediately determine if a spike in host metrics corresponds with the performance of a specific request.

## Further reading

{{< partial name="whats-next/whats-next.html" >}}

[1]: /opentelemetry/correlate/#prerequisite-unified-service-tagging
[2]: /opentelemetry/integrations/host_metrics
[3]: https://app.datadoghq.com/apm/traces
4 changes: 2 additions & 2 deletions content/en/opentelemetry/integrations/host_metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The metrics, mapped to Datadog metrics, are used in the following views:
- [Host default dashboards][8]
- [APM Trace view Host info][9]

**Note**: To correlate trace and host metrics, configure [Universal Service Monitoring attributes][10] for each service, and set the `host.name` resource attribute to the corresponding underlying host for both service and collector instances.
**Note**: To correlate trace and host metrics, configure [Unified Service Tagging attributes][10] for each service, and set the `host.name` resource attribute to the corresponding underlying host for both service and collector instances.

The following table shows which Datadog host metric names are associated with corresponding OpenTelemetry host metric names, and, if applicable, what math is applied to the OTel host metric to transform it to Datadog units during the mapping.

Expand Down Expand Up @@ -168,6 +168,6 @@ Value: 1153183744
[7]: https://app.datadoghq.com/infrastructure
[8]: /opentelemetry/collector_exporter/#out-of-the-box-dashboards
[9]: /tracing/trace_explorer/trace_view/?tab=hostinfo
[10]: /universal_service_monitoring/setup/
[10]: /opentelemetry/correlate/#prerequisite-unified-service-tagging


Loading
Loading