Skip to content

[DO NOT MERGE] LLM support test #32015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

[DO NOT MERGE] LLM support test #32015

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ build-cdocs:
@echo "Compiling .mdoc files to HTML";
@node ./local/bin/js/cdocs-build.js;

llm-support-demo:
@node ./assets/scripts/llm-support-demo.js;

# build .mdoc.md files, then watch for changes
watch-cdocs:
@echo "Compiling .mdoc files to HTML";
Expand Down
16 changes: 16 additions & 0 deletions assets/scripts/llm-support-demo.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
const path = require('path');
const { htmlDirToMdoc } = require('html-to-mdoc');

const inDir = path.resolve(__dirname, '../../public/opentelemetry');
const outDir = path.resolve(__dirname, '../../opentelemetry-mdoc');
console.log(`Converting HTML files in ${inDir} to .mdoc files in ${outDir}`);

htmlDirToMdoc({
inDir,
outDir,
options: {
purgeOutDirBeforeProcessing: true
}
}).then((result) => {
console.log(result.stats);
});
77 changes: 77 additions & 0 deletions opentelemetry-mdoc/compatibility/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
title: Datadog and OpenTelemetry Compatibility
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: Docs > OpenTelemetry in Datadog > Datadog and OpenTelemetry Compatibility
---

# Datadog and OpenTelemetry Compatibility

## Overview{% #overview %}

Datadog offers multiple setup options to accommodate various use cases, from full OpenTelemetry (OTel) implementations to hybrid setups using both OpenTelemetry and Datadog components. This page covers the compatibility between different setups and supported Datadog products and features, helping you choose the best configuration for your needs.

## Setups{% #setups %}

Datadog supports several configurations for using OpenTelemetry. The primary difference between these setups is the choice of SDK (OpenTelemetry or Datadog) and the collector used to process and forward telemetry data.

| Setup Type | API | SDK | Collector/Agent |
| ------------------------------------------------------------------------------------------------ | ----------------------- | ----------- | --------------------------------------------- |
| [**Datadog SDK + DDOT (Recommended)**](http://localhost:1313/opentelemetry/setup/ddot_collector) | Datadog API or OTel API | Datadog SDK | Datadog Distribution of OTel Collector (DDOT) |
| [**OTel SDK + DDOT**](http://localhost:1313/opentelemetry/setup/ddot_collector) | OTel API | OTel SDK | Datadog Distribution of OTel Collector (DDOT) |
| [**OTel SDK + OSS Collector**](http://localhost:1313/opentelemetry/collector_exporter/) | OTel API | OTel SDK | OTel Collector (OSS) |
| [**Direct OTLP Ingest**](http://localhost:1313/opentelemetry/setup/agentless) | OTel API | OTel SDK | N/A (Direct to Datadog endpoint) |

## Feature compatibility{% #feature-compatibility %}

The following table shows feature compatibility across different setups:

| Feature | Datadog SDK + DDOT (Recommended) | OTel SDK + DDOT | OTel SDK + OSS Collector | Direct OTLP Ingest |
| -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | ------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [Cloud SIEM](http://localhost:1313/security/cloud_siem/) | yes | yes | yes | yes |
| [Correlated Traces, Metrics, Logs](http://localhost:1313/opentelemetry/correlate/) | yes | yes | yes | yes |
| [Distributed Tracing](http://localhost:1313/tracing/trace_collection/) | yes | yes | yes | yes |
| [Runtime Metrics](http://localhost:1313/tracing/metrics/runtime_metrics/) | yes | yes(Java, .NET, Go only) | yes(Java, .NET, Go only) | yes(Java, .NET, Go only) |
| [Span Links](http://localhost:1313/tracing/trace_collection/span_links/) | yes | yes | yes | yes |
| [Trace Metrics](http://localhost:1313/tracing/metrics/metrics_namespace/) | yes | yes | yes | yes(Sampled (Trace metrics are calculated on the backend based on ingested spans that have passed through sampling, not on 100% of local traces before sampling.)) |
| [Database Monitoring](http://localhost:1313/opentelemetry/correlate/dbm_and_traces/) (DBM) | yes | yes | yes |
| [Cloud Network Monitoring](http://localhost:1313/network_monitoring/performance/) (CNM) | yes | yes |
| [Live Container Monitoring/Kubernetes Explorer](http://localhost:1313/containers/) | yes | yes |
| [Live Processes](http://localhost:1313/infrastructure/process/) | yes | yes |
| [Universal Service Monitoring](http://localhost:1313/universal_service_monitoring/) (USM) | yes | yes |
| [App and API Protection](http://localhost:1313/security/application_security/) (AAP) | yes |
| [Continuous Profiler](http://localhost:1313/profiler/) | yes |
| [Data Jobs Monitoring](http://localhost:1313/data_jobs/) (DJM) | yes |
| [Data Streams Monitoring](http://localhost:1313/data_streams/) (DSM) | yes | N/A (OTel does not offer DSM functionality) | N/A (OTel does not offer DSM functionality) |
| [Real User Monitoring](http://localhost:1313/opentelemetry/correlate/rum_and_traces/?tab=browserrum#opentelemetry-support) (RUM) | yes |
| [Source code integration](http://localhost:1313/integrations/guide/source-code-integration/) | yes |

## More details{% #more-details %}

### Runtime metrics{% #runtime-metrics %}

Setups using the OpenTelemetry SDK follow the [OpenTelemetry Runtime Metrics](http://localhost:1313/opentelemetry/integrations/runtime_metrics/) specification.

### Real User Monitoring (RUM){% #real-user-monitoring-rum %}

To enable full RUM functionality, you need to [inject supported headers](http://localhost:1313/real_user_monitoring/correlate_with_other_telemetry/apm/) to correlate RUM and traces.

### Cloud Network Monitoring (CNM){% #cloud-network-monitoring-cnm %}

Span-level or endpoint-level monitoring is **not** supported.

For more information, see [Cloud Network Monitoring Setup](http://localhost:1313/network_monitoring/cloud_network_monitoring/setup/).

### Source Code Integration{% #source-code-integration %}

For unsupported languages in OpenTelemetry setups, [configure telemetry tagging](http://localhost:1313/integrations/guide/source-code-integration/?tab=go#configure-telemetry-tagging) to link data to a specific commit.

## Best practices{% #best-practices %}

When using Datadog and OpenTelemetry together, Datadog recommends the following best practices to ensure optimal performance and to avoid potential issues:

- **Avoid mixed instrumentation**: Do not use both a Datadog SDK and an OpenTelemetry SDK to instrument the same application, as this leads to undefined behavior.
- **Avoid Agent and separate Collector on same host**: Do not run the Datadog Agent and a separate OpenTelemetry Collector on the same host, as this may cause issues. However, you can run Agents and Collectors on different hosts within the same fleet.

## Further reading{% #further-reading %}

- [OpenTelemetry Troubleshooting](http://localhost:1313/opentelemetry/troubleshooting/)
76 changes: 76 additions & 0 deletions opentelemetry-mdoc/config/collector_batch_memory/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: Batch and Memory Settings
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: >-
Docs > OpenTelemetry in Datadog > OpenTelemetry Configuration > Batch and
Memory Settings
---

# Batch and Memory Settings

## Overview{% #overview %}

To edit your OpenTelemetry Collector batch and memory settings, configure the [batch processor](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor) in your Datadog Exporter.

For more information, see the OpenTelemetry project documentation for the [batch processor](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor).

## Setup{% #setup %}

{% tab title="Host" %}
Add the following lines to your Collector configuration:

```yaml
processors:
batch:
# Datadog APM Intake limit is 3.2MB.
send_batch_max_size: 1000
send_batch_size: 100
timeout: 10s
memory_limiter:
check_interval: 1s
limit_mib: 1000
```

{% /tab %}

{% tab title="Kubernetes" %}
Add the following lines to `values.yaml`:

```yaml
resources:
limits:
cpu: 512m
memory: 1Gi
```

Add the following in the Collector configuration:

```yaml
processors:
batch:
# Datadog APM Intake limit is 3.2MB.
send_batch_max_size: 1000
send_batch_size: 100
timeout: 10s
```

{% /tab %}

## Data collected{% #data-collected %}

None.

## Full example configuration{% #full-example-configuration %}

For a full working example configuration with the Datadog exporter, see [`batch-memory.yaml`](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/datadogexporter/examples/batch-memory.yaml).

## Example logging output{% #example-logging-output %}

```
2023-12-05T09:52:58.568Z warn [email protected]/memorylimiter.go:276
Memory usage is above hard limit. Forcing a GC.
{"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "cur_mem_mib": 44}
2023-12-05T09:52:58.590Z info [email protected]/memorylimiter.go:266
Memory usage after GC.
{"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "cur_mem_mib": 34}
```
Loading
Loading