-
Notifications
You must be signed in to change notification settings - Fork 9.7k
[DRAFT] Workers Tracing / Exporting OTel #24713
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
45d5bc9
WIP PR for tracing
nevikashah fecb68c
Various tweaks to tracing docs
mikenomitch 1b024ff
Update known limitations and spans documentation for tracing
nevikashah 7a50a48
Update links in tracing documentation for accuracy
nevikashah 7336a37
add sentry instructions (#25996)
AbhiPrasad b9f5829
Enhance OpenTelemetry configuration documentation
nevikashah bb429c3
Update Datadog link and improve comments in OpenTelemetry configurati…
nevikashah 09e2c06
Update spans-and-attributes.mdx (#26036)
jmorrell-cloudflare 6a1f4b8
Update Sentry and spans-and-attributes documentation for clarity
nevikashah 0fd31bc
Clarify Sentry authentication header format in OpenTelemetry document…
nevikashah 80f4ad5
Remove spans / attributes we haven't landed yet (#26099)
jmorrell-cloudflare File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
106 changes: 106 additions & 0 deletions
106
src/content/docs/workers/observability/exporting-opentelemetry-data/axiom.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| --- | ||
| pcx_content_type: how-to | ||
| title: Export to Axiom | ||
| sidebar: | ||
| order: 3 | ||
| --- | ||
|
|
||
| import {WranglerConfig} from "~/components"; | ||
|
|
||
| Axiom is a serverless log analytics platform that helps you store, search, and analyze massive amounts of data. By exporting your Cloudflare Workers application telemetry to Axiom, you can: | ||
| - Store and query logs and traces at scale | ||
| - Create dashboards and alerts to monitor your Workers | ||
|
|
||
|  | ||
|
|
||
| This guide will walk you through exporting OpenTelemetry-compliant traces and logs to Axiom from your Cloudflare Worker application | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before you begin, ensure you have: | ||
|
|
||
| - An active [Axiom account](https://app.axiom.co/register) (free tier available) | ||
| - A deployed Worker that you want to monitor | ||
| - An Axiom dataset to send data to | ||
|
|
||
| ## Step 1: Create a dataset | ||
|
|
||
| If you don't already have a dataset to send data to: | ||
| 1. Log in to your [Axiom account](https://app.axiom.co/) | ||
| 2. Navigate to **Datasets** in the left sidebar | ||
| 3. Click **New Dataset** | ||
| 4. Enter a name (e.g. `cloudflare-workers-otel`) | ||
| 5. Click **Create Dataset** | ||
|
|
||
| ## Step 2: Get your Axiom API token and dataset | ||
|
|
||
| 1. Navigate to **Settings** in the left sidebar | ||
| 2. Click on **API Tokens** | ||
| 3. Click **Create API Token** | ||
| 4. Configure your API token: | ||
| - **Name**: Enter a descriptive name (e.g., `cloudflare-workers-otel`) | ||
| - **Permissions**: Select **Ingest** permission (required for sending telemetry data) | ||
| - **Datasets**: Choose which datasets this token can write to, or select **All Datasets** | ||
| 5. Click **Create** | ||
| 6. **Important**: Copy the API token immediately and store it securely - you won't be able to see it again | ||
|
|
||
| The API token will look something like: `xaat-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx` | ||
|
|
||
| ## Step 3: Configure Cloudflare destinations | ||
|
|
||
| Now you'll create destinations in the Cloudflare dashboard that point to Axiom. | ||
|
|
||
| ### Axiom OTLP endpoints | ||
|
|
||
| Axiom provides separate OTLP endpoints for traces and logs: | ||
| - **Traces**: `https://api.axiom.co/v1/traces` | ||
| - **Logs**: `https://api.axiom.co/v1/logs` | ||
|
|
||
| ### Configure trace or logs destination | ||
|
|
||
| 1. Navigate to your Cloudflare account's [Workers Observability](https://dash.cloudflare.com/?to=/:account/workers-and-pages/observability/pipelines) section | ||
| 2. Click **Add destination** | ||
| 3. Configure your trace destination: | ||
| - **Destination Name**: `axiom-traces` (or any descriptive name) | ||
| - **Destination Type**: Select **Traces** | ||
| - **OTLP Endpoint**: `https://api.axiom.co/v1/traces` (or `/v1/logs`) | ||
| - **Custom Headers**: Add two required headers: | ||
| - Authentication header | ||
| - Header name: `Authorization` | ||
| - Header value: `Bearer <your-api-token>` | ||
| - Dataset header: | ||
| - Header name: `X-Axiom-Dataset` | ||
| - Header value: Your dataset name (e.g., `cloudflare-workers-otel`) | ||
| 4. Click **Save** | ||
|
|
||
| ## Step 3: Configure your Worker | ||
|
|
||
| With your destinations created in the Cloudflare dashboard, update your Worker's configuration to enable telemetry export. | ||
|
|
||
| <WranglerConfig> | ||
|
|
||
| ```json | ||
| { | ||
| "observability": { | ||
| "traces": { | ||
| "enabled": true, | ||
| // Must match the destination name in the dashboard | ||
| "destinations": ["axiom-traces"] | ||
| }, | ||
| "logs": { | ||
| "enabled": true, | ||
| // Must match the destination name in the dashboard | ||
| "destinations": ["axiom-logs"] | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| </WranglerConfig> | ||
|
|
||
| After updating your configuration, deploy your Worker for the changes to take effect. | ||
|
|
||
| :::note | ||
| It may take a few minutes after deployment for data to appear in Axiom. | ||
| ::: | ||
|
|
74 changes: 74 additions & 0 deletions
74
...ntent/docs/workers/observability/exporting-opentelemetry-data/grafana-cloud.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| --- | ||
| pcx_content_type: how-to | ||
| title: Export to Grafana Cloud | ||
| sidebar: | ||
| order: 2 | ||
| --- | ||
| import {WranglerConfig} from "~/components"; | ||
|
|
||
| Grafana Cloud is a fully managed observability platform that provides visualization, alerting, and analytics for your telemetry data. By exporting your Cloudflare Workers telemetry to Grafana Cloud, you can: | ||
| - Visualize distributed traces in **Grafana Tempo** to understand request flows and performance bottlenecks | ||
| - Query and analyze logs in **Grafana Loki** alongside your traces | ||
|
|
||
| This guide will walk you through configuring Cloudflare Workers to export OpenTelemetry-compliant traces and logs to your Grafana Cloud stack. | ||
|
|
||
|  | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before you begin, ensure you have: | ||
|
|
||
| - An active [Grafana Cloud account](https://grafana.com/auth/sign-up/create-user) (free tier available) | ||
| - A deployed Worker that you want to monitor | ||
|
|
||
| ## Step 1: Access the OpenTelemetry setup guide | ||
|
|
||
| 1. Log in to your [Grafana Cloud portal](https://grafana.com/) | ||
| 2. From your organization's home page, navigate to **Connections** → **Add new connection** | ||
| 3. Search for "OpenTelemetry" and select **OpenTelemetry (OTLP)** | ||
| 4. Select **Quickstart** then select **JavaScript** | ||
| 6. Click **Create a new token** | ||
| 7. Enter a name for your token (e.g., `cloudflare-workers-otel`) and click **create token** | ||
nevikashah marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| 8. Click on **Close** without copying the token | ||
| 9. Copy and Save the value for `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` in the `Environment variables` code block as the OTel endpoint and as the Auth header value respectively | ||
|
|
||
|
|
||
| ## Step 2: Set up destination | ||
| 1. Navigate to your Cloudflare account's [Workers Observability](https://dash.cloudflare.com/?to=/:account/workers-and-pages/observability/pipelines) section | ||
| 2. Click **Add destination** and configure a destination name (e.g. `grafana-tracing`) | ||
| 3. From Grafana, copy your Otel endpoint, auth header, and auth value | ||
| * Your OTEL endpoint will look like `https://otlp-gateway-prod-us-east-2.grafana.net/otlp` (append `/v1/traces` for traces and `/v1/logs` for logs) | ||
| * Your custom header should include: | ||
| * Your auth header name `Authorization` | ||
| * Your auth header value `Basic MTMxxx...` | ||
|
|
||
| ## Step 3: Configure your Worker | ||
|
|
||
| With your destination created in the Cloudflare dashboard, update your Worker's configuration to enable telemetry export. | ||
|
|
||
| <WranglerConfig> | ||
|
|
||
| ```json | ||
| { | ||
| "observability": { | ||
| "traces": { | ||
| "enabled": true, | ||
| // Must match the destination name in the dashboard | ||
| "destinations": ["grafana-traces"] | ||
| }, | ||
| "logs": { | ||
| "enabled": true, | ||
| // Must match the destination name in the dashboard | ||
| "destinations": ["grafana-logs"] | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| </WranglerConfig> | ||
|
|
||
| After updating your configuration, deploy your Worker for the changes to take effect. | ||
|
|
||
| :::note | ||
| It may take a few minutes after deployment for data to appear in Grafana Cloud. | ||
| ::: | ||
108 changes: 108 additions & 0 deletions
108
src/content/docs/workers/observability/exporting-opentelemetry-data/honeycomb.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| --- | ||
| pcx_content_type: how-to | ||
| title: Export to Honeycomb | ||
| sidebar: | ||
| order: 1 | ||
| --- | ||
|
|
||
| import {WranglerConfig} from "~/components"; | ||
|
|
||
| Honeycomb is an observability platform built for high-cardinality data that helps you understand and debug your applications. By exporting your Cloudflare Workers application telemetry to Honeycomb, you can: | ||
| - Visualize traces to understand request flows and identify performance bottlenecks | ||
| - Query and analyze logs with unlimited dimensionality across any attribute | ||
| - Create custom queries and dashboards to monitor your Workers | ||
|
|
||
|  | ||
|
|
||
| This guide will walk you through configuring your Cloudflare Worker application to export OpenTelemetry-compliant traces and logs to Honeycomb. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before you begin, ensure you have: | ||
|
|
||
| - An active [Honeycomb account](https://ui.honeycomb.io/signup) (free tier available) | ||
| - A deployed Worker that you want to monitor | ||
|
|
||
| ## Step 1: Get your Honeycomb API key | ||
|
|
||
| 1. Log in to your [Honeycomb account](https://ui.honeycomb.io/) | ||
| 2. Navigate to your account settings by clicking on your profile icon in the top right | ||
| 3. Select **Team Settings** | ||
| 4. In the left sidebar, click **Environments** and click the gear icon | ||
| 5. Find your environment (e.g., `production`, `test`) or create a new one | ||
| 6. Under **API Keys**, click **Create Ingest API Key** | ||
| 7. Configure your API key: | ||
| - **Name**: Enter a descriptive name (e.g., `cloudflare-workers-otel`) | ||
| - **Permissions**: Select **Can create services/datasets** (required for OTLP ingestion) | ||
| 8. Click **Create** | ||
| 9. **Important**: Copy the API key immediately and store it securely - you won't be able to see it again | ||
|
|
||
| The API key will look something like: `hcaik_01hq...` | ||
|
|
||
| ## Step 2: Configure Cloudflare destinations | ||
|
|
||
| Now you'll create destinations in the Cloudflare dashboard that point to Honeycomb. | ||
|
|
||
| ### Honeycomb OTLP endpoints | ||
|
|
||
| Honeycomb provides separate OTLP endpoints for traces and logs: | ||
| - **Traces**: `https://api.honeycomb.io/v1/traces` | ||
| - **Logs**: `https://api.honeycomb.io/v1/logs` | ||
|
|
||
| ### Configure trace destination | ||
|
|
||
| 1. Navigate to your Cloudflare account's [Workers Observability](https://dash.cloudflare.com/?to=/:account/workers-and-pages/observability/pipelines) section | ||
| 2. Click **Add destination** | ||
| 3. Configure your trace destination: | ||
| - **Destination Name**: `honeycomb-traces` (or any descriptive name) | ||
| - **Destination Type**: Select **Traces** | ||
| - **OTLP Endpoint**: `https://api.honeycomb.io/v1/traces` | ||
| - **Custom Headers**: Add the authentication header: | ||
| - Header name: `x-honeycomb-team` | ||
| - Header value: Your Honeycomb API key (e.g., `hcaik_01hq...`) | ||
| 4. Click **Save** | ||
|
|
||
| ### Configure logs destination | ||
|
|
||
| Repeat the process for logs: | ||
|
|
||
| 1. Click **Add destination** again | ||
| 2. Configure your logs destination: | ||
| - **Destination Name**: `honeycomb-logs` (or any descriptive name) | ||
| - **Destination Type**: Select **Logs** | ||
| - **OTLP Endpoint**: `https://api.honeycomb.io/v1/logs` | ||
| - **Custom Headers**: Add the authentication header: | ||
| - Header name: `x-honeycomb-team` | ||
| - Header value: Your Honeycomb API key (same as above) | ||
| 3. Click **Save** | ||
|
|
||
| ## Step 3: Configure your Worker | ||
|
|
||
| With your destinations created in the Cloudflare dashboard, update your Worker's configuration to enable telemetry export. | ||
|
|
||
| <WranglerConfig> | ||
|
|
||
| ```json | ||
| { | ||
| "observability": { | ||
| "traces": { | ||
| "enabled": true, | ||
| // Must match the destination name in the dashboard | ||
| "destinations": ["honeycomb-traces"] | ||
| }, | ||
| "logs": { | ||
| "enabled": true, | ||
| // Must match the destination name in the dashboard | ||
| "destinations": ["honeycomb-logs"] | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| </WranglerConfig> | ||
|
|
||
| After updating your configuration, deploy your Worker for the changes to take effect. | ||
|
|
||
| :::note | ||
| It may take a few minutes after deployment for data to appear in Honeycomb. | ||
| ::: |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this leaking Issues as a feature?