Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
- Amazon OpenSearch authentication username:
- Stored in the environment variable `DD_OP_DESTINATION_AMAZON_OPENSEARCH_USERNAME`.
- The default environment variable is `DD_OP_DESTINATION_AMAZON_OPENSEARCH_USERNAME`.
- Amazon OpenSearch authentication password:
- Stored in the environment variable `DD_OP_DESTINATION_AMAZON_OPENSEARCH_PASSWORD`.
- The default environment variable is `DD_OP_DESTINATION_AMAZON_OPENSEARCH_PASSWORD`.
- Amazon OpenSearch endpoint URL:
- Stored in the environment variable `DD_OP_DESTINATION_AMAZON_OPENSEARCH_ENDPOINT_URL`.
- The default environment variable is `DD_OP_DESTINATION_AMAZON_OPENSEARCH_ENDPOINT_URL`.
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
- Amazon Security Lake TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_DESTINATION_AMAZON_SECURITY_LAKE_KEY_PASS`.
- The default environment variable is `DD_OP_DESTINATION_AMAZON_SECURITY_LAKE_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
- Google Chronicle endpoint URL:
- Stored in the environment variable `DD_OP_DESTINATION_GOOGLE_CHRONICLE_UNSTRUCTURED_ENDPOINT_URL`.
- The default environment variable is `DD_OP_DESTINATION_GOOGLE_CHRONICLE_UNSTRUCTURED_ENDPOINT_URL`.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
- CrowdStrike HEC ingestion URL:
- **Note**: Do **not** include the suffix `/services/collector` in the URL. The URL must follow this format: `https://<your_instance_id>.ingest.us-1.crowdstrike.com`.
- Stored in the environment variable `DD_OP_DESTINATION_CROWDSTRIKE_NEXT_GEN_SIEM_ENDPOINT_URL`.
- The default environment variable is `DD_OP_DESTINATION_CROWDSTRIKE_NEXT_GEN_SIEM_ENDPOINT_URL`.
- CrowdStrike HEC API token:
- Stored in the environment variable `DD_OP_DESTINATION_CROWDSTRIKE_NEXT_GEN_SIEM_TOKEN`.
- The default environment variable is `DD_OP_DESTINATION_CROWDSTRIKE_NEXT_GEN_SIEM_TOKEN`.
- CrowdStrike Next-Gen SIEM HEC TLS passphrase:
- Stored in the environment variable `DD_OP_DESTINATION_CROWDSTRIKE_NEXT_GEN_SIEM_KEY_PASS`.
- The default environment variable is `DD_OP_DESTINATION_CROWDSTRIKE_NEXT_GEN_SIEM_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#### Azure Storage

- Azure connections string to give the Worker access to your Azure Storage bucket.
- Stored in the environment variable `DD_OP_DESTINATION_DATADOG_ARCHIVES_AZURE_BLOB_CONNECTION_STRING`.
- The default environment variable is `DD_OP_DESTINATION_DATADOG_ARCHIVES_AZURE_BLOB_CONNECTION_STRING`.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
- Elasticsearch authentication username:
- Stored in the environment variable `DD_OP_DESTINATION_ELASTICSEARCH_USERNAME`.
- The default environment variable is `DD_OP_DESTINATION_ELASTICSEARCH_USERNAME`.
- Elasticsearch authentication password:
- Stored in the environment variable `DD_OP_DESTINATION_ELASTICSEARCH_PASSWORD`.
- The default environment variable is `DD_OP_DESTINATION_ELASTICSEARCH_PASSWORD`.
- Elasticsearch endpoint URL:
- Stored in the environment variable `DD_OP_DESTINATION_ELASTICSEARCH_ENDPOINT_URL`.
- The default environment variable is `DD_OP_DESTINATION_ELASTICSEARCH_ENDPOINT_URL`.
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ By default the Worker sends data to the global endpoint: `https://pubsub.googlea

If your Pub/Sub topic is region-specific, configure the Google Pub/Sub alternative endpoint URL with the regional endpoint. See [About Pub/Sub endpoints][10240] for more information.

Stored in the environment variable `DD_OP_DESTINATION_GCP_PUBSUB_ENDPOINT_URL`.
The default environment variable is `DD_OP_DESTINATION_GCP_PUBSUB_ENDPOINT_URL`.

#### TLS (when enabled)

- Google Pub/Sub TLS passphrase:
- Stored in the environment variable `DD_OP_DESTINATION_GCP_PUBSUB_KEY_PASS`.
- The default environment variable is `DD_OP_DESTINATION_GCP_PUBSUB_KEY_PASS`.

[10240]: https://cloud.google.com/pubsub/docs/reference/service_apis_overview#pubsub_endpoints
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
- HTTP/S client URI endpoint:
- Stored in the environment variable `DD_OP_DESTINATION_HTTP_CLIENT_URI`.
- The default environment variable is `DD_OP_DESTINATION_HTTP_CLIENT_URI`.
- HTTP/S Client TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_DESTINATION_HTTP_CLIENT_KEY_PASS`.
- The default environment variable is `DD_OP_DESTINATION_HTTP_CLIENT_KEY_PASS`.
- If you are using basic authentication:
- HTTP/S endpoint authentication username and password.
- Stored in the environment variable `DD_OP_DESTINATION_HTTP_CLIENT_USERNAME` and `DD_OP_DESTINATION_HTTP_CLIENT_PASSWORD`.
- The default environment variable is `DD_OP_DESTINATION_HTTP_CLIENT_USERNAME` and `DD_OP_DESTINATION_HTTP_CLIENT_PASSWORD`.
- If you are using bearer authentication:
- HTTP/S endpoint bearer token.
- Stored in the environment variable `DD_OP_DESTINATION_HTTP_CLIENT_BEARER_TOKEN`.
- The default environment variable is `DD_OP_DESTINATION_HTTP_CLIENT_BEARER_TOKEN`.
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
#### Kafka bootstrap servers
- The host and port of the Kafka bootstrap servers.
- This is the bootstrap server that the client uses to connect to the Kafka cluster and discover all the other hosts in the cluster. The host and port must be entered in the format of `host:port`, such as `10.14.22.123:9092`. If there is more than one server, use commas to separate them.
- Stored in the environment: `DD_OP_DESTINATION_KAFKA_BOOTSTRAP_SERVERS`.
- The default environment variable is `DD_OP_DESTINATION_KAFKA_BOOTSTRAP_SERVERS`.

#### TLS (when enabled)

- If TLS is enabled, the Kafka TLS passphrase is needed.
- Stored in the environment: `DD_OP_DESTINATION_KAFKA_KEY_PASS`.
- The default environment variable is `DD_OP_DESTINATION_KAFKA_KEY_PASS`.

#### SASL (when enabled)

- Kafka SASL username
- Stored in the environment: `DD_OP_DESTINATION_KAFKA_SASL_USERNAME`.
- The default environment variable is `DD_OP_DESTINATION_KAFKA_SASL_USERNAME`.
- Kafka SASL password
- Stored in the environment: `DD_OP_DESTINATION_KAFKA_SASL_PASSWORD`.
- The default environment variable is `DD_OP_DESTINATION_KAFKA_SASL_PASSWORD`.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
- Data collection endpoint (DCE)
- The DCE endpoint URL is shown as the **Logs Ingestion Endpoint** or **Data Collection Endpoint** on the DCR Overview page. An example URL: `https://<DCE-ID>.ingest.monitor.azure.com`.
- Stored in the environment variable `DD_OP_DESTINATION_MICROSOFT_SENTINEL_DCE_URI`
- The default environment variable is `DD_OP_DESTINATION_MICROSOFT_SENTINEL_DCE_URI`
- Client secret
- This is the Azure AD application's client secret, such as `550e8400-e29b-41d4-a716-446655440000`.
- Stored in the environment variable `DD_OP_DESTINATION_MICROSOFT_SENTINEL_CLIENT_SECRET`
- The default environment variable is `DD_OP_DESTINATION_MICROSOFT_SENTINEL_CLIENT_SECRET`
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
- New Relic account ID:
- Stored in the environment variable `DD_OP_DESTINATION_NEW_RELIC_ACCOUNT_ID`.
- The default environment variable is `DD_OP_DESTINATION_NEW_RELIC_ACCOUNT_ID`.
- New Relic license:
- Stored in the environment variable `DD_OP_DESTINATION_NEW_RELIC_LICENSE_KEY`.
- The default environment variable is `DD_OP_DESTINATION_NEW_RELIC_LICENSE_KEY`.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
- OpenSearch authentication username:
- Stored in the environment variable `DD_OP_DESTINATION_OPENSEARCH_USERNAME`.
- The default environment variable is `DD_OP_DESTINATION_OPENSEARCH_USERNAME`.
- OpenSearch authentication password:
- Stored in the environment variable `DD_OP_DESTINATION_OPENSEARCH_PASSWORD`.
- The default environment variable is `DD_OP_DESTINATION_OPENSEARCH_PASSWORD`.
- OpenSearch endpoint URL:
- Stored in the environment variable `DD_OP_DESTINATION_OPENSEARCH_ENDPOINT_URL`.
- The default environment variable is `DD_OP_DESTINATION_OPENSEARCH_ENDPOINT_URL`.
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
- SentinelOne write access token:
- Stored in the environment variable `DD_OP_DESTINATION_SENTINEL_ONE_TOKEN`.
- The default environment variable is `DD_OP_DESTINATION_SENTINEL_ONE_TOKEN`.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- Socket address:
- The address to which the Observability Pipelines Worker sends processed logs.
- Stored in the environment variable `DD_OP_DESTINATION_SOCKET_ADDRESS`.
- The default environment variable is `DD_OP_DESTINATION_SOCKET_ADDRESS`.
- TLS passphrase:
- Stored in the environment variable `DD_OP_DESTINATION_SOCKET_KEY_PASS`.
- The default environment variable is `DD_OP_DESTINATION_SOCKET_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
- Splunk HEC token:
- The Splunk HEC token for the Splunk indexer. **Note**: Depending on your shell and environment, you may not want to wrap your environment variable in quotes.
- Stored in the environment variable `DD_OP_DESTINATION_SPLUNK_HEC_TOKEN`.
- The default environment variable is `DD_OP_DESTINATION_SPLUNK_HEC_TOKEN`.
- Base URL of the Splunk instance:
- The Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed logs to. For example, `https://hec.splunkcloud.com:8088`.
- The Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed logs to. For example, `https://hec.splunkcloud.com:8088`.
**Note**: `/services/collector/event` path is automatically appended to the endpoint.
- Stored in the environment variable `DD_OP_DESTINATION_SPLUNK_HEC_ENDPOINT_URL`.
- The default environment variable is `DD_OP_DESTINATION_SPLUNK_HEC_ENDPOINT_URL`.
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
- The Sumo Logic HTTP Source endpoint. The Observability Pipelines Worker sends processed logs to this endpoint. For example, `https://<ENDPOINT>.collection.sumologic.com/receiver/v1/http/<UNIQUE_HTTP_COLLECTOR_CODE>`, where:
- `<ENDPOINT>` is your Sumo collection endpoint.
- `<UNIQUE_HTTP_COLLECTOR_CODE>` is the string that follows the last forward slash (`/`) in the upload URL for the HTTP source.
- Stored in the environment variable `DD_OP_DESTINATION_SUMO_LOGIC_HTTP_COLLECTOR_URL`.
- The default environment variable is `DD_OP_DESTINATION_SUMO_LOGIC_HTTP_COLLECTOR_URL`.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- The rsyslog or syslog-ng endpoint URL. For example, `127.0.0.1:9997`.
- The Observability Pipelines Worker sends logs to this address and port.
- Stored in the environment variable `DD_OP_DESTINATION_SYSLOG_ENDPOINT_URL`.
- The default environment variable is `DD_OP_DESTINATION_SYSLOG_ENDPOINT_URL`.
- The ryslog or syslog-ng TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_DESTINATION_SYSLOG_KEY_PASS`.
- The default environment variable is `DD_OP_DESTINATION_SYSLOG_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- Amazon Data Firehose address:
- The Observability Pipelines Worker listens to this socket address to receive logs from Amazon Data Firehose.
- The address is stored in the environment variable `DD_OP_SOURCE_AWS_DATA_FIREHOSE_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_AWS_DATA_FIREHOSE_ADDRESS`.
- Amazon Data Firehose TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_AWS_DATA_FIREHOSE_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_AWS_DATA_FIREHOSE_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
- Amazon S3 SQS URL:
- The URL of the SQS queue to which the S3 bucket sends the notification events.
- Stored in the environment variable: `DD_OP_SOURCE_AWS_S3_SQS_URL`
- The default environment variable is `DD_OP_SOURCE_AWS_S3_SQS_URL`
- AWS_CONFIG_FILE path:
- The path to the AWS configuration file local to this node.
- Stored in the environment variable `AWS_CONFIG_FILE`.
- The default environment variable is `AWS_CONFIG_FILE`.
- AWS_PROFILE name:
- The name of the profile to use within these files.
- Stored in the environment variable `AWS_PROFILE`.
- The default environment variable is `AWS_PROFILE`.
- AWS S3 TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_AWS_S3_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_AWS_S3_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
- Datadog Agent address:
- The Observability Pipelines Worker listens to this socket address to receive logs from the Datadog Agent.
- Stored in the environment variable `DD_OP_SOURCE_DATADOG_AGENT_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_DATADOG_AGENT_ADDRESS`.
- Datadog Agent TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_DATADOG_AGENT_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_DATADOG_AGENT_KEY_PASS`.


Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- Fluent socket address and port:
- The Observability Pipelines Worker listens on this address for incoming log messages.
- Stored in the environment variable `DD_OP_SOURCE_FLUENT_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_FLUENT_ADDRESS`.
- Fluent Bit TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_FLUENT_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_FLUENT_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
- HTTP/s endpoint URL:
- The Observability Pipelines Worker collects log events from this endpoint. For example, `https://127.0.0.8/logs`.
- Stored in the environment variable `DD_OP_SOURCE_HTTP_CLIENT_ENDPOINT_URL`.
- The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_ENDPOINT_URL`.
- HTTP/S Client TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_HTTP_CLIENT_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_KEY_PASS`.
- If you are using basic authentication:
- HTTP/S endpoint authentication username and password.
- Stored in the environment variables `DD_OP_SOURCE_HTTP_CLIENT_USERNAME` and `DD_OP_SOURCE_HTTP_CLIENT_PASSWORD`.
- The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_USERNAME` and `DD_OP_SOURCE_HTTP_CLIENT_PASSWORD`.
- If you are using bearer authentication:
- HTTP/S endpoint bearer token.
- Stored in the environment variable `DD_OP_SOURCE_HTTP_CLIENT_BEARER_TOKEN`.
- The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_BEARER_TOKEN`.
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
- HTTP/S server address:
- The Observability Pipelines Worker listens to this socket address, such as `0.0.0.0:9997`, for your HTTP client logs.
- Stored in the environment variable `DD_OP_SOURCE_HTTP_SERVER_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_HTTP_SERVER_ADDRESS`.
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
- The host and port of the Kafka bootstrap servers.
- The bootstrap server that the client uses to connect to the Kafka cluster and discover all the other hosts in the cluster. The host and port must be entered in the format of `host:port`, such as `10.14.22.123:9092`. If there is more than one server, use commas to separate them.
- Stored in the environment variable `DD_OP_SOURCE_KAFKA_BOOTSTRAP_SERVERS`.
- The default environment variable is `DD_OP_SOURCE_KAFKA_BOOTSTRAP_SERVERS`.
- SASL (when enabled):
- Kafka SASL username
- Stored in the environment variable `DD_OP_SOURCE_KAFKA_SASL_USERNAME`.
- The default environment variable is `DD_OP_SOURCE_KAFKA_SASL_USERNAME`.
- Kafka SASL password
- Stored in the environment variable `DD_OP_SOURCE_KAFKA_SASL_PASSWORD`.
- The default environment variable is `DD_OP_SOURCE_KAFKA_SASL_PASSWORD`.
- Kafka TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_KAFKA_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_KAFKA_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- Logstash address and port:
- The Observability Pipelines Worker listens on this address, such as `0.0.0.0:9997`, for incoming log messages.
- Stored in the environment variable as `DD_OP_SOURCE_LOGSTASH_ADDRESS`
- The default environment variable is `DD_OP_SOURCE_LOGSTASH_ADDRESS`
- Logstash TLS passphrase:
- Stored in the environment variable `DD_OP_SOURCE_LOGSTASH_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_LOGSTASH_KEY_PASS`.
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ You must provide both HTTP and gRPC endpoints. Configure your OTLP exporters to

- HTTP listener address
- The Observability Pipelines Worker listens to this socket address to receive logs from the OTel collector.
- Stored in the environment variable `DD_OP_SOURCE_OTEL_HTTP_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_OTEL_HTTP_ADDRESS`.

- gRPC listener address
- The Observability Pipelines Worker listens to this socket address to receive logs from the OTel collector.
- Stored in the environment variable `DD_OP_SOURCE_OTEL_GRPC_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_OTEL_GRPC_ADDRESS`.

If TLS is enabled:

- OpenTelemetry TLS passphrase
- Stored in the environment variable `DD_OP_SOURCE_OTEL_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_OTEL_KEY_PASS`.

[10230]: /observability_pipelines/sources/opentelemetry/#send-logs-to-the-observability-pipelines-worker
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
- Socket address:
- The address and port where the Observability Pipelines Worker listens for incoming logs.
- Stored in the environment variable `DD_OP_SOURCE_SOCKET_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_SOCKET_ADDRESS`.

- TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_SOCKET_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_SOCKET_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
- Splunk HEC address:
- The bind address that your Observability Pipelines Worker listens on to receive logs originally intended for the Splunk indexer. For example, `0.0.0.0:8088`
**Note**: `/services/collector/event` is automatically appended to the endpoint.
- Stored in the environment variable `DD_OP_SOURCE_SPLUNK_HEC_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_SPLUNK_HEC_ADDRESS`.
- Splunk HEC TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_SPLUNK_HEC_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_SPLUNK_HEC_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- Splunk TCP address:
- The Observability Pipelines Worker listens to this socket address to receive logs from the Splunk Forwarder. For example, `0.0.0.0:9997`.
- Stored in the environment variable `DD_OP_SOURCE_SPLUNK_TCP_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_SPLUNK_TCP_ADDRESS`.
- Splunk TCP TLS passphrase (when enabled):
- Stored in the environment variable `DD_OP_SOURCE_SPLUNK_TCP_KEY_PASS`.
- The default environment variable is `DD_OP_SOURCE_SPLUNK_TCP_KEY_PASS`.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
- Sumo Logic address:
- The bind address that your Observability Pipelines Worker listens on to receive logs originally intended for the Sumo Logic HTTP Source. For example, `0.0.0.0:80`.
**Note**: `/receiver/v1/http/` path is automatically appended to the endpoint.
- Stored in the environment variable `DD_OP_SOURCE_SUMO_LOGIC_ADDRESS`.
- The default environment variable is `DD_OP_SOURCE_SUMO_LOGIC_ADDRESS`.
Loading
Loading