From 5f4a90c4424cba61abf21d497cd545ef8aafa4f2 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Wed, 22 Oct 2025 14:28:37 -0400 Subject: [PATCH 1/9] Removing a readme file this is not standardized in classic-mode directory. Signed-off-by: Eric D. Schabell --- administration/configuring-fluent-bit/classic-mode/README.md | 1 - 1 file changed, 1 deletion(-) delete mode 100644 administration/configuring-fluent-bit/classic-mode/README.md diff --git a/administration/configuring-fluent-bit/classic-mode/README.md b/administration/configuring-fluent-bit/classic-mode/README.md deleted file mode 100644 index e4db3b567..000000000 --- a/administration/configuring-fluent-bit/classic-mode/README.md +++ /dev/null @@ -1 +0,0 @@ -# Classic mode From ca437dc7286a2b0ce96a7f2fe362aca473716c05 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Wed, 22 Oct 2025 14:29:22 -0400 Subject: [PATCH 2/9] Updated to not use readme file in classic-config directory, but the correct description file. Signed-off-by: Eric D. Schabell --- administration/configuring-fluent-bit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/administration/configuring-fluent-bit.md b/administration/configuring-fluent-bit.md index 8fa4042a1..7da8a1473 100644 --- a/administration/configuring-fluent-bit.md +++ b/administration/configuring-fluent-bit.md @@ -3,7 +3,7 @@ Fluent Bit supports two configuration formats: - [YAML](./configuring-fluent-bit/yaml.md): Standard configuration format as of v3.2. -- [Classic mode](./configuring-fluent-bit/classic-mode/README.md): To be deprecated at the end of 2026. +- [Classic mode](./configuring-fluent-bit/classic-mode.md): To be deprecated at the end of 2026. ## Command line interface From 5bbc9561019320ed081d1bad29c75bdcd6074307 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Wed, 22 Oct 2025 14:30:07 -0400 Subject: [PATCH 3/9] Updated to better sort some sections and update for missing entries. Fixes #2108. Signed-off-by: Eric D. Schabell --- SUMMARY.md | 68 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 35 insertions(+), 33 deletions(-) diff --git a/SUMMARY.md b/SUMMARY.md index 0cc20d535..6f3eeaad9 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -25,16 +25,16 @@ * [Build with static configuration](installation/downloads/source/build-with-static-configuration.md) * [Linux packages](installation/downloads/linux.md) * [Amazon Linux](installation/downloads/linux/amazon-linux.md) - * [Rocky Linux and Alma Linux ](installation/downloads/linux/alma-rocky.md) - * [Red Hat and CentOS](installation/downloads/linux/redhat-centos.md) + * [Buildroot embedded Linux](installation/downloads/linux/buildroot-embedded-linux.md) * [Debian](installation/downloads/linux/debian.md) - * [Ubuntu](installation/downloads/linux/ubuntu.md) * [Raspbian and Raspberry Pi](installation/downloads/linux/raspbian-raspberry-pi.md) + * [Red Hat and CentOS](installation/downloads/linux/redhat-centos.md) + * [Rocky Linux and Alma Linux ](installation/downloads/linux/alma-rocky.md) + * [Ubuntu](installation/downloads/linux/ubuntu.md) * [Yocto embedded Linux](installation/downloads/linux/yocto-embedded-linux.md) - * [Buildroot embedded Linux](installation/downloads/linux/buildroot-embedded-linux.md) - * [Docker](installation/downloads/docker.md) - * [Containers on AWS](installation/downloads/aws-container.md) * [Amazon EC2](installation/downloads/amazon-ec2.md) + * [Containers on AWS](installation/downloads/aws-container.md) + * [Docker](installation/downloads/docker.md) * [Kubernetes](installation/downloads/kubernetes.md) * [macOS](installation/downloads/macos.md) * [Windows](installation/downloads/windows.md) @@ -44,36 +44,37 @@ * [Configure Fluent Bit](administration/configuring-fluent-bit.md) * [YAML configuration](administration/configuring-fluent-bit/yaml.md) + * [Configuration file](administration/configuring-fluent-bit/yaml/configuration-file.md) + * [Environment variables](administration/configuring-fluent-bit/yaml/environment-variables-section.md) + * [Includes](administration/configuring-fluent-bit/yaml/includes-section.md) * [Service](administration/configuring-fluent-bit/yaml/service-section.md) * [Parsers](administration/configuring-fluent-bit/yaml/parsers-section.md) * [Multiline parsers](administration/configuring-fluent-bit/yaml/multiline-parsers-section.md) * [Pipeline](administration/configuring-fluent-bit/yaml/pipeline-section.md) * [Plugins](administration/configuring-fluent-bit/yaml/plugins-section.md) * [Upstream servers](administration/configuring-fluent-bit/yaml/upstream-servers-section.md) - * [Environment variables](administration/configuring-fluent-bit/yaml/environment-variables-section.md) - * [Includes](administration/configuring-fluent-bit/yaml/includes-section.md) * [Classic mode](administration/configuring-fluent-bit/classic-mode.md) - * [Format and schema](administration/configuring-fluent-bit/classic-mode/format-schema.md) - * [Configuration file](administration/configuring-fluent-bit/classic-mode/configuration-file.md) - * [Variables](administration/configuring-fluent-bit/classic-mode/variables.md) + * [Configuration file](administration/configuring-fluent-bit/classic-mode/configuration-file.md) * [Commands](administration/configuring-fluent-bit/classic-mode/commands.md) + * [Format and schema](administration/configuring-fluent-bit/classic-mode/format-schema.md) + * [Record accessor syntax](administration/configuring-fluent-bit/classic-mode/record-accessor.md) * [Upstream servers](administration/configuring-fluent-bit/classic-mode/upstream-servers.md) - * [Record accessor syntax](administration/configuring-fluent-bit/classic-mode/record-accessor.md) - * [Unit Sizes](administration/configuring-fluent-bit/unit-sizes.md) + * [Variables](administration/configuring-fluent-bit/classic-mode/variables.md) * [Multiline parsing](administration/configuring-fluent-bit/multiline-parsing.md) -* [TLS](administration/transport-security.md) -* [Buffering and storage](administration/buffering-and-storage.md) + * [Unit Sizes](administration/configuring-fluent-bit/unit-sizes.md) +* [AWS credentials](administration/aws-credentials.md) * [Backpressure](administration/backpressure.md) -* [Scheduling and retries](administration/scheduling-and-retries.md) -* [Networking](administration/networking.md) +* [Buffering and storage](administration/buffering-and-storage.md) +* [Hot reload](administration/hot-reload.md) +* [HTTP proxy](administration/http-proxy.md) * [Memory management](administration/memory-management.md) * [Monitoring](administration/monitoring.md) * [Multithreading](administration/multithreading.md) -* [HTTP proxy](administration/http-proxy.md) -* [Hot reload](administration/hot-reload.md) -* [Troubleshooting](administration/troubleshooting.md) +* [Networking](administration/networking.md) * [Performance tips](administration/performance.md) -* [AWS credentials](administration/aws-credentials.md) +* [Scheduling and retries](administration/scheduling-and-retries.md) +* [TLS](administration/transport-security.md) +* [Troubleshooting](administration/troubleshooting.md) ## Local testing @@ -91,8 +92,8 @@ * [Dummy](pipeline/inputs/dummy.md) * [eBPF](pipeline/inputs/ebpf.md) * [Elasticsearch](pipeline/inputs/elasticsearch.md) - * [Exec WASI](pipeline/inputs/exec-wasi.md) * [Exec](pipeline/inputs/exec.md) + * [Exec WASI](pipeline/inputs/exec-wasi.md) * [Fluent Bit metrics](pipeline/inputs/fluentbit-metrics.md) * [Forward](pipeline/inputs/forward.md) * [Head](pipeline/inputs/head.md) @@ -124,25 +125,25 @@ * [TCP](pipeline/inputs/tcp.md) * [Thermal](pipeline/inputs/thermal.md) * [UDP](pipeline/inputs/udp.md) - * [Windows Event logs (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md) * [Windows Event logs (winlog)](pipeline/inputs/windows-event-log.md) + * [Windows Event logs (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md) * [Windows exporter metrics](pipeline/inputs/windows-exporter-metrics.md) * [Parsers](pipeline/parsers.md) * [Configuring parsers](pipeline/parsers/configuring-parser.md) + * [Decoders](pipeline/parsers/decoders.md) * [JSON](pipeline/parsers/json.md) - * [Regular expression](pipeline/parsers/regular-expression.md) - * [LTSV](pipeline/parsers/ltsv.md) * [Logfmt](pipeline/parsers/logfmt.md) - * [Decoders](pipeline/parsers/decoders.md) + * [LTSV](pipeline/parsers/ltsv.md) + * [Regular expression](pipeline/parsers/regular-expression.md) * [Processors](pipeline/processors.md) + * [Conditional processing](pipeline/processors/conditional-processing.md) * [Content modifier](pipeline/processors/content-modifier.md) + * [Filters as processors](pipeline/processors/filters.md) * [Labels](pipeline/processors/labels.md) * [Metrics selector](pipeline/processors/metrics-selector.md) * [OpenTelemetry envelope](pipeline/processors/opentelemetry-envelope.md) * [Sampling](pipeline/processors/sampling.md) * [SQL](pipeline/processors/sql.md) - * [Filters as processors](pipeline/processors/filters.md) - * [Conditional processing](pipeline/processors/conditional-processing.md) * [Filters](pipeline/filters.md) * [AWS metadata](pipeline/filters/aws-metadata.md) * [CheckList](pipeline/filters/checklist.md) @@ -169,15 +170,15 @@ * [Buffering](pipeline/buffering.md) * [Routing](pipeline/router.md) * [Outputs](pipeline/outputs.md) + * [Azure Log Analytics](pipeline/outputs/azure.md) + * [Azure Blob](pipeline/outputs/azure_blob.md) + * [Azure Data Explorer](pipeline/outputs/azure_kusto.md) + * [Azure Logs Ingestion API](pipeline/outputs/azure_logs_ingestion.md) * [Amazon CloudWatch](pipeline/outputs/cloudwatch.md) * [Amazon Kinesis Data Firehose](pipeline/outputs/firehose.md) * [Amazon Kinesis Data Streams](pipeline/outputs/kinesis.md) * [Amazon S3](pipeline/outputs/s3.md) * [Apache SkyWalking](pipeline/outputs/skywalking.md) - * [Azure Blob](pipeline/outputs/azure_blob.md) - * [Azure Data Explorer](pipeline/outputs/azure_kusto.md) - * [Azure Log Analytics](pipeline/outputs/azure.md) - * [Azure Logs Ingestion API](pipeline/outputs/azure_logs_ingestion.md) * [Counter](pipeline/outputs/counter.md) * [Dash0](pipeline/outputs/dash0.md) * [Datadog](pipeline/outputs/datadog.md) @@ -199,10 +200,10 @@ * [New Relic](pipeline/outputs/new-relic.md) * [Null](pipeline/outputs/null.md) * [Observe](pipeline/outputs/observe.md) + * [Oracle Cloud Infrastructure Logging Analytics](pipeline/outputs/oci-logging-analytics.md) * [OpenObserve](pipeline/outputs/openobserve.md) * [OpenSearch](pipeline/outputs/opensearch.md) * [OpenTelemetry](pipeline/outputs/opentelemetry.md) - * [Oracle Cloud Infrastructure Logging Analytics](pipeline/outputs/oci-logging-analytics.md) * [Output Formats](pipeline/outputs/output_formats.md) * [Parseable](pipeline/outputs/parseable.md) * [PostgreSQL](pipeline/outputs/postgresql.md) @@ -234,6 +235,7 @@ * [C library API](development/library_api.md) * [Ingest records manually](development/ingest-records-manually.md) * [Golang output plugins](development/golang-output-plugins.md) +* [MessagePack format](development/msgpack-format.md) * [Wasm filter plugins](development/wasm-filter-plugins.md) * [Wasm input plugins](development/wasm-input-plugins.md) * [Contribution guide for beginners](development/developer-guide.md) From e4ea1ccc78f3c23d2356ba8d1a34e1f2688ae995 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Thu, 23 Oct 2025 15:09:44 -0400 Subject: [PATCH 4/9] Updated configuration variables for opentelemetry output plugin, many new ones added. Signed-off-by: Eric D. Schabell --- pipeline/outputs/opentelemetry.md | 126 ++++++++++++++++++++---------- 1 file changed, 83 insertions(+), 43 deletions(-) diff --git a/pipeline/outputs/opentelemetry.md b/pipeline/outputs/opentelemetry.md index ecc09cc63..4cc103048 100644 --- a/pipeline/outputs/opentelemetry.md +++ b/pipeline/outputs/opentelemetry.md @@ -1,54 +1,93 @@ ---- -description: An output plugin to submit Logs, Metrics, or Traces to an OpenTelemetry endpoint ---- +— -# OpenTelemetry +== description: An output plugin to submit Logs, Metrics, or Traces to an OpenTelemetry endpoint + += OpenTelemetry The OpenTelemetry plugin lets you take logs, metrics, and traces from Fluent Bit and submit them to an OpenTelemetry HTTP endpoint. Only HTTP endpoints are supported. -| Key | Description | Default | -|-----|-------------|---------| -| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` | -| `http_user` | Basic Auth username. | _none_ | -| `http_passwd` | Basic Auth password. Requires `HTTP_user` to be set. | _none_ | -| `port` | TCP port of the target HTTP server. | `80` | -| `proxy` | Specify an HTTP Proxy. The expected format value is `http://HOST:PORT`. HTTPS isn't supported. It's recommended to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | _none_ | -| `http2` | Defines whether HTTP/2 protocol is enabled. This setting also supports the `force` option, which forces HTTP/2 over a plain text connection. | `On` | -| `grpc` | Enables gRPC over an HTTP/2 connection. This setting applies to HTTP/2 only. | `off` | -| `metrics_uri` | Specify an optional HTTP URI for the target web server listening for metrics. For example, `/v1/metrics`. | `/` | -| `logs_uri` | Specify an optional HTTP URI for the target web server listening for logs, For example, `/v1/logs`. | `/` | -| `traces_uri` | Specify an optional HTTP URI for the target web server listening for traces, For example, `/v1/traces`. | `/` | -| `header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ | -| `log_response_payload` | Log the response payload within the Fluent Bit log. | `false` | -| `logs_body_key` | The log body key to look up in the log events body/message. Sets the `Body` field of the OpenTelemetry logs data model. | `message` | -| `logs_trace_id_message_key` | The trace id key to look up in the log events body/message. Sets the `TraceId` field of the OpenTelemetry logs data model. | `traceId` | -| `logs_span_id_message_key` | The span id key to look up in the log events body/message. Sets the `SpanId` field of the OpenTelemetry logs data model. | `spanId` | -| `logs_severity_text_message_key` | The severity text id key to look up in the log events body/message. Sets the `SeverityText` field of the OpenTelemetry logs data model. | `severityText` | -| `logs_severity_number_message_key` | The severity number id key to look up in the log events body/message. Sets the `SeverityNumber` field of the OpenTelemetry logs data model. | `severityNumber` | -| `add_label` | Lets you add custom labels to all metrics and logs (resource attributes) exposed through the OpenTelemetry exporter. You can have multiple `add_label` fields for each label you want to add. | _none_ | -| `compress` | Set payload compression mechanism. Allowed value: `gzip`. | _none_ | -| `logs_observed_timestamp_metadata_key` | Specify an `ObservedTimestamp` key to look up in the metadata. | `$ObservedKey` | -| `logs_timestamp_metadata_key` | Specify a `Timestamp` key to look up in the metadata. | `$Timestamp` | -| `logs_severity_key_metadata_key` | Specify a `SeverityText` key to look up in the metadata. | `$SeverityText` | -| `logs_severity_number_metadata_key` | Specify a `SeverityNumber` key to look up in the metadata. | `$SeverityNumber` | -| `logs_trace_flags_metadata_key` | Specify a `Flags` key to look up in the metadata. | `$Flags` | -| `logs_span_id_metadata_key` | Specify a `SpanId` key to look up in the metadata. | `$SpanId` | -| `logs_trace_id_metadata_key` | Specify a `TraceId` key to look up in the metadata. | `$TraceId` | -| `logs_attributes_metadata_key` | Specify an `Attributes` key to look up in the metadata. | `$Attributes` | -| `logs_body_key_attributes` | When enabled (`true`), Fluent Bit will place all parsed JSON fields (except the one mapped by `logs_body_key`) inside the `attributes` section of the OpenTelemetry `LogRecord`. This is useful if you want structured key/value pairs as attributes, while still mapping a specific field (defined by `logs_body_key`) to the `body`. | `false` | -| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | - -## Get started + +| Key | Description | Default | +|-------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------| +| `add_label` | Lets you add custom labels to all metrics and logs (resource attributes) exposed through the OpenTelemetry exporter. You can have multiple `add_label` fields for each label you want to add. | _none_ | +| `alias` | Sets an alias for the output instance. This is useful when using multiple instances of the same output plugin. If no alias is set, the instance will be named using the plugin name and a sequence number. | _none_ | +| `aws_auth` | Enable AWS SigV4 authentication. | `false` | +| `aws_external_id` | Specify an external ID for the STS API, can be used with the `aws_role_arn` parameter if your role requires an external ID. | _none_ | +| `aws_profile` | AWS Profile name. AWS Profiles can be configured with AWS CLI and are usually stored in $HOME/.aws/ directory. | _none_ | +| `aws_region` | AWS region of your service. | _none_ | +| `aws_role_arn` | ARN of an IAM role to assume (ex. for cross account access). | _none_ | +| `aws_service` | AWS destination service code, used by SigV4 authentication. | `logs` | +| `aws_sts_endpoint` | Custom endpoint for the AWS STS API, used with the `aws_role_arn` option. | _none_ | +| `batch_size` | Set the maximum number of log records to be flushed at a time. | `1000` | +| `compress` | Set payload compression mechanism. Options available are `gzip` and `zstd`. | _none_ | +| `grpc` | Enables gRPC over an HTTP/2 connection. Supports `auto` option. This setting applies to HTTP/2 only. | `off` | +| `grpc_metrics_uri` | Specify an optional gRPC URI for the target OTel endpoint. | `/opentelemetry.proto.collector.metrics.v1.MetricsService/Export` | +| `grpc_logs_uri` | Specify an optional gRPCß URI for the target OTel endpoint. | `/opentelemetry.proto.collector.logs.v1.LogsService/Export` | +| `grpc_traces_uri` | Specify an optional gRPC URI for the target OTel endpoint. | `/opentelemetry.proto.collector.trace.v1.TraceService/Export` | +| `grpc_profiles_uri` | Specify an optional gRPC URI for the profiles OTel endpoint. | `/opentelemetry.proto.collector.profiles.v1experimental.ProfilesService/Export` | +| `header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ | +| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` | +| `http2` | Defines whether HTTP/2 protocol is enabled. This setting also supports the `force` option, which forces HTTP/2 over a plain text connection. | `off` | +| `http_passwd` | Basic Auth password. Requires `HTTP_user` to be set. | _none_ | +| `http_user` | Basic Auth username. | _none_ | +| `logs_attributes_metadata_key` | Specify an `Attributes` key to look up in the metadata. | `$Attributes` | +| `log_level` | Specifies the log level for this output plugin. If not set, the plugin will use the global log level defined in the `service` section. If the global log level is also not specified, it uses default. | `info` | +| `log_suppress_interval` | Allows suppression of repetitive log messages from the output plugin that appear similar within a specified time interval. Default is no suppression. | `0` | +| `logs_body_key` | The log body key to look up in the log events body/message. Sets the `Body` field of the OpenTelemetry logs data model. | `message` | +| `logs_body_key_attributes` | When enabled (`true`), Fluent Bit will place all parsed JSON fields (except the one mapped by `logs_body_key`) inside the `attributes` section of the OpenTelemetry `LogRecord`. This is useful if you want structured key/value pairs as attributes, while still mapping a specific field (defined by `logs_body_key`) to the `body`. | `false` | +| `logs_instrumentation_scope_metadata_key` | Specify an `InstrumentationScope` key. | `InstrumentationScope` | +| `logs_metadata_key` | Set the key to lookup in the metadata. | `otlp` | +| `logs_observed_timestamp_metadata_key` | Specify an `ObservedTimestamp` key to look up in the metadata. | `$ObservedKey` | +| `logs_resource_metadata_key` | Specify a `Resource` key. | `Resource` | +| `log_response_payload` | Log the response payload within the Fluent Bit log. | `false` | +| `log_severity_text_metadata_key` | Specify a SeverityText key. | `$SeverityText` | +| `logs_severity_number_metadata_key` | Specify a `SeverityNumber` key to look up in the metadata. | `$SeverityNumber` | +| `logs_severity_number_message_key` | The severity number id key to look up in the log events body/message. Sets the `SeverityNumber` field of the OpenTelemetry logs data model. | `$severityNumber` | +| `logs_severity_text_message_key` | The severity text id key to look up in the log events body/message. Sets the `SeverityText` field of the OpenTelemetry logs data model. | `$SeverityText` | +| `logs_span_id_message_key` | The span id key to look up in the log events body/message. Sets the `SpanId` field of the OpenTelemetry logs data model. | `$SpanId` | +| `logs_span_id_metadata_key` | Specify a `SpanId` key to look up in the metadata. | `$SpanId` | +| `logs_timestamp_metadata_key` | Specify a `Timestamp` key to look up in the metadata. | `$Timestamp` | +| `logs_trace_flags_metadata_key` | Specify a `TraceFlags` key to look up in the metadata. | `$TraceFlags` | +| `logs_trace_id_message_key` | The trace id key to look up in the log events body/message. Sets the `TraceId` field of the OpenTelemetry logs data model. | `$TraceId` | +| `logs_trace_id_metadata_key` | Specify a `TraceId` key to look up in the metadata. | `$TraceId` | +| `logs_uri` | Specify an optional HTTP URI for the target web server listening for logs, For example, `/v1/logs`. | _none_ | +| `match` | Set a tag pattern to match the records that this output should process. Supports exact matches or wildcards (for example '*'). | `default` | +| `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to simple wildcards. | `default` | +| `metrics_uri` | Specify an optional HTTP URI for the target web server listening for metrics. For example, `/v1/metrics`. | `/` | +| `net.dns.mode` | Select the primary DNS connection type (TCP or UDP). | _none_ | +| `port` | TCP port of the target HTTP server. | `80` | +| `profiles_uri` | Specify an optional HTTP URI for the profiles OTel endpoint. | `/v1development/profiles` | +| `proxy` | Specify an HTTP Proxy. The expected format value is `http://HOST:PORT`. HTTPS isn't supported. It's recommended to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | _none_ | +| `retry_limit` | Set the retry limit for the output plugin when delivery fails. Accepted values: a positive integer, `no_limits`, `false`, or `off` to disable retry limits, or `no_retries` to disable retries entirely. | `1` | +| `tls` | Enable or disable TLS/SSL support. | `off` | +| `tls.ca_file` | Absolute path to CA certificate file. | `default` | +| `tls.ca_path` | Absolute path to scan for certificate files. | `default` | +| `tls.cert_file` | Absolute path to Certificate file. | `default` | +| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | `default` | +| `tls.debug` | Set TLS debug verbosity level. It accept the following values: `0` (No debug), `1`(Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` | +| `tls.key_file` | Absolute path to private Key file. | `default` | +| `tls.key_passwd` | Optional password for tls.key_file file. | `default` | +| `tls.max_version` | Specify the maximum version of TLS. | `default` | +| `tls.min_version` | Specify the minimum version of TLS. | `default` | +| `tls.verify` | Force certificate validation. | `on` | +| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` | +| `tls.vhost` | Hostname to be used for TLS SNI extension. | `default` | +| `tls.windows.certstore_name` | Sets the certstore name on an output (Windows). | `default` | +| `tls.windows.use_enterprise_store` | Sets whether using enterprise certstore or not on an output (Windows). | `default` | +| `traces_uri` | Specify an optional HTTP URI for the target web server listening for traces, For example, `/v1/traces`. | `/` | + + +== Get started The OpenTelemetry plugin works with logs and only the metrics collected from one of the metric input plugins. In the following example, log records generated by the dummy plugin and the host metrics collected by the node exporter metrics plugin are exported by the OpenTelemetry output plugin. {% tabs %} {% tab title="fluent-bit.yaml" %} -```yaml - +[source,yaml] +---- # Dummy Logs and traces with Node Exporter Metrics export using OpenTelemetry output plugin # ------------------------------------------- # The following example collects host metrics on Linux and dummy logs and traces and delivers @@ -73,7 +112,7 @@ pipeline: outputs: - name: opentelemetry - match: '*' + match: "*" host: localhost port: 443 metrics_uri: /v1/metrics @@ -91,12 +130,13 @@ pipeline: add_label: - app fluent-bit - color blue -``` +---- {% endtab %} {% tab title="fluent-bit.conf" %} -```text +[source,text] +---- # Dummy Logs and traces with Node Exporter Metrics export using OpenTelemetry output plugin # ------------------------------------------- # The following example collects host metrics on Linux and dummy logs and traces and delivers @@ -139,7 +179,7 @@ pipeline: # add user-defined labels add_label app fluent-bit add_label color blue -``` +---- {% endtab %} {% endtabs %} From f8ff78d500939668019e0ff977aeefd02caf9590 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Thu, 23 Oct 2025 15:21:45 -0400 Subject: [PATCH 5/9] Updated configuration variables for opentelemetry output plugin, many new ones added. Fixes #2083 Signed-off-by: Eric D. Schabell --- pipeline/outputs/opentelemetry.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/pipeline/outputs/opentelemetry.md b/pipeline/outputs/opentelemetry.md index 4cc103048..f0dae7dfc 100644 --- a/pipeline/outputs/opentelemetry.md +++ b/pipeline/outputs/opentelemetry.md @@ -56,7 +56,20 @@ Only HTTP endpoints are supported. | `match` | Set a tag pattern to match the records that this output should process. Supports exact matches or wildcards (for example '*'). | `default` | | `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to simple wildcards. | `default` | | `metrics_uri` | Specify an optional HTTP URI for the target web server listening for metrics. For example, `/v1/metrics`. | `/` | +| `net.connect_timeout` | Set maximum time allowed to establish a connection, this time includes the TLS handshake. | `10s` | +| `net.connect_timeout_log_error` | On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. | `true` | +| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it is retried. | `2000` | | `net.dns.mode` | Select the primary DNS connection type (TCP or UDP). | _none_ | +| `net.dns.prefer_ipv4` | Select the primary DNS resolver type (LEGACY or ASYNC). | _none_ | +| `net.dns.prefer_ipv6` | Prioritize IPv6 DNS results when trying to establish a connection. | _none_ | +| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` | +| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` | +| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` | +| `net.source_address` | Specify network address to bind for data traffic. | _none_ | +| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` | +| `net.tcp_keepalive_time` | Interval between the last data packet sent and the first TCP keepalive probe. | `-1` | +| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a keepidle probe. | `-1` | +| `net.tcp_keepalive_probes` | Number of unacknowledged probes to consider a connection dead. | `-1` | | `port` | TCP port of the target HTTP server. | `80` | | `profiles_uri` | Specify an optional HTTP URI for the profiles OTel endpoint. | `/v1development/profiles` | | `proxy` | Specify an HTTP Proxy. The expected format value is `http://HOST:PORT`. HTTPS isn't supported. It's recommended to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | _none_ | From 8b007c047fe9c68693474f57655fcc9703587274 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Fri, 24 Oct 2025 14:55:19 -0400 Subject: [PATCH 6/9] Fixes reqeusted by reviewers for options table for opentelemetry output plugin. Signed-off-by: Eric D. Schabell --- pipeline/outputs/opentelemetry.md | 165 +++++++++++++++--------------- 1 file changed, 83 insertions(+), 82 deletions(-) diff --git a/pipeline/outputs/opentelemetry.md b/pipeline/outputs/opentelemetry.md index f0dae7dfc..f45be2bb1 100644 --- a/pipeline/outputs/opentelemetry.md +++ b/pipeline/outputs/opentelemetry.md @@ -9,88 +9,89 @@ The OpenTelemetry plugin lets you take logs, metrics, and traces from Fluent Bit Only HTTP endpoints are supported. -| Key | Description | Default | -|-------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------| -| `add_label` | Lets you add custom labels to all metrics and logs (resource attributes) exposed through the OpenTelemetry exporter. You can have multiple `add_label` fields for each label you want to add. | _none_ | -| `alias` | Sets an alias for the output instance. This is useful when using multiple instances of the same output plugin. If no alias is set, the instance will be named using the plugin name and a sequence number. | _none_ | -| `aws_auth` | Enable AWS SigV4 authentication. | `false` | -| `aws_external_id` | Specify an external ID for the STS API, can be used with the `aws_role_arn` parameter if your role requires an external ID. | _none_ | -| `aws_profile` | AWS Profile name. AWS Profiles can be configured with AWS CLI and are usually stored in $HOME/.aws/ directory. | _none_ | -| `aws_region` | AWS region of your service. | _none_ | -| `aws_role_arn` | ARN of an IAM role to assume (ex. for cross account access). | _none_ | -| `aws_service` | AWS destination service code, used by SigV4 authentication. | `logs` | -| `aws_sts_endpoint` | Custom endpoint for the AWS STS API, used with the `aws_role_arn` option. | _none_ | -| `batch_size` | Set the maximum number of log records to be flushed at a time. | `1000` | -| `compress` | Set payload compression mechanism. Options available are `gzip` and `zstd`. | _none_ | -| `grpc` | Enables gRPC over an HTTP/2 connection. Supports `auto` option. This setting applies to HTTP/2 only. | `off` | -| `grpc_metrics_uri` | Specify an optional gRPC URI for the target OTel endpoint. | `/opentelemetry.proto.collector.metrics.v1.MetricsService/Export` | -| `grpc_logs_uri` | Specify an optional gRPCß URI for the target OTel endpoint. | `/opentelemetry.proto.collector.logs.v1.LogsService/Export` | -| `grpc_traces_uri` | Specify an optional gRPC URI for the target OTel endpoint. | `/opentelemetry.proto.collector.trace.v1.TraceService/Export` | -| `grpc_profiles_uri` | Specify an optional gRPC URI for the profiles OTel endpoint. | `/opentelemetry.proto.collector.profiles.v1experimental.ProfilesService/Export` | -| `header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ | -| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` | -| `http2` | Defines whether HTTP/2 protocol is enabled. This setting also supports the `force` option, which forces HTTP/2 over a plain text connection. | `off` | -| `http_passwd` | Basic Auth password. Requires `HTTP_user` to be set. | _none_ | -| `http_user` | Basic Auth username. | _none_ | -| `logs_attributes_metadata_key` | Specify an `Attributes` key to look up in the metadata. | `$Attributes` | -| `log_level` | Specifies the log level for this output plugin. If not set, the plugin will use the global log level defined in the `service` section. If the global log level is also not specified, it uses default. | `info` | -| `log_suppress_interval` | Allows suppression of repetitive log messages from the output plugin that appear similar within a specified time interval. Default is no suppression. | `0` | -| `logs_body_key` | The log body key to look up in the log events body/message. Sets the `Body` field of the OpenTelemetry logs data model. | `message` | -| `logs_body_key_attributes` | When enabled (`true`), Fluent Bit will place all parsed JSON fields (except the one mapped by `logs_body_key`) inside the `attributes` section of the OpenTelemetry `LogRecord`. This is useful if you want structured key/value pairs as attributes, while still mapping a specific field (defined by `logs_body_key`) to the `body`. | `false` | -| `logs_instrumentation_scope_metadata_key` | Specify an `InstrumentationScope` key. | `InstrumentationScope` | -| `logs_metadata_key` | Set the key to lookup in the metadata. | `otlp` | -| `logs_observed_timestamp_metadata_key` | Specify an `ObservedTimestamp` key to look up in the metadata. | `$ObservedKey` | -| `logs_resource_metadata_key` | Specify a `Resource` key. | `Resource` | -| `log_response_payload` | Log the response payload within the Fluent Bit log. | `false` | -| `log_severity_text_metadata_key` | Specify a SeverityText key. | `$SeverityText` | -| `logs_severity_number_metadata_key` | Specify a `SeverityNumber` key to look up in the metadata. | `$SeverityNumber` | -| `logs_severity_number_message_key` | The severity number id key to look up in the log events body/message. Sets the `SeverityNumber` field of the OpenTelemetry logs data model. | `$severityNumber` | -| `logs_severity_text_message_key` | The severity text id key to look up in the log events body/message. Sets the `SeverityText` field of the OpenTelemetry logs data model. | `$SeverityText` | -| `logs_span_id_message_key` | The span id key to look up in the log events body/message. Sets the `SpanId` field of the OpenTelemetry logs data model. | `$SpanId` | -| `logs_span_id_metadata_key` | Specify a `SpanId` key to look up in the metadata. | `$SpanId` | -| `logs_timestamp_metadata_key` | Specify a `Timestamp` key to look up in the metadata. | `$Timestamp` | -| `logs_trace_flags_metadata_key` | Specify a `TraceFlags` key to look up in the metadata. | `$TraceFlags` | -| `logs_trace_id_message_key` | The trace id key to look up in the log events body/message. Sets the `TraceId` field of the OpenTelemetry logs data model. | `$TraceId` | -| `logs_trace_id_metadata_key` | Specify a `TraceId` key to look up in the metadata. | `$TraceId` | -| `logs_uri` | Specify an optional HTTP URI for the target web server listening for logs, For example, `/v1/logs`. | _none_ | -| `match` | Set a tag pattern to match the records that this output should process. Supports exact matches or wildcards (for example '*'). | `default` | -| `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to simple wildcards. | `default` | -| `metrics_uri` | Specify an optional HTTP URI for the target web server listening for metrics. For example, `/v1/metrics`. | `/` | -| `net.connect_timeout` | Set maximum time allowed to establish a connection, this time includes the TLS handshake. | `10s` | -| `net.connect_timeout_log_error` | On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. | `true` | -| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it is retried. | `2000` | -| `net.dns.mode` | Select the primary DNS connection type (TCP or UDP). | _none_ | -| `net.dns.prefer_ipv4` | Select the primary DNS resolver type (LEGACY or ASYNC). | _none_ | -| `net.dns.prefer_ipv6` | Prioritize IPv6 DNS results when trying to establish a connection. | _none_ | -| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` | -| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` | -| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` | -| `net.source_address` | Specify network address to bind for data traffic. | _none_ | -| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` | -| `net.tcp_keepalive_time` | Interval between the last data packet sent and the first TCP keepalive probe. | `-1` | -| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a keepidle probe. | `-1` | -| `net.tcp_keepalive_probes` | Number of unacknowledged probes to consider a connection dead. | `-1` | -| `port` | TCP port of the target HTTP server. | `80` | -| `profiles_uri` | Specify an optional HTTP URI for the profiles OTel endpoint. | `/v1development/profiles` | -| `proxy` | Specify an HTTP Proxy. The expected format value is `http://HOST:PORT`. HTTPS isn't supported. It's recommended to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | _none_ | -| `retry_limit` | Set the retry limit for the output plugin when delivery fails. Accepted values: a positive integer, `no_limits`, `false`, or `off` to disable retry limits, or `no_retries` to disable retries entirely. | `1` | -| `tls` | Enable or disable TLS/SSL support. | `off` | -| `tls.ca_file` | Absolute path to CA certificate file. | `default` | -| `tls.ca_path` | Absolute path to scan for certificate files. | `default` | -| `tls.cert_file` | Absolute path to Certificate file. | `default` | -| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | `default` | -| `tls.debug` | Set TLS debug verbosity level. It accept the following values: `0` (No debug), `1`(Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` | -| `tls.key_file` | Absolute path to private Key file. | `default` | -| `tls.key_passwd` | Optional password for tls.key_file file. | `default` | -| `tls.max_version` | Specify the maximum version of TLS. | `default` | -| `tls.min_version` | Specify the minimum version of TLS. | `default` | -| `tls.verify` | Force certificate validation. | `on` | -| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` | -| `tls.vhost` | Hostname to be used for TLS SNI extension. | `default` | -| `tls.windows.certstore_name` | Sets the certstore name on an output (Windows). | `default` | -| `tls.windows.use_enterprise_store` | Sets whether using enterprise certstore or not on an output (Windows). | `default` | -| `traces_uri` | Specify an optional HTTP URI for the target web server listening for traces, For example, `/v1/traces`. | `/` | - +| Key | Description | Default | +|-------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------| +| `add_label` | Adds a custom label to the metrics use format: `add_label name value`. | _none_ | +| `alias` | Sets an alias, useful for multiple instances of the same output plugin. | _none_ | +| `aws_auth` | Enable AWS SigV4 authentication. | `false` | +| `aws_external_id` | Specify an external ID for the STS API, can be used with the `aws_role_arn` parameter. | _none_ | +| `aws_profile` | AWS Profile name. AWS Profiles can be configured with AWS CLI. | _none_ | +| `aws_region` | AWS region of your service. | _none_ | +| `aws_role_arn` | ARN of an IAM role to assume (ex. for cross account access). | _none_ | +| `aws_service` | AWS destination service code, used by SigV4 authentication. | `logs` | +| `aws_sts_endpoint` | Custom endpoint for the AWS STS API, used with the `aws_role_arn` option. | _none_ | +| `grpc` | Enable, disable or force gRPC usage. Accepted values : `on`, `off`, `auto`. | `off` | +| `batch_size` | Set the maximum number of log records to be flushed at a time. | `1000` | +| `compress` | Set payload compression mechanism. Options available are `gzip` and `zstd`. | _none_ | +| `grpc_logs_uri` | Specify an optional gRPCß URI for the target OTel endpoint. | `/opentelemetry.proto.collector.logs.v1.LogsService/Export` | +| `grpc_metrics_uri` | Specify an optional gRPC URI for the target OTel endpoint. | `/opentelemetry.proto.collector.metrics.v1.MetricsService/Export` | +| `grpc_profiles_uri` | Specify an optional gRPC URI for profiles OTel endpoint. | `/opentelemetry.proto.collector.profiles.v1experimental.ProfilesService/Export` | +| `grpc_traces_uri` | Specify an optional gRPC URI for the target OTel endpoint. | `/opentelemetry.proto.collector.trace.v1.TraceService/Export` | +| `header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ | +| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` | +| `http2` | Enable, disable or force HTTP/2 usage. Accepted values : `on`, `off`, or `force`. | `off` | +| `http_passwd` | Set HTTP auth password. | _none_ | +| `http_user` | Set HTTP auth user. | _none_ | +| `log_level` | Specifies the log level for output plugin. If not set here, plugin uses global log level in `service` section. | `info` | +| `logs_attributes_metadata_key` | Specify an `Attributes` key. | `$Attributes` | +| `logs_body_key` | Specify an optional HTTP URI for the target OTel endpoint. | _none_ | +| `logs_body_key_attributes` | If set and it matched a pattern, it includes the remaining fields in the record as attributes. | `false` | +| `logs_instrumentation_scope_metadata_key` | Specify an `InstrumentationScope` key. | `InstrumentationScope` | +| `logs_observed_timestamp_metadata_key` | Specify an `ObservedTimestamp` key. | `$ObservedTimestamp` | +| `logs_max_resources` | Set the maximum number of OTLP log resources per export request (`0` disables the limit). | `0` | +| `logs_max_scopes` | Set the maximum number of OTLP log scopes per resource (`0` disables the limit). | `0` | +| `logs_metadata_key` | Set the key to lookup in the metadata. | `otlp` | +| `logs_resource_metadata_key` | Specify a `Resource` key. | `Resource` | +| `log_response_payload` | Specify if the response payload should be logged or not. | `true` | +| `logs_severity_number_message_key` | Specify a `SeverityNumber` key. | `$severityNumber` | +| `logs_severity_number_metadata_key` | Specify a `SeverityNumber` key. | `$SeverityNumber` | +| `logs_severity_text_message_key` | Specify a `SeverityText` key. | `$SeverityText` | +| `log_severity_text_metadata_key` | Specify a `SeverityText` key. | `$SeverityText` | +| `log_supress_interval` | Suppresses log messages from output plugin that appear similar within a specified time interval. `0` no suppression. | `0` | +| `logs_span_id_message_key` | Specify a `SpanId` key. | `$SpanId` | +| `logs_span_id_metadata_key` | Specify a `SpanId` key. | `$SpanId` | +| `logs_timestamp_metadata_key` | Specify a `Timestamp` key. | `$Timestamp` | +| `logs_trace_flags_metadata_key` | Specify a `TraceFlags` key. | `$TraceFlags` | +| `logs_trace_id_message_key` | Specify a `TraceId` key. | `$TraceId` | +| `logs_trace_id_metadata_key` | Specify a `TraceId` key. | `$TraceId` | +| `logs_uri` | Specify an optional HTTP URI for the target OTel endpoint. | `/v1/logs` | +| `match` | Set a tag pattern to match records that output should process. Exact matches or wildcards (for example `*`). | _none_ | +| `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to simple wildcards. | _none_ | +| `metrics_uri` | Specify an optional HTTP URI for the target OTel endpoint. | `/v1/metrics` | +| `net.connect_timeout` | Set maximum time allowed to establish a connection, this time includes the TLS handshake. | `10s` | +| `net.connect_timeout_log_error` | On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. | `true` | +| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it is retried. | `2000` | +| `net.dns.mode` | Select the primary DNS connection type (TCP or UDP). | _none_ | +| `net.dns.prefer_ipv4` | Select the primary DNS resolver type (LEGACY or ASYNC). | _none_ | +| `net.dns.prefer_ipv6` | Prioritize IPv6 DNS results when trying to establish a connection. | _none_ | +| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` | +| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` | +| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` | +| `net.source_address` | Specify network address to bind for data traffic. | _none_ | +| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` | +| `net.tcp_keepalive_time` | Interval between the last data packet sent and the first TCP keepalive probe. | `-1` | +| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a keepidle probe. | `-1` | +| `net.tcp_keepalive_probes` | Number of unacknowledged probes to consider a connection dead. | `-1` | +| `port` | TCP port of the target HTTP server. | `80` | +| `profiles_uri` | Specify an optional HTTP URI for the profiles OTel endpoint. | `/v1development/profiles` | +| `proxy` | Specify an HTTP Proxy. The expected format of this value is `http://host:port`. | _none_ | +| `retry_limit` | Set retry limit for output plugin when delivery fails. Integer, `no_limits`, `false`, or `off` to disable, or `no_retries` to disable retries entirely. | `1` | +| `tls` | Enable or disable TLS/SSL support. | `off` | +| `tls.ca_file` | Absolute path to CA certificate file. | _none_ | +| `tls.ca_path` | Absolute path to scan for certificate files. | _none_ | +| `tls.cert_file` | Absolute path to Certificate file. | _none_ | +| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | _none_ | +| `tls.debug` | Set TLS debug level. Accepts `0` (No debug), `1`(Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` | +| `tls.key_file` | Absolute path to private Key file. | _none_ | +| `tls.key_passwd` | Optional password for tls.key_file file. | _none_ | +| `tls.max_version` | Specify the maximum version of TLS. | _none_ | +| `tls.min_version` | Specify the minimum version of TLS. | _none_ | +| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` | +| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ | +| `tls.verify` | Force certificate validation. | `on` | +| `tls.windows.certstore_name` | Sets the certstore name on an output (Windows). | _none_ | +| `tls.windows.use_enterprise_store` | Sets whether using enterprise certstore or not on an output (Windows). | _none_ | +| `traces_uri` | Specify an optional HTTP URI for the target OTel endpoint. | `/v1/traces` | == Get started From 05132376ea237fa9aaf9c88c703ab13bd64f8c07 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 27 Oct 2025 13:59:42 -0400 Subject: [PATCH 7/9] Update config table for cloudwatch output plugin to match v4.1 release and added add_entity. Fixes #2085. Signed-off-by: Eric D. Schabell --- pipeline/outputs/cloudwatch.md | 53 ++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 22 deletions(-) diff --git a/pipeline/outputs/cloudwatch.md b/pipeline/outputs/cloudwatch.md index 9fe7904f8..b8110b987 100644 --- a/pipeline/outputs/cloudwatch.md +++ b/pipeline/outputs/cloudwatch.md @@ -14,28 +14,37 @@ See [AWS credentials](https://docs.fluentbit.io/manual/administration/aws-creden ## Configuration parameters -| Key | Description | -|----- |------------------ | -| `region` | The AWS region. | -| `log_group_name` | The name of the CloudWatch log group that you want log records sent to. | -| `log_group_template` | Optional. Template for the log group name using Fluent Bit [`record_accessor`](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/record-accessor) syntax. If configured, this field overrides the `log_group_name`. If the template translation fails, an error is logged and the `log_group_name` is used instead. See the tutorial for an example. | -| `log_stream_name` | The name of the CloudWatch log stream that you want log records sent to. | -| `log_stream_prefix` | Prefix for the log stream name. The tag is appended to the prefix to construct the full log stream name. Not compatible with the `log_stream_name` option. | -| `log_stream_template` | Optional. Template for log stream name using Fluent Bit [`record_accessor`](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/record-accessor) syntax. If configured, it overrides other log stream options. If the template translation fails, an error is logged and `log_stream_name` or `log_stream_prefix` are used instead (one of those fields must be configured). See the tutorial for an example. | -| `log_key` | By default, the whole log record will be sent to CloudWatch. If you specify a key name with this option, then only the value of that key will be sent to CloudWatch. For example, if you are using the Fluentd Docker log driver, you can specify `log_key log` and only the log message will be sent to CloudWatch. | -| `log_format` | Optional. A parameter that can be used to tell CloudWatch the format of the data. A value of `json/emf` enables CloudWatch to extract custom metrics embedded in a JSON payload. See the [Embedded Metric Format](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html) specification. | -| `role_arn` | ARN of an IAM role to assume for cross account access. | -| `auto_create_group` | Automatically create the log group. Allowed values: `true`, `false` (case insensitive). Defaults to `false`. | -| `log_group_class` | Optional. Specifies the log storage class for new log groups when `auto_create_group` is set to `true`. You can't modify the storage class of existing log groups. Allowed values: `STANDARD`, `INFREQUENT_ACCESS`. Default: `STANDARD`. | -| `log_retention_days` | If set to a number greater than zero, and newly create log group's retention policy is set to this many days. Allowed values: [`1`, `3`, `5`, `7`, `14`, `30`, `60`, `90`, `120`, `150`, `180`, `365`, `400`, `545`, `731`, `1827`, `3653`] | -| `endpoint` | Specify a custom endpoint for the CloudWatch Logs API. | -| `metric_namespace` | An optional string representing the CloudWatch namespace for the metrics. See the [Metrics tutorial](#metrics-tutorial) section for a full configuration. | -| `metric_dimensions` | A list of lists containing the dimension keys that will be applied to all metrics. The values within a dimension set must be members on the root-node. For more information about dimensions, see [Dimension](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_Dimension.html) and [Dimensions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Dimension). In the Fluent Bit configurations, `metric_dimensions` is a comma and semicolon separated string. If you have only one list of dimensions, put the values as a comma separated string. If you want to put list of lists, use the list as semicolon separated strings. For example, if you set the value as `dimension_1,dimension_2;dimension_3`, Fluent Bit converts it as `[[dimension_1, dimension_2],[dimension_3]]`. | -| `sts_endpoint` | Specify a custom STS endpoint for the AWS STS API. | -| `profile` | Option to specify an AWS Profile for credentials. Defaults to `default`. | -| `auto_retry_requests` | Immediately retry failed requests to AWS services once. This option doesn't affect the normal Fluent Bit retry mechanism with backoff. Instead, it enables an immediate retry with no delay for networking errors, which can help improve throughput when there are transient/random networking issues. Defaults to `true`. | -| `external_id` | Specify an external ID for the STS API, can be used with the `role_arn` parameter if your role requires an external ID. | -| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. Default: `1`. | +| Key | Description | Default | +|------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------| +| `add_enity` | Add entity to `PutLogEvent` calls. | `false` | +| `alias` | Sets an alias, useful for multiple instances of the same output plugin. | _none_ | +| `auto_create_group` | Automatically create the log group. Allowed values: `true`, `false` (case insensitive). | `false` | +| `auto_retry_requests` | Immediately retry failed requests to AWS services once. This option doesn't affect the normal Fluent Bit retry mechanism with backoff. Instead, it enables an immediate retry with no delay for networking errors, which can help improve throughput when there are transient/random networking issues. | `true` | +| `endpoint` | Specify a custom endpoint for the CloudWatch Logs API. | _none_ | +| `external_id` | Specify an external ID for the STS API, can be used with the `role_arn` parameter if your role requires an external ID. | _none_ | +| `extra_user_agent` | This option appends a string to the default user agent. AWS asks that you not manually set this field yourself, it is reserved for use in our vended configurations, for example, EKS Container Insights. | _none_ | +| `log_format` | An optional parameter that can be used to tell CloudWatch the format of the data. A value of `json/emf` enables CloudWatch to extract custom metrics embedded in a `JSON` payload. | _none_ | +| `log_group_class` | Specifies the log storage class for new log groups when `auto_create_group` is set to `true`. You can't modify the storage class of existing log groups. Allowed values: `STANDARD`, `INFREQUENT_ACCESS`. | `STANDARD` | +| `log_group_name` | The name of the CloudWatch log group that you want log records sent to. | _none_ | +| `log_group_template` | OTemplate for CW Log Group name using record accessor syntax. Plugin falls back to the log_group_name configured if needed. | _none_ | +| `log_key` | By default, the whole log record will be sent to CloudWatch. If you specify a key name with this option, then only the value of that key will be sent to CloudWatch. For example, if you are using the Fluentd Docker log driver, you can specify `log_key log` and only the log message will be sent to CloudWatch. | _none_ | +| `log_level` | Specifies the log level for output plugin. If not set here, plugin uses global log level in `service` section. | `info` | +| `log_retention_days` | If set to a number greater than zero, and newly create log group's retention policy is set to this many days. Allowed values: [`1`, `3`, `5`, `7`, `14`, `30`, `60`, `90`, `120`, `150`, `180`, `365`, `400`, `545`, `731`, `1827`, `3653`] | `0` | +| `log_stream_name` | The name of the CloudWatch log stream that you want log records sent to. | _none_ | +| `log_stream_prefix` | Prefix for the log stream name. The tag is appended to the prefix to construct the full log stream name. Not compatible with the `log_stream_name` option. | _none_ | +| `log_stream_template` | Template for CloudWatch Log Stream name using record accessor syntax. Plugin falls back to the `log_stream_name` or `log_stream_prefix` configured if needed. | _none_ | +| `log_supress_interval` | Suppresses log messages from output plugin that appear similar within a specified time interval. `0` no suppression. | `0` | +| `match` | Set a tag pattern to match records that output should process. Exact matches or wildcards (for example `*`). | _none_ | +| `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to simple wildcards. | _none_ | +| `metric_dimensions` | Metric dimensions is a list of lists. If you have only one list of dimensions, put the values as a comma seperated string. If you want to put list of lists, use the list as semicolon seperated strings. If your value is `d1,d2;d3`, we will consider it as `[[d1, d2],[d3]]`. | _none_ | +| `metric_namespace` | An optional string representing the CloudWatch namespace for the metrics. See the [Metrics tutorial](#metrics-tutorial) section for a full configuration. | _none_ | +| `profile` | Option to specify an AWS Profile for credentials. | _none_ | +| `region` | The AWS region to send logs to. | _none_ | +| `retry_limit` | Set retry limit for output plugin when delivery fails. Integer, `no_limits`, `false`, or `off` to disable, or `no_retries` to disable retries entirely. | `1` | +| `role_arn` | ARN of an IAM role to assume for cross account access. | _none_ | +| `sts_endpoint` | Specify a custom STS endpoint for the AWS STS API. | _none_ | +| `tls.windows.certstore_name` | Sets the certstore name on an output (Windows). | _none_ | +| `tls.windows.use_enterprise_store` | Sets whether using enterprise certstore or not on an output (Windows). | _none_ | ## Get started From e5d64b43e7a0628fc7e381273b5888facbbfb2eb Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 27 Oct 2025 14:04:19 -0400 Subject: [PATCH 8/9] Fixing vale issue. Signed-off-by: Eric D. Schabell --- pipeline/outputs/cloudwatch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/cloudwatch.md b/pipeline/outputs/cloudwatch.md index b8110b987..50bbafec4 100644 --- a/pipeline/outputs/cloudwatch.md +++ b/pipeline/outputs/cloudwatch.md @@ -36,7 +36,7 @@ See [AWS credentials](https://docs.fluentbit.io/manual/administration/aws-creden | `log_supress_interval` | Suppresses log messages from output plugin that appear similar within a specified time interval. `0` no suppression. | `0` | | `match` | Set a tag pattern to match records that output should process. Exact matches or wildcards (for example `*`). | _none_ | | `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to simple wildcards. | _none_ | -| `metric_dimensions` | Metric dimensions is a list of lists. If you have only one list of dimensions, put the values as a comma seperated string. If you want to put list of lists, use the list as semicolon seperated strings. If your value is `d1,d2;d3`, we will consider it as `[[d1, d2],[d3]]`. | _none_ | +| `metric_dimensions` | Metric dimensions is a list of lists. If you have only one list of dimensions, put the values as a comma seperated string. If you want to put list of lists, use the list as semicolon seperated strings. If your value is `d1,d2;d3`, it will consider it as `[[d1, d2],[d3]]`. | _none_ | | `metric_namespace` | An optional string representing the CloudWatch namespace for the metrics. See the [Metrics tutorial](#metrics-tutorial) section for a full configuration. | _none_ | | `profile` | Option to specify an AWS Profile for credentials. | _none_ | | `region` | The AWS region to send logs to. | _none_ | From 482dc34297882fed5ae674e4213733834e299212 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 27 Oct 2025 14:06:48 -0400 Subject: [PATCH 9/9] Fixing vale issue. Signed-off-by: Eric D. Schabell --- pipeline/outputs/cloudwatch.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pipeline/outputs/cloudwatch.md b/pipeline/outputs/cloudwatch.md index 50bbafec4..f6a681756 100644 --- a/pipeline/outputs/cloudwatch.md +++ b/pipeline/outputs/cloudwatch.md @@ -43,8 +43,8 @@ See [AWS credentials](https://docs.fluentbit.io/manual/administration/aws-creden | `retry_limit` | Set retry limit for output plugin when delivery fails. Integer, `no_limits`, `false`, or `off` to disable, or `no_retries` to disable retries entirely. | `1` | | `role_arn` | ARN of an IAM role to assume for cross account access. | _none_ | | `sts_endpoint` | Specify a custom STS endpoint for the AWS STS API. | _none_ | -| `tls.windows.certstore_name` | Sets the certstore name on an output (Windows). | _none_ | -| `tls.windows.use_enterprise_store` | Sets whether using enterprise certstore or not on an output (Windows). | _none_ | +| `tls.windows.certstore_name` | Sets the certificate store name on an output (Windows). | _none_ | +| `tls.windows.use_enterprise_store` | Sets whether using enterprise certificate store or not on an output (Windows). | _none_ | ## Get started