diff --git a/.github/workflows/linkcheck.yaml b/.github/workflows/linkcheck.yaml new file mode 100644 index 000000000..ef22bd38f --- /dev/null +++ b/.github/workflows/linkcheck.yaml @@ -0,0 +1,27 @@ +name: Links + +on: + pull_request: + +jobs: + linkChecker: + runs-on: ubuntu-latest + permissions: + issues: write # required for peter-evans/create-issue-from-file + steps: + - uses: actions/checkout@v4 + + - name: Link Checker + id: lychee + uses: lycheeverse/lychee-action@v2 + with: + fail: false + jobSummary: true + +# - name: Create Issue From File +# if: steps.lychee.outputs.exit_code != 0 +# uses: peter-evans/create-issue-from-file@v5 +# with: +# title: Link Checker Report +# content-filepath: ./lychee/out.md +# labels: report, automated issue diff --git a/SUMMARY.md b/SUMMARY.md index a44401c6c..412d6b773 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -26,7 +26,7 @@ * [Download source code](installation/sources/download-source-code.md) * [Build and install](installation/sources/build-and-install.md) * [Build with static configuration](installation/sources/build-with-static-configuration.md) -* [Linux packages](installation/linux/README.md) +* [Linux packages](installation/linux.md) * [Amazon Linux](installation/linux/amazon-linux.md) * [Rocky Linux and Alma Linux ](installation/linux/alma-rocky.md) * [Red Hat and CentOS](installation/linux/redhat-centos.md) @@ -44,8 +44,8 @@ ## Administration -* [Configure Fluent Bit](administration/configuring-fluent-bit/README.md) - * [YAML configuration](administration/configuring-fluent-bit/yaml/README.md) +* [Configure Fluent Bit](administration/configuring-fluent-bit.md) + * [YAML configuration](administration/configuring-fluent-bit/yaml.md) * [Service](administration/configuring-fluent-bit/yaml/service-section.md) * [Parsers](administration/configuring-fluent-bit/yaml/parsers-section.md) * [Multiline parsers](administration/configuring-fluent-bit/yaml/multiline-parsers-section.md) @@ -137,7 +137,7 @@ * [LTSV](pipeline/parsers/ltsv.md) * [Logfmt](pipeline/parsers/logfmt.md) * [Decoders](pipeline/parsers/decoders.md) -* [Processors](pipeline/processors/README.md) +* [Processors](pipeline/processors.md) * [Content modifier](pipeline/processors/content-modifier.md) * [Labels](pipeline/processors/labels.md) * [Metrics selector](pipeline/processors/metrics-selector.md) @@ -223,7 +223,7 @@ * [Introduction to stream processing](stream-processing/introduction.md) * [Overview](stream-processing/overview.md) * [Changelog](stream-processing/changelog.md) -* [Get started](stream-processing/getting-started/README.md) +* [Get started](stream-processing/get-started.md) * [Fluent Bit and SQL](stream-processing/getting-started/fluent-bit-sql.md) * [Check keys and null values](stream-processing/getting-started/check-keys-null-values.md) * [Tutorial](stream-processing/getting-started/hands-on.md) diff --git a/administration/configuring-fluent-bit/README.md b/administration/configuring-fluent-bit.md similarity index 96% rename from administration/configuring-fluent-bit/README.md rename to administration/configuring-fluent-bit.md index 2ebce7678..948e06885 100644 --- a/administration/configuring-fluent-bit/README.md +++ b/administration/configuring-fluent-bit.md @@ -2,8 +2,8 @@ Fluent Bit supports two configuration formats: -- [YAML](yaml/README.md): Standard configuration format as of v3.2. -- [Classic mode](classic-mode/README.md): To be deprecated at the end of 2026. +- [YAML](./configuring-fluent-bit/yaml.md): Standard configuration format as of v3.2. +- [Classic mode](./configuring-fluent-bit/classic-mode/README.md): To be deprecated at the end of 2026. ## Command line interface diff --git a/administration/configuring-fluent-bit/yaml/README.md b/administration/configuring-fluent-bit/yaml.md similarity index 83% rename from administration/configuring-fluent-bit/yaml/README.md rename to administration/configuring-fluent-bit/yaml.md index 7d8c51566..cbba131d4 100644 --- a/administration/configuring-fluent-bit/yaml/README.md +++ b/administration/configuring-fluent-bit/yaml.md @@ -26,19 +26,19 @@ Configuring Fluent Bit with YAML introduces the following root-level sections: To access detailed configuration guides for each section, use the following links: -- [Service Section documentation](service-section.md) +- [Service Section documentation](./yaml/service-section.md) - Overview of global settings, configuration options, and examples. -- [Parsers Section documentation](parsers-section.md) +- [Parsers Section documentation](./yaml/parsers-section.md) - Detailed guide on defining parsers and supported formats. -- [Multiline Parsers Section documentation](multiline-parsers-section.md) +- [Multiline Parsers Section documentation](./yaml/multiline-parsers-section.md) - Explanation of multiline parsing configuration. -- [Pipeline Section documentation](pipeline-section.md) +- [Pipeline Section documentation](./yaml/pipeline-section.md) - Details on setting up pipelines and using processors. -- [Plugins Section documentation](plugins-section.md) +- [Plugins Section documentation](./yaml/plugins-section.md) - How to load external plugins. -- [Upstream Servers Section documentation](upstream-servers-section.md) +- [Upstream Servers Section documentation](./yaml/upstream-servers-section.md) - Guide on setting up and using upstream nodes with supported plugins. -- [Environment Variables Section documentation](environment-variables-section.md) +- [Environment Variables Section documentation](./yaml/environment-variables-section.md) - Information on setting environment variables and their scope within Fluent Bit. -- [Includes Section documentation](includes-section.md) +- [Includes Section documentation](./yaml/includes-section.md) - Description on how to include external YAML files. diff --git a/administration/multithreading.md b/administration/multithreading.md index 2e68452de..926a70884 100644 --- a/administration/multithreading.md +++ b/administration/multithreading.md @@ -9,14 +9,14 @@ timers, receiving internal messages, scheduling flushes, and handling retries. This event loop runs in the main Fluent Bit thread. To free up resources in the main thread, you can configure -[inputs](../pipeline/inputs/README.md) and [outputs](../pipeline/outputs/README.md) +[inputs](../pipeline/inputs.md) and [outputs](../pipeline/outputs.md) to run in their own self-contained threads. However, inputs and outputs implement multithreading in distinct ways: inputs can run in `threaded` mode, and outputs can use one or more `workers`. Threading also affects certain processes related to inputs and outputs. For example, -[filters](../pipeline/filters/README.md) always run in the main thread, but -[processors](../pipeline/processors/README.md) run in the self-contained threads of +[filters](../pipeline/filters.md) always run in the main thread, but +[processors](../pipeline/processors.md) run in the self-contained threads of their respective inputs or outputs, if applicable. ## Inputs diff --git a/installation/linux/README.md b/installation/linux.md similarity index 90% rename from installation/linux/README.md rename to installation/linux.md index 7aac3e7e0..ffacf16d4 100644 --- a/installation/linux/README.md +++ b/installation/linux.md @@ -29,7 +29,7 @@ The GPG Key fingerprint of the old key is: F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A ``` -Refer to the [supported platform documentation](./../supported-platforms.md) to see which platforms are supported in each release. +Refer to the [supported platform documentation](supported-platforms.md) to see which platforms are supported in each release. ## Migration to Fluent Bit diff --git a/installation/sources/build-with-static-configuration.md b/installation/sources/build-with-static-configuration.md index 9be722484..75bcb8774 100644 --- a/installation/sources/build-with-static-configuration.md +++ b/installation/sources/build-with-static-configuration.md @@ -12,7 +12,7 @@ The following steps assume you are familiar with configuring Fluent Bit using te #### Configuration directory -In your file system, prepare a specific directory that will be used as an entry point for the build system to lookup and parse the configuration files. This directory must contain a minimum of one configuration file, called `fluent-bit.conf`, that contains the required [`SERVICE`](/administration/configuring-fluent-bit/yaml/service-section.md), [`INPUT`](../../pipeline/input.md), and [`OUTPUT`](../../pipeline/output.md) sections. +In your file system, prepare a specific directory that will be used as an entry point for the build system to lookup and parse the configuration files. This directory must contain a minimum of one configuration file, called `fluent-bit.conf`, that contains the required [`SERVICE`](/administration/configuring-fluent-bit/yaml/service-section.md), [`INPUT`](../../pipeline/inputs.md), and [`OUTPUT`](../../pipeline/outputs.md) sections. As an example, create a new `fluent-bit.yaml` file or `fluent-bit.conf` file: diff --git a/pipeline/outputs/logdna.md b/pipeline/outputs/logdna.md index af77a76f7..99a75c6f5 100644 --- a/pipeline/outputs/logdna.md +++ b/pipeline/outputs/logdna.md @@ -18,7 +18,7 @@ This plugin uses the following configuration parameters: | `tags` | A list of comma-separated strings to group records in LogDNA and simplify the query with filters. | _none_ | | `file` | Optional name of a file being monitored. This value is only set if the record doesn't contain a reference to it. | _none_ | | `app` | Name of the application. This value is automatically discovered on each record. If no value is found, the default value is used. | `Fluent Bit` | -| `workers` | The number of [workers](../../administration/multithreading#outputs) to perform flush operations for this output. | `0` | +| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | ## Data discovery and enrichment diff --git a/pipeline/outputs/observe.md b/pipeline/outputs/observe.md index 36bbd95d2..ebc32b532 100644 --- a/pipeline/outputs/observe.md +++ b/pipeline/outputs/observe.md @@ -1,6 +1,6 @@ # Observe -Use the [HTTP output plugin](`http`) to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html). It issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format. +Use the [HTTP output plugin](./http.md) to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html). It issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format. ## Configuration parameters diff --git a/pipeline/processors.md b/pipeline/processors.md new file mode 100644 index 000000000..1c22bce2d --- /dev/null +++ b/pipeline/processors.md @@ -0,0 +1,29 @@ +# Processors + +Processors are components that modify, transform, or enhance data as it flows through Fluent Bit. Unlike [filters](filters.md), processors are tightly coupled to inputs, which means they execute immediately and avoid creating a performance bottleneck. + +Additionally, filters can be implemented in a way that mimics the behavior of processors, but processors can't be implemented in a way that mimics filters. + +{% hint style="info" %} + +Only [YAML configuration files](../administration/configuring-fluent-bit/yaml.md) support processors. + +{% endhint %} + +## Available processors + +Fluent Bit offers the following processors: + +- [Content modifier](./processors/content-modifier.md): Manipulate the content, metadata, and attributes of logs and traces. +- [Labels](./processors/labels.md): Add, update, or delete metric labels. +- [Metrics selector](./processors/metrics-selector.md): Choose which metrics to keep or discard. +- [OpenTelemetry envelope](./processors/opentelemetry-envelope.md): Transform logs into an OpenTelemetry-compatible format. +- [Sampling](./processors/sampling.md): Apply head or tail sampling to incoming traces. +- [SQL](./processors/sql.md): Use SQL queries to extract log content. +- [Filters as processors](filters.md): Use filters as processors. + +## Features + +Compatible processors include the following features: + +- [Conditional processing](./processors/conditional-processing.md): Selectively apply processors to logs based on the value of fields that those logs contain. diff --git a/pipeline/processors/README.md b/pipeline/processors/README.md deleted file mode 100644 index a231a0b86..000000000 --- a/pipeline/processors/README.md +++ /dev/null @@ -1,29 +0,0 @@ -# Processors - -Processors are components that modify, transform, or enhance data as it flows through Fluent Bit. Unlike [filters](../filters/README.md), processors are tightly coupled to inputs, which means they execute immediately and avoid creating a performance bottleneck. - -Additionally, filters can be implemented in a way that mimics the behavior of processors, but processors can't be implemented in a way that mimics filters. - -{% hint style="info" %} - -Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/configuration-file.md) support processors. - -{% endhint %} - -## Available processors - -Fluent Bit offers the following processors: - -- [Content modifier](content-modifier.md): Manipulate the content, metadata, and attributes of logs and traces. -- [Labels](labels.md): Add, update, or delete metric labels. -- [Metrics selector](metrics-selector.md): Choose which metrics to keep or discard. -- [OpenTelemetry envelope](opentelemetry-envelope.md): Transform logs into an OpenTelemetry-compatible format. -- [Sampling](sampling.md): Apply head or tail sampling to incoming traces. -- [SQL](sql.md): Use SQL queries to extract log content. -- [Filters as processors](filters.md): Use filters as processors. - -## Features - -Compatible processors include the following features: - -- [Conditional processing](conditional-processing.md): Selectively apply processors to logs based on the value of fields that those logs contain. diff --git a/pipeline/processors/conditional-processing.md b/pipeline/processors/conditional-processing.md index eaf1ca0e4..51f5272e6 100644 --- a/pipeline/processors/conditional-processing.md +++ b/pipeline/processors/conditional-processing.md @@ -1,6 +1,6 @@ # Conditional processing -Conditional processing lets you selectively apply [processors](README.md) to logs based on the value of fields within those logs. This feature lets you create processing pipelines that only process records that meet certain criteria, and ignore the rest. +Conditional processing lets you selectively apply [processors](../processors.md) to logs based on the value of fields within those logs. This feature lets you create processing pipelines that only process records that meet certain criteria, and ignore the rest. Conditional processing is available in Fluent Bit version 4.0 and greater. @@ -10,8 +10,8 @@ You can turn a standard processor into a conditional processor by adding a `cond {% hint style="info" %} -- Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors, including conditional processors. -- Conditional processing isn't supported if you're using a [filter as a processor](../filters). +- Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors, including conditional processors. +- Conditional processing isn't supported if you're using a [filter as a processor](../filters.md). {% endhint %} diff --git a/pipeline/processors/content-modifier.md b/pipeline/processors/content-modifier.md index fd380386b..f6ee26d33 100644 --- a/pipeline/processors/content-modifier.md +++ b/pipeline/processors/content-modifier.md @@ -6,7 +6,7 @@ Similar to how filters work, this processor uses a unified mechanism to perform {% hint style="info" %} -Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors. +Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors. {% endhint %} diff --git a/pipeline/processors/filters.md b/pipeline/processors/filters.md index 9879f2b94..3ed879f3c 100644 --- a/pipeline/processors/filters.md +++ b/pipeline/processors/filters.md @@ -1,10 +1,10 @@ # Filters as processors -You can use any [filter](../filters/README.md) as a processor in Fluent Bit. +You can use any [filter](../filters.md) as a processor in Fluent Bit. {% hint style="info" %} -Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors. +Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors. {% endhint %} @@ -33,4 +33,4 @@ pipeline: ``` {% endtab %} -{% endtabs %} \ No newline at end of file +{% endtabs %} diff --git a/pipeline/processors/labels.md b/pipeline/processors/labels.md index c6a902b06..b48b85af5 100644 --- a/pipeline/processors/labels.md +++ b/pipeline/processors/labels.md @@ -6,7 +6,7 @@ Similar to filters, this processor presents an enriching/modifying mechanism to {% hint style="info" %} -Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors. +Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors. {% endhint %} @@ -36,7 +36,7 @@ pipeline: metrics: - name: labels update: name fluentbit - + outputs: - name : stdout match: '*' @@ -86,7 +86,7 @@ pipeline: metrics: - name: labels upsert: name fluentbit - + outputs: - name : stdout match: '*' @@ -111,7 +111,7 @@ pipeline: metrics: - name: labels delete: name - + outputs: - name : stdout match: '*' @@ -143,4 +143,4 @@ pipeline: ``` {% endtab %} -{% endtabs %} \ No newline at end of file +{% endtabs %} diff --git a/pipeline/processors/opentelemetry-envelope.md b/pipeline/processors/opentelemetry-envelope.md index d22662adb..941b2317a 100644 --- a/pipeline/processors/opentelemetry-envelope.md +++ b/pipeline/processors/opentelemetry-envelope.md @@ -6,7 +6,7 @@ The _OpenTelemetry envelope_ processor transforms your data to be compatible wit {% hint style="info" %} -Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors. +Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors. {% endhint %} @@ -178,4 +178,5 @@ The collector JSON output will resemble the following: } ``` -For more details about further processing, read the [content modifier](../processors/content-modifier.md) processor documentation. \ No newline at end of file +For more details about further processing, read the [content +modifier](../processors/content-modifier.md) processor documentation. diff --git a/pipeline/processors/sampling.md b/pipeline/processors/sampling.md index 8232de6bd..abfd36a7e 100644 --- a/pipeline/processors/sampling.md +++ b/pipeline/processors/sampling.md @@ -4,7 +4,7 @@ The _sampling_ processor uses an extendable architecture that supports multiple {% hint style="info" %} -Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors. +Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors. {% endhint %} diff --git a/pipeline/processors/sql.md b/pipeline/processors/sql.md index 4ade4b640..1b26776a5 100644 --- a/pipeline/processors/sql.md +++ b/pipeline/processors/sql.md @@ -6,7 +6,7 @@ This processor differs from the stream processor interface that runs after filte {% hint style="info" %} -Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors. +Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors. {% endhint %} @@ -82,4 +82,4 @@ The resulting output resembles the following: "date": 1711059261.630668, "http_domain": "fluentbit.io" } -``` \ No newline at end of file +``` diff --git a/pipeline/router.md b/pipeline/router.md index 57a3cb138..fe4e0c176 100644 --- a/pipeline/router.md +++ b/pipeline/router.md @@ -4,7 +4,7 @@ description: Create flexible routing rules # Router -Routing is a core feature that lets you route your data through filters and then to one or multiple destinations. The router relies on the concept of [Tags](../concepts/key-concepts.md) and [Matching](../key-concepts.md) rules. +Routing is a core feature that lets you route your data through filters and then to one or multiple destinations. The router relies on the concept of [Tags](../concepts/key-concepts.md) and [Matching](../concepts/key-concepts.md) rules. ```mermaid graph LR diff --git a/stream-processing/getting-started/README.md b/stream-processing/get-started.md similarity index 100% rename from stream-processing/getting-started/README.md rename to stream-processing/get-started.md