Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/linkcheck.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Links

on:
pull_request:

jobs:
linkChecker:
runs-on: ubuntu-latest
permissions:
issues: write # required for peter-evans/create-issue-from-file
steps:
- uses: actions/checkout@v4

- name: Link Checker
id: lychee
uses: lycheeverse/lychee-action@v2
with:
fail: false
jobSummary: true

# - name: Create Issue From File
# if: steps.lychee.outputs.exit_code != 0
# uses: peter-evans/create-issue-from-file@v5
# with:
# title: Link Checker Report
# content-filepath: ./lychee/out.md
# labels: report, automated issue
10 changes: 5 additions & 5 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
* [Download source code](installation/sources/download-source-code.md)
* [Build and install](installation/sources/build-and-install.md)
* [Build with static configuration](installation/sources/build-with-static-configuration.md)
* [Linux packages](installation/linux/README.md)
* [Linux packages](installation/linux.md)
* [Amazon Linux](installation/linux/amazon-linux.md)
* [Rocky Linux and Alma Linux ](installation/linux/alma-rocky.md)
* [Red Hat and CentOS](installation/linux/redhat-centos.md)
Expand All @@ -44,8 +44,8 @@

## Administration

* [Configure Fluent Bit](administration/configuring-fluent-bit/README.md)
* [YAML configuration](administration/configuring-fluent-bit/yaml/README.md)
* [Configure Fluent Bit](administration/configuring-fluent-bit.md)
* [YAML configuration](administration/configuring-fluent-bit/yaml.md)
* [Service](administration/configuring-fluent-bit/yaml/service-section.md)
* [Parsers](administration/configuring-fluent-bit/yaml/parsers-section.md)
* [Multiline parsers](administration/configuring-fluent-bit/yaml/multiline-parsers-section.md)
Expand Down Expand Up @@ -137,7 +137,7 @@
* [LTSV](pipeline/parsers/ltsv.md)
* [Logfmt](pipeline/parsers/logfmt.md)
* [Decoders](pipeline/parsers/decoders.md)
* [Processors](pipeline/processors/README.md)
* [Processors](pipeline/processors.md)
* [Content modifier](pipeline/processors/content-modifier.md)
* [Labels](pipeline/processors/labels.md)
* [Metrics selector](pipeline/processors/metrics-selector.md)
Expand Down Expand Up @@ -223,7 +223,7 @@
* [Introduction to stream processing](stream-processing/introduction.md)
* [Overview](stream-processing/overview.md)
* [Changelog](stream-processing/changelog.md)
* [Get started](stream-processing/getting-started/README.md)
* [Get started](stream-processing/get-started.md)
* [Fluent Bit and SQL](stream-processing/getting-started/fluent-bit-sql.md)
* [Check keys and null values](stream-processing/getting-started/check-keys-null-values.md)
* [Tutorial](stream-processing/getting-started/hands-on.md)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

Fluent Bit supports two configuration formats:

- [YAML](yaml/README.md): Standard configuration format as of v3.2.
- [Classic mode](classic-mode/README.md): To be deprecated at the end of 2026.
- [YAML](./configuring-fluent-bit/yaml.md): Standard configuration format as of v3.2.
- [Classic mode](./configuring-fluent-bit/classic-mode/README.md): To be deprecated at the end of 2026.

## Command line interface

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,19 +26,19 @@ Configuring Fluent Bit with YAML introduces the following root-level sections:

To access detailed configuration guides for each section, use the following links:

- [Service Section documentation](service-section.md)
- [Service Section documentation](./yaml/service-section.md)
- Overview of global settings, configuration options, and examples.
- [Parsers Section documentation](parsers-section.md)
- [Parsers Section documentation](./yaml/parsers-section.md)
- Detailed guide on defining parsers and supported formats.
- [Multiline Parsers Section documentation](multiline-parsers-section.md)
- [Multiline Parsers Section documentation](./yaml/multiline-parsers-section.md)
- Explanation of multiline parsing configuration.
- [Pipeline Section documentation](pipeline-section.md)
- [Pipeline Section documentation](./yaml/pipeline-section.md)
- Details on setting up pipelines and using processors.
- [Plugins Section documentation](plugins-section.md)
- [Plugins Section documentation](./yaml/plugins-section.md)
- How to load external plugins.
- [Upstream Servers Section documentation](upstream-servers-section.md)
- [Upstream Servers Section documentation](./yaml/upstream-servers-section.md)
- Guide on setting up and using upstream nodes with supported plugins.
- [Environment Variables Section documentation](environment-variables-section.md)
- [Environment Variables Section documentation](./yaml/environment-variables-section.md)
- Information on setting environment variables and their scope within Fluent Bit.
- [Includes Section documentation](includes-section.md)
- [Includes Section documentation](./yaml/includes-section.md)
- Description on how to include external YAML files.
6 changes: 3 additions & 3 deletions administration/multithreading.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@ timers, receiving internal messages, scheduling flushes, and handling retries.
This event loop runs in the main Fluent Bit thread.

To free up resources in the main thread, you can configure
[inputs](../pipeline/inputs/README.md) and [outputs](../pipeline/outputs/README.md)
[inputs](../pipeline/inputs.md) and [outputs](../pipeline/outputs.md)
to run in their own self-contained threads. However, inputs and outputs implement
multithreading in distinct ways: inputs can run in `threaded` mode, and outputs
can use one or more `workers`.

Threading also affects certain processes related to inputs and outputs. For example,
[filters](../pipeline/filters/README.md) always run in the main thread, but
[processors](../pipeline/processors/README.md) run in the self-contained threads of
[filters](../pipeline/filters.md) always run in the main thread, but
[processors](../pipeline/processors.md) run in the self-contained threads of
their respective inputs or outputs, if applicable.

## Inputs
Expand Down
2 changes: 1 addition & 1 deletion installation/linux/README.md → installation/linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The GPG Key fingerprint of the old key is:
F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A
```

Refer to the [supported platform documentation](./../supported-platforms.md) to see which platforms are supported in each release.
Refer to the [supported platform documentation](supported-platforms.md) to see which platforms are supported in each release.

## Migration to Fluent Bit

Expand Down
2 changes: 1 addition & 1 deletion installation/sources/build-with-static-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The following steps assume you are familiar with configuring Fluent Bit using te

#### Configuration directory

In your file system, prepare a specific directory that will be used as an entry point for the build system to lookup and parse the configuration files. This directory must contain a minimum of one configuration file, called `fluent-bit.conf`, that contains the required [`SERVICE`](/administration/configuring-fluent-bit/yaml/service-section.md), [`INPUT`](../../pipeline/input.md), and [`OUTPUT`](../../pipeline/output.md) sections.
In your file system, prepare a specific directory that will be used as an entry point for the build system to lookup and parse the configuration files. This directory must contain a minimum of one configuration file, called `fluent-bit.conf`, that contains the required [`SERVICE`](/administration/configuring-fluent-bit/yaml/service-section.md), [`INPUT`](../../pipeline/inputs.md), and [`OUTPUT`](../../pipeline/outputs.md) sections.

As an example, create a new `fluent-bit.yaml` file or `fluent-bit.conf` file:

Expand Down
2 changes: 1 addition & 1 deletion pipeline/outputs/logdna.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ This plugin uses the following configuration parameters:
| `tags` | A list of comma-separated strings to group records in LogDNA and simplify the query with filters. | _none_ |
| `file` | Optional name of a file being monitored. This value is only set if the record doesn't contain a reference to it. | _none_ |
| `app` | Name of the application. This value is automatically discovered on each record. If no value is found, the default value is used. | `Fluent Bit` |
| `workers` | The number of [workers](../../administration/multithreading#outputs) to perform flush operations for this output. | `0` |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |

## Data discovery and enrichment

Expand Down
2 changes: 1 addition & 1 deletion pipeline/outputs/observe.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Observe

Use the [HTTP output plugin](`http`) to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html). It issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.
Use the [HTTP output plugin](./http.md) to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html). It issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.

## Configuration parameters

Expand Down
29 changes: 29 additions & 0 deletions pipeline/processors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Processors

Processors are components that modify, transform, or enhance data as it flows through Fluent Bit. Unlike [filters](filters.md), processors are tightly coupled to inputs, which means they execute immediately and avoid creating a performance bottleneck.

Additionally, filters can be implemented in a way that mimics the behavior of processors, but processors can't be implemented in a way that mimics filters.

{% hint style="info" %}

Only [YAML configuration files](../administration/configuring-fluent-bit/yaml.md) support processors.

{% endhint %}

## Available processors

Fluent Bit offers the following processors:

- [Content modifier](./processors/content-modifier.md): Manipulate the content, metadata, and attributes of logs and traces.
- [Labels](./processors/labels.md): Add, update, or delete metric labels.
- [Metrics selector](./processors/metrics-selector.md): Choose which metrics to keep or discard.
- [OpenTelemetry envelope](./processors/opentelemetry-envelope.md): Transform logs into an OpenTelemetry-compatible format.
- [Sampling](./processors/sampling.md): Apply head or tail sampling to incoming traces.
- [SQL](./processors/sql.md): Use SQL queries to extract log content.
- [Filters as processors](filters.md): Use filters as processors.

## Features

Compatible processors include the following features:

- [Conditional processing](./processors/conditional-processing.md): Selectively apply processors to logs based on the value of fields that those logs contain.
29 changes: 0 additions & 29 deletions pipeline/processors/README.md

This file was deleted.

6 changes: 3 additions & 3 deletions pipeline/processors/conditional-processing.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Conditional processing

Conditional processing lets you selectively apply [processors](README.md) to logs based on the value of fields within those logs. This feature lets you create processing pipelines that only process records that meet certain criteria, and ignore the rest.
Conditional processing lets you selectively apply [processors](../processors.md) to logs based on the value of fields within those logs. This feature lets you create processing pipelines that only process records that meet certain criteria, and ignore the rest.

Conditional processing is available in Fluent Bit version 4.0 and greater.

Expand All @@ -10,8 +10,8 @@ You can turn a standard processor into a conditional processor by adding a `cond

{% hint style="info" %}

- Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors, including conditional processors.
- Conditional processing isn't supported if you're using a [filter as a processor](../filters).
- Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors, including conditional processors.
- Conditional processing isn't supported if you're using a [filter as a processor](../filters.md).

{% endhint %}

Expand Down
2 changes: 1 addition & 1 deletion pipeline/processors/content-modifier.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Similar to how filters work, this processor uses a unified mechanism to perform

{% hint style="info" %}

Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors.
Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors.

{% endhint %}

Expand Down
6 changes: 3 additions & 3 deletions pipeline/processors/filters.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Filters as processors

You can use any [filter](../filters/README.md) as a processor in Fluent Bit.
You can use any [filter](../filters.md) as a processor in Fluent Bit.

{% hint style="info" %}

Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors.
Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors.

{% endhint %}

Expand Down Expand Up @@ -33,4 +33,4 @@ pipeline:
```

{% endtab %}
{% endtabs %}
{% endtabs %}
10 changes: 5 additions & 5 deletions pipeline/processors/labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Similar to filters, this processor presents an enriching/modifying mechanism to

{% hint style="info" %}

Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors.
Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors.

{% endhint %}

Expand Down Expand Up @@ -36,7 +36,7 @@ pipeline:
metrics:
- name: labels
update: name fluentbit

outputs:
- name : stdout
match: '*'
Expand Down Expand Up @@ -86,7 +86,7 @@ pipeline:
metrics:
- name: labels
upsert: name fluentbit

outputs:
- name : stdout
match: '*'
Expand All @@ -111,7 +111,7 @@ pipeline:
metrics:
- name: labels
delete: name

outputs:
- name : stdout
match: '*'
Expand Down Expand Up @@ -143,4 +143,4 @@ pipeline:
```

{% endtab %}
{% endtabs %}
{% endtabs %}
5 changes: 3 additions & 2 deletions pipeline/processors/opentelemetry-envelope.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The _OpenTelemetry envelope_ processor transforms your data to be compatible wit

{% hint style="info" %}

Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors.
Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors.

{% endhint %}

Expand Down Expand Up @@ -178,4 +178,5 @@ The collector JSON output will resemble the following:
}
```

For more details about further processing, read the [content modifier](../processors/content-modifier.md) processor documentation.
For more details about further processing, read the [content
modifier](../processors/content-modifier.md) processor documentation.
2 changes: 1 addition & 1 deletion pipeline/processors/sampling.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The _sampling_ processor uses an extendable architecture that supports multiple

{% hint style="info" %}

Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors.
Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors.

{% endhint %}

Expand Down
4 changes: 2 additions & 2 deletions pipeline/processors/sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This processor differs from the stream processor interface that runs after filte

{% hint style="info" %}

Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors.
Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml.md) support processors.

{% endhint %}

Expand Down Expand Up @@ -82,4 +82,4 @@ The resulting output resembles the following:
"date": 1711059261.630668,
"http_domain": "fluentbit.io"
}
```
```
2 changes: 1 addition & 1 deletion pipeline/router.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Create flexible routing rules

# Router

Routing is a core feature that lets you route your data through filters and then to one or multiple destinations. The router relies on the concept of [Tags](../concepts/key-concepts.md) and [Matching](../key-concepts.md) rules.
Routing is a core feature that lets you route your data through filters and then to one or multiple destinations. The router relies on the concept of [Tags](../concepts/key-concepts.md) and [Matching](../concepts/key-concepts.md) rules.

```mermaid
graph LR
Expand Down