Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e04a513
Standardizing shell usage in stream processing hands-on doc. Part of …
eschabell Jul 29, 2025
c857800
Standardizing shell usage in installation linux raspbeerry-pi doc. P…
eschabell Jul 29, 2025
062db83
Standardizing shell usage in installation linux rh-centos doc. Part …
eschabell Jul 29, 2025
4e1b7a8
Standardizing shell usage in installation linux ubuntu doc. Part of i…
eschabell Jul 29, 2025
6546bf8
Standardizing shell usage in installation linux alma-rocky doc. Part …
eschabell Jul 29, 2025
54746b6
Standardizing shell usage in installation linux amazon linux doc. Par…
eschabell Jul 29, 2025
7342955
Standardizing shell usage in installation linux debian doc. Part of i…
eschabell Jul 29, 2025
e5e5692
Standardizing shell usage in installation source macos doc. Part of i…
eschabell Jul 29, 2025
4f95881
Standardizing shell usage in installation aws container doc. Part of …
eschabell Jul 29, 2025
d04c4b0
Standardizing shell usage and added yaml code example to development …
eschabell Jul 29, 2025
75df41e
Standardizing shell usage to developement msgpack format doc. Part of…
eschabell Jul 29, 2025
44f757a
Standardizing shell usage in admin networking doc. Part of issue #2020.
eschabell Jul 29, 2025
a8131a8
Standardizing shell usage in installation sources build and install d…
eschabell Jul 29, 2025
bd60f98
Standardizing shell usage in installation soruces download doc. Part …
eschabell Jul 29, 2025
6eef99d
Standardizing shell usage in installation soruces build with static c…
eschabell Jul 29, 2025
e84fbde
Standardizing shell usage in installation local testing logging pipel…
eschabell Jul 29, 2025
1dd4bc4
Standardizing shell usage in admin transport security doc. Part of is…
eschabell Jul 29, 2025
71a669e
Standardizing shell usage in admin config classic mode variables doc.…
eschabell Jul 29, 2025
adaf397
Standardizing shell usage in admin config yaml env var section doc. P…
eschabell Jul 29, 2025
c7b6944
Standardizing shell usage in installation upgrade notes doc. Part of …
eschabell Jul 29, 2025
6777126
Standardizing shell usage in installation windows doc. Part of issue …
eschabell Jul 29, 2025
3018c20
Standardizing shell usage in admin config classic mode config file do…
eschabell Jul 29, 2025
ef5a91d
Standardizing shell usage in admin config classic mode format schema …
eschabell Jul 29, 2025
6bd997d
Standardizing shell usage in admin config classic mode upstream serve…
eschabell Jul 29, 2025
85cd5e7
Standardizing shell usage in admin config multiline parsing doc. Part…
eschabell Jul 29, 2025
86ff9cc
Standardizing shell usage in inputs exec plugin doc. Part of issue #2…
eschabell Jul 29, 2025
0fcbcc6
Standardizing shell usage in inputs standard input plugin doc. Part o…
eschabell Jul 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Configuration file

<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5e67142e-3887-4b56-b940-18494bcc23a7" />

One of the ways to configure Fluent Bit is using a main configuration file. Fluent Bit allows the use one configuration file that works at a global scope and uses the defined [Format and Schema](format-schema.md).

The main configuration file supports four sections:
Expand Down Expand Up @@ -39,11 +37,11 @@ The `Service` section defines global properties of the service. The following ke

The following is an example of a `SERVICE` section:

```python
```text
[SERVICE]
Flush 5
Daemon off
Log_Level debug
Flush 5
Daemon off
Log_Level debug
```

For scheduler and retry details, see [scheduling and retries](../../scheduling-and-retries.md#Scheduling-and-Retries).
Expand All @@ -64,10 +62,10 @@ The `INPUT` section defines a source (related to an input plugin). Each [input p

The following is an example of an `INPUT` section:

```python
```text
[INPUT]
Name cpu
Tag my_cpu
Name cpu
Tag my_cpu
```

## Config filter
Expand All @@ -87,11 +85,11 @@ The `FILTER` section defines a filter (related to an filter plugin). Each filter

The following is an example of a `FILTER` section:

```python
```text
[FILTER]
Name grep
Match *
Regex log aa
Name grep
Match *
Regex log aa
```

## Config output
Expand All @@ -109,29 +107,29 @@ The `OUTPUT` section specifies a destination that certain records should go to a

The following is an example of an `OUTPUT` section:

```python
```text
[OUTPUT]
Name stdout
Match my*cpu
Name stdout
Match my*cpu
```

### Collecting `cpu` metrics example

The following configuration file example demonstrates how to collect CPU metrics and flush the results every five seconds to the standard output:

```python
```text
[SERVICE]
Flush 5
Daemon off
Log_Level debug
Flush 5
Daemon off
Log_Level debug

[INPUT]
Name cpu
Tag my_cpu
Name cpu
Tag my_cpu

[OUTPUT]
Name stdout
Match my*cpu
Name stdout
Match my*cpu
```

## Config include file
Expand All @@ -156,4 +154,4 @@ Wildcard character (`*`) supports including multiple files. For example:
@INCLUDE input_*.conf
```

Files matching the wildcard character are included unsorted. If plugin ordering between files needs to be preserved, the files should be included explicitly.
Files matching the wildcard character are included unsorted. If plugin ordering between files needs to be preserved, the files should be included explicitly.
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ The schema is defined by three concepts:

An example of a configuration file is as follows:

```python
```text
[SERVICE]
# This is a commented line
Daemon off
log_level debug
# This is a commented line
Daemon off
log_level debug
```

## Sections
Expand Down Expand Up @@ -42,15 +42,15 @@ Commented lines are set prefixing the `#` character. Commented lines aren't proc

Fluent Bit configuration files are based in a strict indented mode. Each configuration file must follow the same pattern of alignment from left to right when writing text. By default, an indentation level of four spaces from left to right is suggested. Example:

```python
```text
[FIRST_SECTION]
# This is a commented line
Key1 some value
Key2 another value
# more comments
# This is a commented line
Key1 some value
Key2 another value
# more comments

[SECOND_SECTION]
KeyN 3.14
KeyN 3.14
```

This example shows two sections with multiple entries and comments. Empty lines are allowed.
This example shows two sections with multiple entries and comments. Empty lines are allowed.
Original file line number Diff line number Diff line change
Expand Up @@ -37,27 +37,27 @@ The following example defines an `Upstream` called forward-balancing which aims
- node-2: connects to 127.0.0.1:44000
- node-3: connects to 127.0.0.1:45000 using TLS without verification. It also defines a specific configuration option required by Forward output called `shared_key`.

```python
```text
[UPSTREAM]
name forward-balancing
name forward-balancing

[NODE]
name node-1
host 127.0.0.1
port 43000
name node-1
host 127.0.0.1
port 43000

[NODE]
name node-2
host 127.0.0.1
port 44000
name node-2
host 127.0.0.1
port 44000

[NODE]
name node-3
host 127.0.0.1
port 45000
tls on
tls.verify off
shared_key secret
name node-3
host 127.0.0.1
port 45000
tls on
tls.verify off
shared_key secret
```

Every `Upstream` definition must exists in its own configuration file in the file system. Adding multiple `Upstream` configurations in the same file or different files isn't allowed.
Every `Upstream` definition must exists in its own configuration file in the file system. Adding multiple `Upstream` configurations in the same file or different files isn't allowed.
29 changes: 11 additions & 18 deletions administration/configuring-fluent-bit/classic-mode/variables.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Variables

<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=1731c7b5-34c6-424f-bfc6-88c2aa71e81f" />

Fluent Bit supports the usage of environment variables in any value associated to a key when using a configuration file.

The variables are case sensitive and can be used in the following format:
Expand All @@ -25,37 +23,32 @@ Create the following configuration file \(`fluent-bit.conf`\):

```yaml
[SERVICE]
Flush 1
Daemon Off
Log_Level info
Flush 1
Daemon Off
Log_Level info

[INPUT]
Name cpu
Tag cpu.local
Name cpu
Tag cpu.local

[OUTPUT]
Name ${MY_OUTPUT}
Match *
Name ${MY_OUTPUT}
Match *
```

Open a terminal and set the environment variable:

```bash
```shell
export MY_OUTPUT=stdout
```

The previous command sets the `stdout` value to the variable `MY_OUTPUT`.

Run Fluent Bit with the recently created configuration file:

```text
```shell
$ bin/fluent-bit -c fluent-bit.conf
Fluent Bit v1.4.0
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2020/03/03 12:25:25] [ info] [engine] started
...
[0] cpu.local: [1491243925, {"cpu_p"=>1.750000, "user_p"=>1.750000, "system_p"=>0.000000, "cpu0.p_cpu"=>3.000000, "cpu0.p_user"=>2.000000, "cpu0.p_system"=>1.000000, "cpu1.p_cpu"=>0.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>0.000000, "cpu2.p_cpu"=>4.000000, "cpu2.p_user"=>4.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>1.000000, "cpu3.p_user"=>1.000000, "cpu3.p_system"=>0.000000}]
```
```
8 changes: 3 additions & 5 deletions administration/configuring-fluent-bit/multiline-parsing.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,6 @@ To simplify the configuration of regular expressions, you can use the [Rubular](
The following example provides a full Fluent Bit configuration file for multiline parsing by using the definition explained previously. It's provided in following YAML and classic configuration examples:

{% tabs %}

{% tab title="fluent-bit.yaml" %}

This is the primary Fluent Bit YAML configuration file. It includes the `parsers_multiline.yaml` and tails the file `test.log` by applying the multiline parser `multiline-regex-test`. Then it sends the processing to the standard output.
Expand Down Expand Up @@ -224,7 +223,7 @@ another line...

By running Fluent Bit with the corresponding configuration file you will obtain the following output:

```text
```shell
# For YAML configuration.
$ ./fluent-bit --config fluent-bit.yaml

Expand Down Expand Up @@ -399,15 +398,14 @@ Dec 14 06:41:08 Exception in thread "main" java.lang.RuntimeException: Something
at com.myproject.module.MyProject.someMethod(MyProject.java:10)
at com.myproject.module.MyProject.main(MyProject.java:6)
another line...

```

{% endtab %}
{% endtabs %}

By running Fluent Bit with the corresponding configuration file you will obtain:

```text
```shell
# For YAML configuration.
$ ./fluent-bit --config fluent-bit.yaml

Expand All @@ -425,4 +423,4 @@ $ ./fluent-bit --config fluent-bit.conf
"}]
[2] tail.0: [[1750333602.460998000, {}], {"log"=>"another line...
"}]
```
```
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Variables set in the `env` section can't be overridden by system environment var

For example, to set the `FLUSH_INTERVAL` system environment variable to `2` and use it in your configuration:

```bash
```shell
export FLUSH_INTERVAL=2
```

Expand All @@ -62,4 +62,4 @@ pipeline:
format: json_lines
```

This approach lets you manage and override configuration values using environment variables, providing flexibility in various deployment environments.
This approach lets you manage and override configuration values using environment variables, providing flexibility in various deployment environments.
5 changes: 3 additions & 2 deletions administration/networking.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,14 +134,15 @@ pipeline:

In another terminal, start `nc` and make it listen for messages on TCP port 9090:

```text
```shell
nc -l 9090
```

Start Fluent Bit with the configuration file you defined previously to see data flowing to netcat:

```text
```shell
$ nc -l 9090

{"date":1587769732.572266,"rand_value":9704012962543047466}
{"date":1587769733.572354,"rand_value":7609018546050096989}
{"date":1587769734.572388,"rand_value":17035865539257638950}
Expand Down
4 changes: 2 additions & 2 deletions administration/transport-security.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ pipeline:

By default, the HTTP output plugin uses plain TCP. Run the following command to enable TLS:

```bash
```shell
fluent-bit -i cpu -t cpu -o http://192.168.2.3:80/something \
-p tls=on \
-p tls.verify=off \
Expand Down Expand Up @@ -317,4 +317,4 @@ This outgoing connect will fail and disconnect:
[2024/06/17 16:51:31] [error] [tls] error: unexpected EOF with reason: certificate verify failed
[2024/06/17 16:51:31] [debug] [upstream] connection #50 failed to other.fluent-aggregator.net:24224
[2024/06/17 16:51:31] [error] [output:forward:forward.0] no upstream connections available
```
```
Loading