Skip to content

Commit f9dfcf2

Browse files
alexakreizingeralexakreizinger
authored
Merge branch 'master' into alexakreizinger/sc-136247/update-fluent-bit-docs-pipeline-outputs-kinesis
2 parents cec810e + 90a40e3 commit f9dfcf2

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+2452
-2811
lines changed

‎administration/configuring-fluent-bit/yaml/service-section.md

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -2,30 +2,30 @@
22

33
The `service` section defines global properties of the service. The available configuration keys are:
44

5-
| Key | Description | Default |
6-
|---|---|---|
7-
| `flush` | Sets the flush time in `seconds.nanoseconds`. The engine loop uses a flush timeout to determine when to flush records ingested by input plugins to output plugins. | `1` |
8-
| `grace` | Sets the grace time in `seconds` as an integer value. The engine loop uses a grace timeout to define the wait time before exiting. | `5` |
9-
| `daemon` | Boolean. Specifies whether Fluent Bit should run as a daemon (background process). Allowed values are: `yes`, `no`, `on`, and `off`. Don't enable when using a Systemd-based unit, such as the one provided in Fluent Bit packages. | `off` |
10-
| `dns.mode` | Sets the primary transport layer protocol used by the asynchronous DNS resolver. Can be overridden on a per-plugin basis. | `UDP` |
11-
| `log_file` | Absolute path for an optional log file. By default, all logs are redirected to the standard error interface (stderr). | _none_ |
12-
| `log_level` | Sets the logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. If `debug` is set, it will include `error`, `warn`, `info`, and `debug`. Trace mode is only available if Fluent Bit was built with the _`WITH_TRACE`_ option enabled. | `info` |
13-
| `parsers_file` | Path for a `parsers` configuration file. Multiple `parsers_file` entries can be defined within the section. However, with the new YAML configuration schema, defining parsers using this key is now optional. Parsers can be declared directly in the `parsers` section of your YAML configuration, offering a more streamlined and integrated approach. | _none_ |
14-
| `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to external plugins (.so files) that Fluent Bit can load at runtime. With the new YAML schema, the `plugins_file` key is optional. External plugins can now be referenced directly within the `plugins` section, simplifying the plugin management process. [See an example](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). | _none_ |
5+
| Key | Description | Default |
6+
|---|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---|
7+
| `flush` | Sets the flush time in `seconds.nanoseconds`. The engine loop uses a flush timeout to determine when to flush records ingested by input plugins to output plugins. | `1` |
8+
| `grace` | Sets the grace time in `seconds` as an integer value. The engine loop uses a grace timeout to define the wait time before exiting. | `5` |
9+
| `daemon` | Boolean. Specifies whether Fluent Bit should run as a daemon (background process). Allowed values are: `yes`, `no`, `on`, and `off`. Don't enable when using a Systemd-based unit, such as the one provided in Fluent Bit packages. | `off` |
10+
| `dns.mode` | Sets the primary transport layer protocol used by the asynchronous DNS resolver. Can be overridden on a per-plugin basis. | `UDP` |
11+
| `log_file` | Absolute path for an optional log file. By default, all logs are redirected to the standard error interface (stderr). | _none_ |
12+
| `log_level` | Sets the logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. If `debug` is set, it will include `error`, `warn`, `info`, and `debug`. Trace mode is only available if Fluent Bit was built with the _`WITH_TRACE`_ option enabled. | `info` |
13+
| `parsers_file` | Path for a `parsers` configuration file. Multiple `parsers_file` entries can be defined within the section. However, with the new YAML configuration schema, defining parsers using this key is now optional. Parsers can be declared directly in the `parsers` section of your YAML configuration, offering a more streamlined and integrated approach. | _none_ |
14+
| `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to external plugins (.so files) that Fluent Bit can load at runtime. With the new YAML schema, the `plugins_file` key is optional. External plugins can now be referenced directly within the `plugins` section, simplifying the plugin management process. [See an example](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). | _none_ |
1515
| `streams_file` | Path for the Stream Processor configuration file. This file defines the rules and operations for stream processing within Fluent Bit. The `streams_file` key is optional, as Stream Processor configurations can be defined directly in the `streams` section of the YAML schema. This flexibility allows for easier and more centralized configuration. [Learn more about Stream Processing configuration](../../../stream-processing/introduction.md). | _none_ |
16-
| `http_server` | Enables the built-in HTTP Server. | `off` |
17-
| `http_listen` | Sets the listening interface for the HTTP Server when it's enabled. | `0.0.0.0` |
18-
| `http_port` | Sets the TCP port for the HTTP Server. | `2020` |
19-
| `hot_reload` | Enables hot [reloading](../../hot_reload.md) of configuration with SIGHUP. | `on` |
20-
| `coro_stack_size` | Sets the coroutine stack size in bytes. The value must be greater than the page size of the running system. Setting the value too small (`4096`) can cause coroutine threads to overrun the stack buffer. The default value of this parameter shouldn't be changed. | `24576` |
21-
| `scheduler.cap` | Sets a maximum retry time in seconds. Supported in v1.8.7 and greater. | `2000` |
22-
| `scheduler.base` | Sets the base of exponential backoff. Supported in v1.8.7 and greater. | `5` |
23-
| `json.convert_nan_to_null` | If enabled, `NaN` is converted to `null` when Fluent Bit converts `msgpack` to `json`. | `false` |
24-
| `sp.convert_from_str_to_num` | If enabled, the Stream Processor converts strings that represent numbers to a numeric type. | `true` |
16+
| `http_server` | Enables the built-in HTTP Server. | `off` |
17+
| `http_listen` | Sets the listening interface for the HTTP Server when it's enabled. | `0.0.0.0` |
18+
| `http_port` | Sets the TCP port for the HTTP Server. | `2020` |
19+
| `hot_reload` | Enables [hot reloading](../../hot-reload.md) of configuration with SIGHUP. | `on` |
20+
| `coro_stack_size` | Sets the coroutine stack size in bytes. The value must be greater than the page size of the running system. Setting the value too small (`4096`) can cause coroutine threads to overrun the stack buffer. The default value of this parameter shouldn't be changed. | `24576` |
21+
| `scheduler.cap` | Sets a maximum retry time in seconds. Supported in v1.8.7 and greater. | `2000` |
22+
| `scheduler.base` | Sets the base of exponential backoff. Supported in v1.8.7 and greater. | `5` |
23+
| `json.convert_nan_to_null` | If enabled, `NaN` is converted to `null` when Fluent Bit converts `msgpack` to `json`. | `false` |
24+
| `sp.convert_from_str_to_num` | If enabled, the Stream Processor converts strings that represent numbers to a numeric type. | `true` |
2525

2626
## Configuration example
2727

28-
The following configuration example that defines a `service` section with [hot reloading](../../hot_reload.md) enabled and a pipeline with a `random` input and `stdout` output:
28+
The following configuration example that defines a `service` section with [hot reloading](../../hot-reload.md) enabled and a pipeline with a `random` input and `stdout` output:
2929

3030
```yaml
3131
service:
@@ -43,4 +43,4 @@ pipeline:
4343
outputs:
4444
- name: stdout
4545
match: '*'
46-
```
46+
```

‎administration/hot-reload.md

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,25 +15,27 @@ To get started with reloading over HTTP, enable the HTTP Server
1515
in the configuration file:
1616

1717
{% tabs %}
18+
{% tab title="fluent-bit.yaml" %}
19+
20+
```yaml
21+
service:
22+
http_server: on
23+
http_listen: 0.0.0.0
24+
http_port: 2020
25+
hot_reload: on
26+
```
27+
28+
{% endtab %}
1829
{% tab title="fluent-bit.conf" %}
30+
1931
```text
2032
[SERVICE]
21-
HTTP_Server On
22-
HTTP_Listen 0.0.0.0
23-
HTTP_PORT 2020
24-
Hot_Reload On
25-
...
33+
HTTP_Server On
34+
HTTP_Listen 0.0.0.0
35+
HTTP_PORT 2020
36+
Hot_Reload On
2637
```
27-
{% endtab %}
2838

29-
{% tab title="fluent-bit.yaml" %}
30-
```yaml
31-
service:
32-
http_server: on
33-
http_listen: 0.0.0.0
34-
http_port: 2020
35-
hot_reload: on
36-
```
3739
{% endtab %}
3840
{% endtabs %}
3941

@@ -51,7 +53,7 @@ Use the following HTTP endpoints to perform a hot reload:
5153

5254
For using curl to reload Fluent Bit, users must specify an empty request body as:
5355

54-
```text
56+
```shell
5557
curl -X POST -d '{}' localhost:2020/api/v2/reload
5658
```
5759

@@ -77,4 +79,4 @@ The endpoint returns `hot_reload_count` as follows:
7779
{"hot_reload_count":3}
7880
```
7981

80-
The default value of the counter is `0`.
82+
The default value of the counter is `0`.

‎administration/networking.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,9 +73,13 @@ plugin issues a retry.
7373

7474
### Listener backlog
7575

76-
When Fluent Bit listens for incoming connections (for example, in input plugins like HTTP, TCP, OpenTelemetry, Forward, Syslog, etc.), the operating system maintains a queue of pending connections. The `net.backlog` option controls the maximum number of pending connections that can be queued before new connection attempts are refused. Increasing this value can help Fluent Bit handle bursts of incoming connections more gracefully. The default value is `128`.
76+
When Fluent Bit listens for incoming connections (for example, in input plugins like HTTP, TCP, OpenTelemetry, Forward, and Syslog), the operating system maintains a queue of pending connections. The `net.backlog` option controls the maximum number of pending connections that can be queued before new connection attempts are refused. Increasing this value can help Fluent Bit handle bursts of incoming connections more gracefully. The default value is `128`.
7777

78-
> **Note:** On Linux, the effective backlog value may be capped by the kernel parameter `net.core.somaxconn`. If you need to allow a higher number of pending connections, you may need to increase this system setting.
78+
{% hint style="info" %}
79+
80+
On Linux, the effective backlog value might be capped by the kernel parameter `net.core.somaxconn`. If you need to allow a greater number of pending connections, you can increase this system setting.
81+
82+
{% endhint %}
7983

8084
## Configuration options
8185

@@ -95,7 +99,7 @@ that rely on networking I/O:
9599
| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it's retired. | `2000` |
96100
| `net.max_worker_connections` | Set maximum number of TCP connections that can be established per worker. | `0` (unlimited) |
97101
| `net.source_address` | Specify network address to bind for data traffic. | _none_ |
98-
| `net.backlog` | Set the maximum number of pending connections for listening sockets. This option is vailable on versions >= 4.0.4. | `128` |
102+
| `net.backlog` | Set the maximum number of pending connections for listening sockets. This option requires Fluent Bit version 4.0.4 or later. | `128` |
99103

100104
## Example
101105

@@ -183,4 +187,4 @@ If the `net.keepalive` option isn't enabled, Fluent Bit closes the TCP connectio
183187
and netcat quits.
184188

185189
After the five records arrive, the connection idles. After 10 seconds, the connection
186-
closes due to `net.keepalive_idle_timeout`.
190+
closes due to `net.keepalive_idle_timeout`.

‎installation/linux/debian.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Follow the official
3030
[Debian wiki guidance](https://wiki.debian.org/DebianRepository/UseThirdParty#OpenPGP_Key_distribution).
3131

3232
```bash copy
33-
curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg
33+
sudo sh -c 'curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg'
3434
```
3535

3636
### Updated key from March 2022
@@ -61,10 +61,10 @@ which platforms are supported in each release.
6161
## Update your sources lists
6262

6363
For Debian, you must add the Fluent Bit APT server entry to your sources lists.
64-
Add the following content at bottom of your `/etc/apt/sources.list` file.
64+
6565

6666
```bash copy
67-
deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/debian/${CODENAME} ${CODENAME} main
67+
echo "deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/ubuntu/${CODENAME} ${CODENAME} main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
6868
```
6969

7070
Replace _`CODENAME`_ with your specific

‎installation/linux/raspbian-raspberry-pi.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ The first step is to add the Fluent Bit server GPG key to your keyring so you
1414
can get FLuent Bit signed packages:
1515

1616
```shell
17-
curl https://packages.fluentbit.io/fluentbit.key | sudo apt-key add -
17+
sudo sh -c 'curl https://packages.fluentbit.io/fluentbit.key | sudo apt-key add - '
1818
```
1919

2020
### Updated key from March 2022
@@ -52,19 +52,19 @@ Add the following content at bottom of your `/etc/apt/sources.list` file.
5252
### Raspbian 12 (Bookworm)
5353

5454
```text
55-
deb https://packages.fluentbit.io/raspbian/bookworm bookworm main
55+
echo "deb https://packages.fluentbit.io/raspbian/bookworm bookworm main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
5656
```
5757

5858
### Raspbian 11 (Bullseye)
5959

6060
```text
61-
deb https://packages.fluentbit.io/raspbian/bullseye bullseye main
61+
echo "deb https://packages.fluentbit.io/raspbian/bullseye bullseye main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
6262
```
6363

6464
### Raspbian 10 (Buster)
6565

6666
```text
67-
deb https://packages.fluentbit.io/raspbian/buster buster main
67+
echo "deb https://packages.fluentbit.io/raspbian/buster buster main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
6868
```
6969

7070
### Update your repositories database

‎installation/linux/ubuntu.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Follow the official
2525
[Debian wiki guidance](https://wiki.debian.org/DebianRepository/UseThirdParty#OpenPGP_Key_distribution).
2626

2727
```bash
28-
curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg
28+
sudo sh -c 'curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg'
2929
```
3030

3131
### Updated key from March 2022
@@ -56,12 +56,11 @@ which platforms are supported in each release.
5656
## Update your sources lists
5757

5858
On Ubuntu, you need to add the Fluent Bit APT server entry to your sources lists.
59-
Add the following content at bottom of your `/etc/apt/sources.list` file. Ensure
60-
`CODENAME` is set to your specific [Ubuntu release name](https://wiki.ubuntu.com/Releases).
59+
Ensure `CODENAME` is set to your specific [Ubuntu release name](https://wiki.ubuntu.com/Releases).
6160
For example, `focal` for Ubuntu 20.04.
6261

6362
```bash
64-
deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/ubuntu/${CODENAME} ${CODENAME} main
63+
echo "deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/ubuntu/${CODENAME} ${CODENAME} main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
6564
```
6665

6766
### Update your repositories database

0 commit comments

Comments
 (0)