Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

First of all, thanks for taking the time to read this guide. The fact that you're here means you're interested in contributing to Fluent Bit, and we greatly appreciate your time.

This repository contains the files for the [Fluent Bit documentation library](https://docs.fluentbit.io/). Keeping these docs separate from the [main Fluent Bit repository](https://github.com/fluent/fluent-bit) helps reduce the number of commits to the Fluent Bit source code and makes it easier to maintain both projects.
This repository contains the files for the [Fluent Bit documentation library](https://docs.fluentbit.io/manual/). Keeping these docs separate from the [main Fluent Bit repository](https://github.com/fluent/fluent-bit) helps reduce the number of commits to the Fluent Bit source code and makes it easier to maintain both projects.

Fluent Bit has a group of dedicated maintainers who oversee this repository, including several technical writers. These writers will review any pull requests you open, so don't be afraid to contribute, even if you're not a writer by trade. Your suggestions are valuable, and we'll help you wrangle any stray commas.

Expand Down Expand Up @@ -66,7 +66,7 @@ The active [linters](#linters) in this repository flag certain style errors and,

## Formatting guidelines

The Fluent Bit docs library is built and hosted through [GitBook](https://docs.gitbook.com/). Unfortunately, GitBook doesn't support local previews for contributors, but a Fluent Bit maintainer with a dedicated GitBook account can verify that things are formatted correctly after you open a new pull request.
The Fluent Bit docs library is built and hosted through [GitBook](https://gitbook.com/docs/). Unfortunately, GitBook doesn't support local previews for contributors, but a Fluent Bit maintainer with a dedicated GitBook account can verify that things are formatted correctly after you open a new pull request.

### Line wraps

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces

<figure><img src=".gitbook/assets/fluent_bit_logo.png" alt=""><figcaption></figcaption></figure>

[Fluent Bit](http://fluentbit.io) is a fast and lightweight telemetry agent for logs, metrics, and traces for Linux, macOS, Windows, and BSD family operating systems. Fluent Bit has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)
[Fluent Bit](https://fluentbit.io) is a fast and lightweight telemetry agent for logs, metrics, and traces for Linux, macOS, Windows, and BSD family operating systems. Fluent Bit has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)

## Features

Expand Down Expand Up @@ -39,7 +39,7 @@ For more details about changes in each release, refer to the [official release n

## Fluent Bit, Fluentd, and CNCF

Fluent Bit is a [CNCF](https://cncf.io) graduated sub-project under the umbrella of [Fluentd](http://fluentd.org).
Fluent Bit is a [CNCF](https://cncf.io) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org).

Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/) and is now sponsored by [Chronosphere](https://chronosphere.io/). As a CNCF-hosted project, it's a fully vendor-neutral and community-driven project.

Expand Down
2 changes: 1 addition & 1 deletion about/fluentd-and-fluent-bit.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ The following table describes a comparison of different areas of the projects:
| Performance | Medium Performance | High Performance |
| Dependencies | Built as a Ruby Gem, depends on other gems. | Zero dependencies, unless required by a plugin. |
| Plugins | Over 1,000 external plugins available. | Over 100 built-in plugins available. |
| License | [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0) | [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0) |
| License | [Apache License v2.0](https://apache.org/licenses/LICENSE-2.0) | [Apache License v2.0](https://apache.org/licenses/LICENSE-2.0) |

Both [Fluentd](https://www.fluentd.org) and [Fluent Bit](https://fluentbit.io) can work as Aggregators or Forwarders, and can complement each other or be used as standalone solutions.

Expand Down
4 changes: 2 additions & 2 deletions administration/aws-credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Plugins read the shared `config` file at `$AWS_CONFIG_FILE` (or `$HOME/.aws/conf
The shared settings evaluate in the following order:

| Setting | File | Description |
|---|---|---|
| --- | --- | --- |
| `credential_process` | `config` | Linux only. See [Sourcing credentials with an external process in the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html). |
| `aws_access_key_id`, `aws_secret_access_key`, `aws_session_token` | `credentials` | Access key ID and secret key to use to authenticate. The session token must be set for temporary credentials. |

Expand All @@ -33,7 +33,7 @@ Credentials are fetched using a signed web identity token for a Kubernetes servi

## ECS HTTP credentials endpoint

Credentials are fetched for the ECS task's role. See [Amazon ECS task IAM role](https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-iam-roles.html).
Credentials are fetched for the ECS task's role. See [Amazon ECS task IAM role](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html).

## EKS Pod Identity credentials

Expand Down
2 changes: 1 addition & 1 deletion administration/configuring-fluent-bit.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Fluent Bit uses configuration files to store information about your specified [i

## Unit sizes

Some configuration settings in Fluent Bit use standardized unit sizes to define data and storage limits. For example, the `buffer_chunk_size` and `buffer_max_size` parameters for the [Tail](../data-pipeline/inputs/tail.md) input plugin use unit sizes.
Some configuration settings in Fluent Bit use standardized unit sizes to define data and storage limits. For example, the `buffer_chunk_size` and `buffer_max_size` parameters for the [Tail](../pipeline/inputs/tail.md) input plugin use unit sizes.

The following table describes the unit sizes you can use and what they mean.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ For scheduler and retry details, see [scheduling and retries](../../scheduling-a

## Config input

The `INPUT` section defines a source (related to an input plugin). Each [input plugin](https://docs.fluentbit.io/manual/pipeline/inputs) can add its own configuration keys:
The `INPUT` section defines a source (related to an input plugin). Each [input plugin](../../../pipeline/inputs.md) can add its own configuration keys:

| Key | Description |
| ----------- | ------------|
Expand Down Expand Up @@ -160,4 +160,4 @@ Wildcard character (`*`) supports including multiple files. For example:

Files matching the wildcard character are included unsorted. If plugin ordering between files needs to be preserved, the files should be included explicitly.

Environment variables aren't supported in includes section. The file path must be specified as a literal string.
Environment variables aren't supported in the `includes` section. The path to the file must be specified as a literal string.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Upstream servers

Fluent Bit [output plugins](../../../pipeline/outputs/) aim to connect to external services to deliver logs over the network. Being able to connect to one node (host) is normal and enough for more of the use cases, but there are other scenarios where balancing across different nodes is required. The `Upstream` feature provides this capability.
Fluent Bit [output plugins](../../../pipeline/outputs.md) aim to connect to external services to deliver logs over the network. Being able to connect to one node (host) is normal and enough for more of the use cases, but there are other scenarios where balancing across different nodes is required. The `Upstream` feature provides this capability.

An `Upstream` defines a set of nodes that will be targeted by an output plugin, by the nature of the implementation an output plugin must support the `Upstream` feature. The following plugin has `Upstream` support:

Expand Down Expand Up @@ -60,4 +60,6 @@ The following example defines an `Upstream` called forward-balancing which aims
shared_key secret
```

Every `Upstream` definition must exists in its own configuration file in the file system. Adding multiple `Upstream` configurations in the same file or different files isn't allowed.
Every `Upstream` definition must exists in its own configuration file in the file
system. Adding multiple `Upstream` configurations in the same file or different files
isn't allowed.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Multiline parsers

You can define custom [multiline parsers](../../pipeline/parsers/multiline-parsing.md) in the `multiline_parsers` section of YAML configuration files.
You can define custom [multiline parsers](../../../pipeline/parsers/multiline-parsing.md) in the `multiline_parsers` section of YAML configuration files.

{% hint style="info" %}

Expand Down Expand Up @@ -34,4 +34,4 @@ multiline_parsers:

This example defines a multiline parser named `multiline-regex-test` that uses regular expressions to handle multi-event logs. The parser contains two rules: the first rule transitions from `start_state` to cont when a matching log entry is detected, and the second rule continues to match subsequent lines.

For information about supported configuration options for custom multiline parsers, see [configuring multiline parsers](../../pipeline/parsers/multiline-parsing.md#configuring-multiline-parsers).
For information about supported configuration options for custom multiline parsers, see [configuring multiline parsers](../../../pipeline/parsers/multiline-parsing.md#configuring-multiline-parsers).
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Parsers

You can define customer [parsers](../pipeline/parsers.md) in the `parsers` section of YAML configuration files.
You can define customer [parsers](../../../pipeline/parsers.md) in the `parsers` section of YAML configuration files.

{% hint style="info" %}

Expand Down Expand Up @@ -35,7 +35,7 @@ parsers:
{% endtab %}
{% endtabs %}

For information about supported configuration options for custom parsers, see [configuring parsers](../../pipeline/parsers/configuring-parser.md).
For information about supported configuration options for custom parsers, see [configuring custom parsers](../../../pipeline/parsers/configuring-parser.md).

## Standalone parsers files

Expand All @@ -47,10 +47,10 @@ To add a standalone parsers file to Fluent Bit, use the `parsers_file` parameter

To add a standalone parsers file to Fluent Bit, follow these steps.

1. Define custom parsers in a standalone YAML file. For example, `my-parsers.yaml` defines two custom parsers:
1. Define custom parsers in a standalone YAML file. For example, `custom-parsers.yaml` defines two custom parsers:

{% tabs %}
{% tab title="my-parsers.yaml" %}
{% tab title="custom-parsers.yaml" %}

```yaml
parsers:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ The `service` section of YAML configuration files defines global properties of t
| `dns.mode` | Sets the primary transport layer protocol used by the asynchronous DNS resolver. Can be overridden on a per-plugin basis. | `UDP` |
| `log_file` | Absolute path for an optional log file. By default, all logs are redirected to the standard error interface (`stderr`). | _none_ |
| `log_level` | Sets the logging verbosity level. Possible values: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. For example, if `debug` is set, it will include `error`, `warning`, `info`, and `debug`. The `trace` mode is only available if Fluent Bit was built with the `WITH_TRACE` option enabled. | `info` |
| `parsers_file` | Path for [standalone parsers configuration files](../administration/configuring-fluent-bit/yaml/parsers-section.md#standalone-parsers-files). You can include one or more files. | _none_ |
| `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to external plugins (.so files) that Fluent Bit can load at runtime. Plugins can also be declared directly in the [`plugins` section](../administration/configuring-fluent-bit/yaml/plugins-section.md) of YAML configuration files. | _none_ |
| `streams_file` | Path for the [stream processor](../stream-processing/overview.md) configuration file. This file defines the rules and operations for stream processing in Fluent Bit. Stream processor configurations can also be defined directly in the `streams` section of YAML configuration files. | _none_ |
| `parsers_file` | Path for [standalone parsers configuration files](../yaml/parsers-section.md#standalone-parsers-files). You can include one or more files. | _none_ |
| `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to external plugins (.so files) that Fluent Bit can load at runtime. Plugins can also be declared directly in the [`plugins` section](../yaml/plugins-section.md) of YAML configuration files. | _none_ |
| `streams_file` | Path for the [stream processor](../../../stream-processing/overview.md) configuration file. This file defines the rules and operations for stream processing in Fluent Bit. Stream processor configurations can also be defined directly in the `streams` section of YAML configuration files. | _none_ |
| `http_server` | Enables the built-in HTTP server. | `off` |
| `http_listen` | Sets the listening interface for the HTTP Server when it's enabled. | `0.0.0.0` |
| `http_port` | Sets the TCP port for the HTTP server. | `2020` |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Upstream servers

The `upstream_servers` section of YAML configuration files defines a group of endpoints, referred to as nodes. Nodes are used by output plugins to distribute data in a round-robin fashion. Use this section for plugins that require load balancing when sending data. Examples of plugins that support this capability include [Forward](https://docs.fluentbit.io/manual/pipeline/outputs/forward) and [Elasticsearch](https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch).
The `upstream_servers` section of YAML configuration files defines a group of endpoints, referred to as nodes. Nodes are used by output plugins to distribute data in a round-robin fashion. Use this section for plugins that require load balancing when sending data. Examples of plugins that support this capability include [Forward](../../../pipeline/outputs/forward.md) and [Elasticsearch](../../../pipeline/outputs/elasticsearch).

The `upstream_servers` section require specifying a `name` for the group and a list
of `nodes`. The following example defines two upstream server groups, `forward-balancing` and `forward-balancing-2`:
Expand Down
2 changes: 1 addition & 1 deletion administration/http-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The `HTTP_PROXY` environment variable is a [standard way](https://docs.docker.co

{% hint style="info" %}

The [HTTP output plugin](https://docs.fluentbit.io/manual/pipeline/outputs/http) also supports configuring an HTTP proxy. This configuration works, but shouldn't be used with the `HTTP_PROXY` or `http_proxy` environment variable. The environment variable-based proxy configuration is implemented by creating a TCP connection tunnel using [HTTP CONNECT](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT). Unlike the plugin's implementation, this supports both HTTP and HTTPS egress traffic.
The [HTTP output plugin](../pipeline/outputs/http.md) also supports configuring an HTTP proxy. This configuration works, but shouldn't be used with the `HTTP_PROXY` or `http_proxy` environment variable. The environment variable-based proxy configuration is implemented by creating a TCP connection tunnel using [HTTP CONNECT](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods/CONNECT). Unlike the plugin's implementation, this supports both HTTP and HTTPS egress traffic.

{% endhint %}

Expand Down
4 changes: 2 additions & 2 deletions administration/memory-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ When imposing a limit of `10MB` for the input plugins, and a worst case scenario

In intensive environments where memory allocations happen in the orders of magnitude, the default memory allocator provided by Glibc could lead to high fragmentation, reporting a high memory usage by the service.

It's strongly suggested that in any production environment, Fluent Bit should be built with [jemalloc](http://jemalloc.net/) enabled (`-DFLB_JEMALLOC=On`). The jemalloc implementation of `malloc` is an alternative memory allocator that can reduce fragmentation, resulting in better performance.
It's strongly suggested that in any production environment, Fluent Bit should be built with [jemalloc](https://jemalloc.net/) enabled (`-DFLB_JEMALLOC=On`). The jemalloc implementation of `malloc` is an alternative memory allocator that can reduce fragmentation, resulting in better performance.

Use the following command to determine if Fluent Bit has been built with jemalloc:

Expand All @@ -35,4 +35,4 @@ FLB_HAVE_PROXY_GO FLB_HAVE_JEMALLOC JEMALLOC_MANGLE FLB_HAVE_REGEX
FLB_HAVE_C_TLS FLB_HAVE_SETJMP FLB_HAVE_ACCEPT4 FLB_HAVE_INOTIFY
```

If the `FLB_HAVE_JEMALLOC` option is listed in `Build Flags`, jemalloc is enabled.
If the `FLB_HAVE_JEMALLOC` option is listed in `Build Flags`, jemalloc is enabled.
2 changes: 1 addition & 1 deletion administration/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,7 @@ When querying the related metrics, the aliases are returned instead of the plugi

You can create Grafana dashboards and alerts using Fluent Bit exposed Prometheus style metrics.

The provided [example dashboard](https://github.com/fluent/fluent-bit-docs/blob/master/monitoring/dashboard.json) is heavily inspired by [Banzai Cloud](https://github.com/banzaicloud)'s [logging operator dashboard](https://grafana.com/grafana/dashboards/7752) with a few key differences, such as the use of the `instance` label, stacked graphs, and a focus on Fluent Bit metrics. See [this blog post](https://www.robustperception.io/controlling-the-instance-label) for more information.
The provided [example dashboard](https://github.com/fluent/fluent-bit-docs/blob/master/monitoring/dashboard.json) is heavily inspired by [Banzai Cloud](https://github.com/banzaicloud)'s [logging operator dashboard](https://grafana.com/grafana/dashboards/7752-logging-dashboard) with a few key differences, such as the use of the `instance` label, stacked graphs, and a focus on Fluent Bit metrics. See [this blog post](https://www.robustperception.io/controlling-the-instance-label/) for more information.

![dashboard](/.gitbook/assets/dashboard.png)

Expand Down
Loading