Skip to content

Commit 3ada183

Browse files
authored
Merge branch 'master' into cosmo0920-add-descriptions-for-metadata-or-records-storing-parameters-on-in_splunk
Signed-off-by: Hiroshi Hatake <[email protected]>
2 parents 8aa4267 + 25e161d commit 3ada183

File tree

122 files changed

+1218
-365
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

122 files changed

+1218
-365
lines changed

.gitbook/assets/3.1.png

1.29 MB
Loading

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@ _book/*
22
node_modules/*
33
.gitignore
44
old/*
5-
5+
.DS_Store

README.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,15 @@
22
description: High Performance Telemetry Agent for Logs, Metrics and Traces
33
---
44

5-
# Fluent Bit v3.0 Documentation
5+
# Fluent Bit v3.1 Documentation
66

7-
<figure><img src=".gitbook/assets/3.0.png" alt=""><figcaption></figcaption></figure>
7+
<figure><img src=".gitbook/assets/3.1.png" alt=""><figcaption></figcaption></figure>
88

9-
[Fluent Bit](http://fluentbit.io) is a Fast and Lightweight **Telemetry Agent** for Logs, Metrics, and Traces for Linux, macOS, Windows, and BSD family operating systems. It has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)
9+
[Fluent Bit](http://fluentbit.io) is a fast and lightweight **telemetry agent**
10+
for logs, metrics, and traces for Linux, macOS, Windows, and BSD family
11+
operating systems. It has been made with a strong focus on performance to allow
12+
the collection and processing of telemetry data from different sources without
13+
complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)
1014

1115
## Features
1216

@@ -37,4 +41,7 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces
3741

3842
[Fluent Bit](http://fluentbit.io) is a [CNCF](https://cncf.io) **graduated** sub-project under the umbrella of [Fluentd](http://fluentd.org). Fluent Bit is licensed under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).
3943

40-
Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/). As a CNCF-hosted project, it is a fully **vendor-neutral** and community-driven project.
44+
Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/)
45+
and is now sponsored by [Chronosphere](https://chronosphere.io/). As a
46+
CNCF-hosted project, it is a fully **vendor-neutral** and community-driven
47+
project.

SUMMARY.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Table of contents
22

3-
* [Fluent Bit v3.0 Documentation](README.md)
3+
* [Fluent Bit v3.1 Documentation](README.md)
44

55
## About
66

@@ -67,6 +67,7 @@
6767
* [Networking](administration/networking.md)
6868
* [Memory Management](administration/memory-management.md)
6969
* [Monitoring](administration/monitoring.md)
70+
* [Multithreading](administration/multithreading.md)
7071
* [HTTP Proxy](administration/http-proxy.md)
7172
* [Hot Reload](administration/hot-reload.md)
7273
* [Troubleshooting](administration/troubleshooting.md)
@@ -131,7 +132,9 @@
131132
* [Decoders](pipeline/parsers/decoders.md)
132133
* [Processors](pipeline/processors/README.md)
133134
* [Content Modifier](pipeline/processors/content-modifier.md)
135+
* [Labels](pipeline/processors/labels.md)
134136
* [Metrics Selector](pipeline/processors/metrics-selector.md)
137+
* [OpenTelemetry Envelope](pipeline/processors/opentelemetry-envelope.md)
135138
* [SQL](pipeline/processors/sql.md)
136139
* [Filters](pipeline/filters/README.md)
137140
* [AWS Metadata](pipeline/filters/aws-metadata.md)

administration/backpressure.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Backpressure
22

3+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=63e37cfe-9ce3-4a18-933a-76b9198958c1" />
4+
35
Under certain scenarios it is possible for logs or data to be ingested or created faster than the ability to flush it to some destinations. One such common scenario is when reading from big log files, especially with a large backlog, and dispatching the logs to a backend over the network, which takes time to respond. This generates backpressure leading to high memory consumption in the service.
46

57
In order to avoid backpressure, Fluent Bit implements a mechanism in the engine that restricts the amount of data that an input plugin can ingest, this is done through the configuration parameters **Mem\_Buf\_Limit** and **storage.Max\_Chunks\_Up**.

administration/buffering-and-storage.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Buffering & Storage
22

3+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=cde12327-09ed-409c-ac02-7c0afa5eff51" />
4+
35
The end-goal of [Fluent Bit](https://fluentbit.io) is to collect, parse, filter and ship logs to a central place. In this workflow there are many phases and one of the critical pieces is the ability to do _buffering_ : a mechanism to place processed data into a temporary location until is ready to be shipped.
46

57
By default when Fluent Bit processes data, it uses Memory as a primary and temporary place to store the records, but there are certain scenarios where it would be ideal to have a persistent buffering mechanism based in the filesystem to provide aggregation and data safety capabilities.

administration/configuring-fluent-bit/classic-mode/configuration-file.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ description: This page describes the main configuration file used by Fluent Bit
44

55
# Configuration File
66

7+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5e67142e-3887-4b56-b940-18494bcc23a7" />
8+
79
One of the ways to configure Fluent Bit is using a main configuration file. Fluent Bit allows to use one configuration file which works at a global scope and uses the [Format and Schema](format-schema.md) defined previously.
810

911
The main configuration file supports four types of sections:

administration/configuring-fluent-bit/classic-mode/variables.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Variables
22

3+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=1731c7b5-34c6-424f-bfc6-88c2aa71e81f" />
4+
35
Fluent Bit supports the usage of environment variables in any value associated to a key when using a configuration file.
46

57
The variables are case sensitive and can be used in the following format:

administration/configuring-fluent-bit/multiline-parsing.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
In an ideal world, applications might log their messages within a single line, but in reality applications generate multiple log messages that sometimes belong to the same context. But when is time to process such information it gets really complex. Consider application stack traces which always have multiple log lines.
44

5+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=e19a4c14-a9e4-4163-8f3a-52196eb9a585" />
6+
57
Starting from Fluent Bit v1.8, we have implemented a unified Multiline core functionality to solve all the user corner cases. In this section, you will learn about the features and configuration options available.
68

79
## Concepts

administration/configuring-fluent-bit/yaml/configuration-file.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22
description: This page describes the yaml configuration file used by Fluent Bit
33
---
44

5+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=864c6f0e-8977-4838-8772-84416943548e" />
6+
57
# YAML Configuration File
68

79
One of the ways to configure Fluent Bit is using a YAML configuration file that works at a global scope.

0 commit comments

Comments
 (0)