Skip to content

Commit 495b355

Browse files
esmerelesmerel
authored
Merge branch 'master' into out-es-custom-http-headers
Signed-off-by: esmerel <[email protected]>
2 parents 2293735 + 0d26e33 commit 495b355

File tree

181 files changed

+3552
-1160
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

181 files changed

+3552
-1160
lines changed

‎.gitbook/assets/3.1.png‎

1.29 MB
Loading

‎.gitbook/assets/image.png‎

-162 KB
Binary file not shown.

‎.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@ _book/*
22
node_modules/*
33
.gitignore
44
old/*
5-
5+
.DS_Store

‎.vale.ini‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
StylesPath = vale-styles
2+
MinAlertLevel = suggestion # suggestion, warning or error
3+
4+
[formats]
5+
mdx = md
6+
7+
[*.{md,markdown,txt,htm,html}]
8+
BasedOnStyles = FluentBit
9+
IgnoredScopes = tt, code
10+
11+
FluentBit.Terms = YES
12+
FluentBit.Titles = YES
13+
FluentBit.FutureTense = NO
14+
FluentBit.Headings = NO
15+
FluentBit.Passive = NO
16+
FluentBit.Subjunctive = NO
17+
FluentBit.Colons = NO

‎README.md‎

Lines changed: 43 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -2,39 +2,55 @@
22
description: High Performance Telemetry Agent for Logs, Metrics and Traces
33
---
44

5-
# Fluent Bit v3.0 Documentation
5+
# Fluent Bit v3.2 Documentation
66

7-
<figure><img src=".gitbook/assets/3.0.png" alt=""><figcaption></figcaption></figure>
7+
<figure><img src=".gitbook/assets/3.1.png" alt=""><figcaption></figcaption></figure>
88

9-
[Fluent Bit](http://fluentbit.io) is a Fast and Lightweight **Telemetry Agent** for Logs, Metrics, and Traces for Linux, macOS, Windows, and BSD family operating systems. It has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)
9+
[Fluent Bit](http://fluentbit.io) is a fast and lightweight telemetry agent
10+
for logs, metrics, and traces for Linux, macOS, Windows, and BSD family
11+
operating systems. Fluent Bit has been made with a strong focus on performance to allow
12+
the collection and processing of telemetry data from different sources without
13+
complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)
1014

1115
## Features
1216

13-
* High Performance: High throughput with low resources consumption
14-
* Data Parsing
15-
* Convert your unstructured messages using our parsers: [JSON](pipeline/parsers/json.md), [Regex](pipeline/parsers/regular-expression.md), [LTSV](pipeline/parsers/ltsv.md) and [Logfmt](pipeline/parsers/logfmt.md)
16-
* Metrics Support: Prometheus and OpenTelemetry compatible
17-
* Reliability and Data Integrity
18-
* [Backpressure](administration/backpressure.md) Handling
19-
* [Data Buffering](administration/buffering-and-storage.md) in memory and file system
20-
* Networking
21-
* Security: built-in TLS/SSL support
22-
* Asynchronous I/O
23-
* Pluggable Architecture and [Extensibility](development/library_api.md): Inputs, Filters and Outputs
24-
* Connect nearly any source to nearly any destination using preexisting plugins
25-
* Extensibility
26-
* Write any input, filter or output plugin in C language
27-
* WASM: [WASM Filter Plugins](development/wasm-filter-plugins.md) or [WASM Input Plugins](development/wasm-input-plugins.md)
28-
* Bonus: write [Filters in Lua](pipeline/filters/lua.md) or [Output plugins in Golang](development/golang-output-plugins.md)
29-
* [Monitoring](administration/monitoring.md): expose internal metrics over HTTP in JSON and [Prometheus](https://prometheus.io/) format
30-
* [Stream Processing](stream-processing/introduction.md): Perform data selection and transformation using simple SQL queries
31-
* Create new streams of data using query results
32-
* Aggregation Windows
33-
* Data analysis and prediction: Timeseries forecasting
34-
* Portable: runs on Linux, macOS, Windows and BSD systems
17+
- High performance: High throughput with low resources consumption
18+
- Data parsing
19+
- Convert your unstructured messages using our parsers:
20+
[JSON](pipeline/parsers/json.md),
21+
[Regex](pipeline/parsers/regular-expression.md),
22+
[LTSV](pipeline/parsers/ltsv.md) and [Logfmt](pipeline/parsers/logfmt.md)
23+
- Metrics support: Prometheus and OpenTelemetry compatible
24+
- Reliability and data integrity
25+
- [Backpressure](administration/backpressure.md) handling
26+
- [Data buffering](administration/buffering-and-storage.md) in memory and file system
27+
- Networking
28+
- Security: Built-in TLS/SSL support
29+
- Asynchronous I/O
30+
- Pluggable architecture and [extensibility](development/library_api.md): Inputs,
31+
Filters and Outputs:
32+
- Connect nearly any source to nearly any destination using preexisting plugins
33+
- Extensibility:
34+
- Write input, filter, or output plugins in the C language
35+
- WASM: [WASM Filter Plugins](development/wasm-filter-plugins.md) or
36+
[WASM Input Plugins](development/wasm-input-plugins.md)
37+
- Write [Filters in Lua](pipeline/filters/lua.md) or
38+
[Output plugins in Golang](development/golang-output-plugins.md)
39+
- [Monitoring](administration/monitoring.md): Expose internal metrics over HTTP
40+
in JSON and [Prometheus](https://prometheus.io/) format
41+
- [Stream Processing](stream-processing/introduction.md): Perform data selection
42+
and transformation using simple SQL queries
43+
- Create new streams of data using query results
44+
- Aggregation windows
45+
- Data analysis and prediction: Timeseries forecasting
46+
- Portable: Runs on Linux, macOS, Windows and BSD systems
3547

3648
## Fluent Bit, Fluentd and CNCF
3749

38-
[Fluent Bit](http://fluentbit.io) is a [CNCF](https://cncf.io) **graduated** sub-project under the umbrella of [Fluentd](http://fluentd.org). Fluent Bit is licensed under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).
50+
[Fluent Bit](http://fluentbit.io) is a [CNCF](https://cncf.io) graduated sub-project
51+
under the umbrella of [Fluentd](http://fluentd.org). Fluent Bit is licensed under
52+
the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).
3953

40-
Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/). As a CNCF-hosted project, it is a fully **vendor-neutral** and community-driven project.
54+
Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/)
55+
and is now sponsored by [Chronosphere](https://chronosphere.io/). As a
56+
CNCF-hosted project, it is a fully vendor-neutral and community-driven project.

‎SUMMARY.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Table of contents
22

3-
* [Fluent Bit v3.0 Documentation](README.md)
3+
* [Fluent Bit v3.2 Documentation](README.md)
44

55
## About
66

@@ -67,6 +67,7 @@
6767
* [Networking](administration/networking.md)
6868
* [Memory Management](administration/memory-management.md)
6969
* [Monitoring](administration/monitoring.md)
70+
* [Multithreading](administration/multithreading.md)
7071
* [HTTP Proxy](administration/http-proxy.md)
7172
* [Hot Reload](administration/hot-reload.md)
7273
* [Troubleshooting](administration/troubleshooting.md)
@@ -131,7 +132,9 @@
131132
* [Decoders](pipeline/parsers/decoders.md)
132133
* [Processors](pipeline/processors/README.md)
133134
* [Content Modifier](pipeline/processors/content-modifier.md)
135+
* [Labels](pipeline/processors/labels.md)
134136
* [Metrics Selector](pipeline/processors/metrics-selector.md)
137+
* [OpenTelemetry Envelope](pipeline/processors/opentelemetry-envelope.md)
135138
* [SQL](pipeline/processors/sql.md)
136139
* [Filters](pipeline/filters/README.md)
137140
* [AWS Metadata](pipeline/filters/aws-metadata.md)
@@ -184,6 +187,7 @@
184187
* [NATS](pipeline/outputs/nats.md)
185188
* [New Relic](pipeline/outputs/new-relic.md)
186189
* [NULL](pipeline/outputs/null.md)
190+
* [OpenObserve](pipeline/inputs/openobserve.md)
187191
* [Observe](pipeline/outputs/observe.md)
188192
* [Oracle Log Analytics](pipeline/outputs/oci-logging-analytics.md)
189193
* [OpenSearch](pipeline/outputs/opensearch.md)

‎about/fluentd-and-fluent-bit.md‎

Lines changed: 33 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,45 @@
11
---
2-
description: The Production Grade Telemetry Ecosystem
2+
description: The production grade telemetry ecosystem
33
---
44

5-
# Fluentd & Fluent Bit
5+
# Fluentd and Fluent Bit
66

7-
Telemetry data processing in general can be complex, and at scale a bit more, that's why [Fluentd](https://www.fluentd.org) was born. Fluentd has become more than a simple tool, it has grown into a fullscale ecosystem that contains SDKs for different languages and sub-projects like [Fluent Bit](https://fluentbit.io).
7+
Telemetry data processing can be complex, especially at scale. That's why
8+
[Fluentd](https://www.fluentd.org) was created. Fluentd is more than a simple tool,
9+
it's grown into a fullscale ecosystem that contains SDKs for different languages
10+
and subprojects like [Fluent Bit](https://fluentbit.io).
811

9-
On this page, we will describe the relationship between the [Fluentd](http://fluentd.org) and [Fluent Bit](http://fluentbit.io) open source projects, as a summary we can say both are:
12+
Here, we describe the relationship between the [Fluentd](http://fluentd.org)
13+
and [Fluent Bit](http://fluentbit.io) open source projects.
1014

11-
* Licensed under the terms of Apache License v2.0
12-
* **Graduated** Hosted projects by the [Cloud Native Computing Foundation (CNCF)](https://cncf.io)
13-
* Production Grade solutions: deployed **million** of times every single day.
14-
* **Vendor neutral** and community driven projects
15-
* Widely Adopted by the Industry: trusted by all major companies like AWS, Microsoft, Google Cloud and hundreds of others.
15+
Both projects are:
1616

17-
Both projects share a lot of similarities, [Fluent Bit](https://fluentbit.io) is fully designed and built on top of the best ideas of [Fluentd](https://www.fluentd.org) architecture and general design. Choosing which one to use depends on the end-user needs.
17+
- Licensed under the terms of Apache License v2.0.
18+
- Graduated hosted projects by the [Cloud Native Computing Foundation (CNCF)](https://cncf.io).
19+
- Production grade solutions: Deployed millions of times every single day.
20+
- Vendor neutral and community driven.
21+
- Widely adopted by the industry: Trusted by major companies like AWS, Microsoft,
22+
Google Cloud, and hundreds of others.
23+
24+
The projects have many similarities: [Fluent Bit](https://fluentbit.io) is
25+
designed and built on top of the best ideas of [Fluentd](https://www.fluentd.org)
26+
architecture and general design. Which one you choose depends on your end-users' needs.
1827

1928
The following table describes a comparison of different areas of the projects:
2029

21-
| | Fluentd | Fluent Bit |
22-
| ------------ | ----------------------------------------------------------------- | ----------------------------------------------------------------- |
23-
| Scope | Containers / Servers | Embedded Linux / Containers / Servers |
24-
| Language | C & Ruby | C |
25-
| Memory | > 60MB | \~1MB |
26-
| Performance | Medium Performance | High Performance |
27-
| Dependencies | Built as a Ruby Gem, it requires a certain number of gems. | Zero dependencies, unless some special plugin requires them. |
28-
| Plugins | More than 1000 external plugins are available | More than 100 built-in plugins are available |
30+
| Attribute | Fluentd | Fluent Bit |
31+
| ------------ | --------------------- | --------------------- |
32+
| Scope | Containers / Servers | Embedded Linux / Containers / Servers |
33+
| Language | C & Ruby | C |
34+
| Memory | Greater than 60&nbsp;MB | Approximately 1&nbsp;MB |
35+
| Performance | Medium Performance | High Performance |
36+
| Dependencies | Built as a Ruby Gem, depends on other gems. | Zero dependencies, unless required by a plugin. |
37+
| Plugins | Over 1,000 external plugins available. | Over 100 built-in plugins available. |
2938
| License | [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0) | [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0) |
3039

31-
Both [Fluentd](https://www.fluentd.org) and [Fluent Bit](https://fluentbit.io) can work as Aggregators or Forwarders, they both can complement each other or use them as standalone solutions.\
32-
\
33-
In the recent years, Cloud Providers switched from Fluentd to Fluent Bit for performance and compatibility reasons. Fluent Bit is now considered the **next generation** solution.
40+
Both [Fluentd](https://www.fluentd.org) and [Fluent Bit](https://fluentbit.io)
41+
can work as Aggregators or Forwarders, and can complement each other or be used
42+
as standalone solutions.
43+
44+
In the recent years, cloud providers have switched from Fluentd to Fluent Bit for
45+
performance and compatibility. Fluent Bit is now considered the next-generation solution.

‎about/history.md‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,20 @@
11
---
2+
title: A brief history of Fluent Bit
23
description: Every project has a story
34
---
45

5-
# A Brief History of Fluent Bit
6+
# A brief history of Fluent Bit
67

7-
On 2014, the [Fluentd](https://www.fluentd.org/) team at [Treasure Data](https://www.treasuredata.com/) was forecasting the need for a lightweight log processor for constraint environments like Embedded Linux and Gateways, the project aimed to be part of the Fluentd Ecosystem; at that moment, Eduardo created [Fluent Bit](https://fluentbit.io/), a new open source solution written from scratch available under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).\
8+
In 2014, the [Fluentd](https://www.fluentd.org/) team at
9+
[Treasure Data](https://www.treasuredata.com/) was forecasting the need for a
10+
lightweight log processor for constraint environments like embedded Linux and
11+
gateways. The project aimed to be part of the Fluentd ecosystem. At that moment,
12+
Eduardo Silva created [Fluent Bit](https://fluentbit.io/), a new open source solution,
13+
written from scratch and available under the terms of the
14+
[Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).
815

9-
10-
After the project was around for some time, it got more traction for normal Linux systems, also with the new containerized world, the Cloud Native community asked to extend the project scope to support more sources, filters, and destinations. Not so long after, Fluent Bit became one of the preferred solutions to solve the logging challenges in Cloud environments.
16+
After the project matured, it gained traction for normal Linux systems. With the
17+
new containerized world, the Cloud Native community asked to extend the
18+
project scope to support more sources, filters, and destinations. Not long after,
19+
Fluent Bit became one of the preferred solutions to solve the logging challenges
20+
in Cloud environments.

‎about/license.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
---
2-
description: Strong Commitment to the Openness and Collaboration
2+
title: License
3+
description: Fluent Bit license description
34
---
45

56
# License
67

7-
[Fluent Bit](http://fluentbit.io), including its core, plugins and tools are distributed under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0):
8+
[Fluent Bit](http://fluentbit.io), including its core, plugins, and tools are
9+
distributed under the terms of the
10+
[Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0):
811

912
```text
1013
Apache License
@@ -184,4 +187,3 @@ description: Strong Commitment to the Openness and Collaboration
184187
185188
END OF TERMS AND CONDITIONS
186189
```
187-

‎about/what-is-fluent-bit.md‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,24 @@ description: Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
44

55
# What is Fluent Bit?
66

7+
[Fluent Bit](https://fluentbit.io) is an open source telemetry agent specifically
8+
designed to efficiently handle the challenges of collecting and processing telemetry
9+
data across a wide range of environments, from constrained systems to complex cloud
10+
infrastructures. Managing telemetry data from various sources and formats can be a
11+
constant challenge, particularly when performance is a critical factor.
12+
13+
Rather than serving as a drop-in replacement, Fluent Bit enhances the observability
14+
strategy for your infrastructure by adapting and optimizing your existing logging
15+
layer, and adding metrics and traces processing. Fluent Bit supports a
16+
vendor-neutral approach, seamlessly integrating with other ecosystems such as
17+
Prometheus and OpenTelemetry. Trusted by major cloud providers, banks, and companies
18+
in need of a ready-to-use telemetry agent solution, Fluent Bit effectively manages
19+
diverse data sources and formats while maintaining optimal performance and keeping
20+
resource consumption low.
21+
22+
Fluent Bit can be deployed as an edge agent for localized telemetry data handling or
23+
utilized as a central aggregator/collector for managing telemetry data across
24+
multiple sources and environments.
725

826

9-
[Fluent Bit](https://fluentbit.io) is an open-source telemetry agent specifically designed to efficiently handle the challenges of collecting and processing telemetry data across a wide range of environments, from constrained systems to complex cloud infrastructures. Managing telemetry data from various sources and formats can be a constant challenge, particularly when performance is a critical factor.
10-
11-
Rather than serving as a drop-in replacement, Fluent Bit enhances the observability strategy for your infrastructure by adapting and optimizing your existing logging layer, as well as metrics and traces processing. Furthermore, Fluent Bit supports a vendor-neutral approach, seamlessly integrating with other ecosystems such as Prometheus and OpenTelemetry. Trusted by major cloud providers, banks, and companies in need of a ready-to-use telemetry agent solution, Fluent Bit effectively manages diverse data sources and formats while maintaining optimal performance.
12-
13-
Fluent Bit can be deployed as an edge agent for localized telemetry data handling or utilized as a central aggregator/collector for managing telemetry data across multiple sources and environments.
14-
15-
[Fluent Bit](https://fluentbit.io) has been designed with performance and low resource consumption in mind.
16-
1727
{% embed url="https://www.youtube.com/watch?v=3ELc1helke4" %}

0 commit comments

Comments
 (0)