Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Fluent Bit provides a set of predefined environment variables that can be used i

| Name | Description |
|--|--|
| `${HOSTNAME}` | The systems hostname. |
| `${HOSTNAME}` | The system's hostname. |

## External Variables

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The `pipeline` section defines the flow of how data is collected, processed, and

## Example Configuration

Heres a simple example of a pipeline configuration:
Here's a simple example of a pipeline configuration:

```yaml
pipeline:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ While Fluent Bit comes with a variety of built-in plugins, it also supports load

## 1. Inline YAML Section

You can specify external plugins directly within your main YAML configuration file using the `plugins` section. Heres an example:
You can specify external plugins directly within your main YAML configuration file using the `plugins` section. Here's an example:

```yaml
plugins:
Expand All @@ -22,9 +22,9 @@ pipeline:
match: '*'
```

## 2. YAML Plugins File Included via plugins_file Option
## 2. YAML Plugins File Included using the `plugins_file` Option

Alternatively, you can load external plugins from a separate YAML file by specifying the plugins_file option in the service section. Heres how to configure this:
Alternatively, you can load external plugins from a separate YAML file by specifying the `plugins_file` option in the service section. Here's how to configure this:

```yaml
service:
Expand All @@ -49,6 +49,5 @@ plugins:

### Key Points

- Built-in vs. External: Fluent Bit comes with many built-in plugins, but you can load external plugins at runtime to extend the tools functionality.
- Built-in versus External: Fluent Bit comes with many built-in plugins, but you can load external plugins at runtime to extend the tool's functionality.
- Loading Mechanism: External plugins must be shared object files (.so). You can define them inline in the main YAML configuration or include them from a separate YAML file for better modularity.

2 changes: 1 addition & 1 deletion administration/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ By increasing the buffer size, Fluent Bit will make fewer system calls (read(2))

## Fluent Bit and SIMD for JSON Encoding

Starting in Fluent Bit v3.2, performance improvements have been introduced for JSON encoding. Plugins that convert logs from Fluent Bits internal binary representation to JSON can now do so up to 30% faster using SIMD (Single Instruction, Multiple Data) optimizations.
Starting in Fluent Bit v3.2, performance improvements have been introduced for JSON encoding. Plugins that convert logs from Fluent Bit's internal binary representation to JSON can now do so up to 30% faster using SIMD (Single Instruction, Multiple Data) optimizations.

### Enabling SIMD Support

Expand Down
2 changes: 1 addition & 1 deletion development/msgpack-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ MessagePack is a standard and well-defined format, refer to the official documen
This section provides an overview of the specific types used by Fluent Bit within the format to help anyone consuming it.

* The data structure used by Fluent Bit is a 2-length [`fixarray`](https://github.com/msgpack/msgpack/blob/master/spec.md#array-format-family) of the timestamp and the data.
* The timestamp comes from [`flb_time_append_to_msgpack`])(https://github.com/fluent/fluent-bit/blob/2138cee8f4878733956d42d82f6dcf95f0aa9339/src/flb_time.c#L197), so its either a `uint64`, a `float64`, or a [`fixext`](https://github.com/msgpack/msgpack/blob/master/spec.md#ext-format-family) where the 4 MSBs are the seconds (big-endian `uint32`) and 4 LSBs are nanoseconds.
* The timestamp comes from [`flb_time_append_to_msgpack`])(https://github.com/fluent/fluent-bit/blob/2138cee8f4878733956d42d82f6dcf95f0aa9339/src/flb_time.c#L197), so it's either a `uint64`, a `float64`, or a [`fixext`](https://github.com/msgpack/msgpack/blob/master/spec.md#ext-format-family) where the 4 MSBs are the seconds (big-endian `uint32`) and 4 LSBs are nanoseconds.
* The data itself is just a [`msgpack` map](https://github.com/msgpack/msgpack/blob/master/spec.md#map-format-family) with the keys as strings.

## Example
Expand Down
4 changes: 2 additions & 2 deletions development/wasm-filter-plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ For Rust:
// #[no_mangle] attribute is needed for preventing mangles and align C ABI.
// Also we can use an arbitrary function name for filter operations w/ WASM.
#[no_mangle]
pub extern “C” fn rust_filter(tag: *const c_char, tag_len: u32, time_sec: u32, time_nsec: u32, record: *const c_char, record_len: u32)
pub extern "C" fn rust_filter(tag: *const c_char, tag_len: u32, time_sec: u32, time_nsec: u32, record: *const c_char, record_len: u32)
```

Note that `//export XXX` on TinyGo and `#[no_mangle]` attributes on Rust are required. This is because TinyGo and Rust will mangle their function names if they are not specified.
Expand Down Expand Up @@ -158,7 +158,7 @@ The output after the filter logic will be:
```text
[0] iis.*: [[1692131925.559486675, {}], {"c_authorization_header"=>"-", "c_ip"=>"::1", "cs_bytes"=>756, "cs_cookie"=>"-", "cs_host"=>"localhost", "cs_method"=>"GET", "cs_referer"=>"-", "cs_uri_query"=>"-", "cs_uri_stem"=>"/", "cs_user_agent"=>"Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/115.0.0.0+Safari/537.36+Edg/115.0.1901.200", "date"=>"2023-08-11 19:56:44", "s_computername"=>"WIN-PC1", "s_ip"=>"::1", "s_port"=>"80", "s_sitename"=>"W3SVC1", "sc_bytes"=>142, "sc_status"=>"304", "source"=>"LogEntryIIS", "tag"=>"iis.*", "time"=>"2023-08-15T20:38:45.559486675 +0000", "time_taken"=>1078}]
```
This filter approach provides us with several powerful advantages inherent to programming languages.
This filter approach provides us with several powerful advantages inherent to programming languages.
For instance, it:
- Can be extended by adding type conversion to fields such as `sc_bytes, cs_bytes, time_taken`. This is particularly useful when we need to validate our data results.
- Allows for the use of conditions to apply more descriptive filters, for example, "get only all logs that contain status codes above 4xx or 5xx".
Expand Down
2 changes: 1 addition & 1 deletion pipeline/outputs/vivo-exporter.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Here is a simple configuration of Vivo Exporter, note that this example is not b

### How it works

Vivo Exporter provides buffers that serve as streams for each telemetry data type, in this case, `logs`, `metrics`, and `traces`. Each buffer contains a fixed capacity in terms of size (20M by default). When the data arrives at a stream, its appended to the end. If the buffer is full, it removes the older entries to make room for new data.
Vivo Exporter provides buffers that serve as streams for each telemetry data type, in this case, `logs`, `metrics`, and `traces`. Each buffer contains a fixed capacity in terms of size (20M by default). When the data arrives at a stream, it's appended to the end. If the buffer is full, it removes the older entries to make room for new data.

The `data` that arrives is a `chunk`. A chunk is a group of events that belongs to the same type (logs, metrics or traces) and contains the same `tag`. Every chunk placed in a stream is assigned with an auto-incremented `id`.

Expand Down