Skip to content

Commit dff3f09

Browse files
authored
Merge pull request #1999 from fluent/lynettemiles/sc-143290/remove-remaining-and-turn-on-vale-to-error
2 parents e1ba7f2 + 154355c commit dff3f09

File tree

5 files changed

+13
-13
lines changed

5 files changed

+13
-13
lines changed

administration/backpressure.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,17 @@
22

33
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=63e37cfe-9ce3-4a18-933a-76b9198958c1" />
44

5-
It's possible for logs or data to be ingested or created faster than the ability toflush it to some destinations. A common scenario is when reading from big log files, especially with a large backlog, and dispatching the logs to a backend over the network, which takes time to respond. This generates _backpressure_, leading to high memory consumption in the service.
5+
It's possible for logs or data to be ingested or created faster than the ability to flush it to some destinations. A common scenario is when reading from big log files, especially with a large backlog, and dispatching the logs to a backend over the network, which takes time to respond. This generates _backpressure_, leading to high memory consumption in the service.
66

77
To avoid backpressure, Fluent Bit implements a mechanism in the engine that restricts the amount of data an input plugin can ingest. Restriction is done through the configuration parameters `Mem_Buf_Limit` and `storage.Max_Chunks_Up`.
88

99
As described in the [Buffering](../concepts/buffering.md) concepts section, Fluent Bit offers two modes for data handling: in-memory only (default) and in-memory and filesystem (optional).
1010

1111
The default `storage.type memory` buffer can be restricted with `Mem_Buf_Limit`. If memory reaches this limit and you reach a backpressure scenario, you won't be able to ingest more data until the data chunks that are in memory can be flushed. The input pauses and Fluent Bit [emits](https://github.com/fluent/fluent-bit/blob/v2.0.0/src/flb_input_chunk.c#L1334) a `[warn] [input] {input name or alias} paused (mem buf overlimit)` log message.
1212

13-
Depending on the input plugin in use, this might cause incoming data to be discarded (for example, TCP input plugin). The tail plugin can handle pauses without data ingloss, storing its current file offset and resuming reading later. When buffer memory is available, the input resumes accepting logs. Fluent Bit [emits](https://github.com/fluent/fluent-bit/blob/v2.0.0/src/flb_input_chunk.c#L1277) a `[info] [input] {input name or alias} resume (mem buf overlimit)` message.
13+
Depending on the input plugin in use, this might cause incoming data to be discarded (for example, TCP input plugin). The tail plugin can handle pauses without data loss, storing its current file offset and resuming reading later. When buffer memory is available, the input resumes accepting logs. Fluent Bit [emits](https://github.com/fluent/fluent-bit/blob/v2.0.0/src/flb_input_chunk.c#L1277) a `[info] [input] {input name or alias} resume (mem buf overlimit)` message.
1414

15-
Mitigate the risk of data loss by configuring secondary storage on the filesystem using the `storage.type` of `filesystem` (as described in [Buffering & Storage](buffering-and-storage.md)). Initially, logs will be buffered to both memory and the filesystem. When the `storage.max_chunks_up` limit is reached, all new data will be stored in the filesystem. Fluent Bit stops queueing new data in memory and buffers only to the filesystem. When `storage.type filesystem` is set, the `Mem_Buf_Limit` setting no longer has any effect. Instead, the `[SERVICE]` level `storage.max_chunks_up` setting controls the size of the memory buffer.
15+
Mitigate the risk of data loss by configuring secondary storage on the filesystem using the `storage.type` of `filesystem` (as described in [Buffering and Storage](buffering-and-storage.md)). Initially, logs will be buffered to both memory and the filesystem. When the `storage.max_chunks_up` limit is reached, all new data will be stored in the filesystem. Fluent Bit stops queueing new data in memory and buffers only to the filesystem. When `storage.type filesystem` is set, the `Mem_Buf_Limit` setting no longer has any effect. Instead, the `[SERVICE]` level `storage.max_chunks_up` setting controls the size of the memory buffer.
1616

1717
## `Mem_Buf_Limit`
1818

@@ -38,7 +38,7 @@ In a few seconds, if the scheduler was able to flush the initial 700&nbsp;KB of
3838

3939
- Upon data buffer release (700&nbsp;KB), the internal counters get updated.
4040
- Counters now are set at 500&nbsp;KB.
41-
- Because 500&nbsp;KB isless than 1&nbsp;MB, it checks the input plugin state.
41+
- Because 500&nbsp;KB is less than 1&nbsp;MB, it checks the input plugin state.
4242
- If the plugin is paused, it invokes a `resume` callback.
4343
- The input plugin can continue appending more data.
4444

@@ -50,7 +50,7 @@ The setting behaves similar to the `Mem_Buf_Limit` scenario when the non-default
5050

5151
When (default) `storage.pause_on_chunks_overlimit` is disabled, the input won't pause when the memory limit is reached. Instead, it switches to buffering logs only in the filesystem. Limit the disk spaced used for filesystem buffering with `storage.total_limit_size`.
5252

53-
See [Buffering & Storage](buffering-and-storage.md) docs for more information.
53+
See [Buffering and Storage](buffering-and-storage.md) docs for more information.
5454

5555
## About pause and resume callbacks
5656

administration/transport-security.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ The following **output** plugins can take advantage of the TLS feature:
5151
- [Splunk](../pipeline/outputs/splunk.md)
5252
- [Stackdriver](../pipeline/outputs/stackdriver.md)
5353
- [Syslog](../pipeline/outputs/syslog.md)
54-
- [TCP & TLS](../pipeline/outputs/tcp-and-tls.md)
54+
- [TCO and TLS](../pipeline/outputs/tcp-and-tls.md)
5555
- [Treasure Data](../pipeline/outputs/treasure-data.md)
5656
- [WebSocket](../pipeline/outputs/websocket.md)
5757

@@ -92,7 +92,7 @@ By default, the HTTP input plugin uses plain TCP. Run the following command to e
9292
```
9393

9494
{% hint style="info" %}
95-
See Tips & Trick section below for details on generating `self_signed.crt` and `self_signed.key` files shown in these examples.
95+
See the Tips and Tricks section below for details on generating `self_signed.crt` and `self_signed.key` files shown in these examples.
9696
{% endhint %}
9797

9898
In the previous command, the two properties `tls` and `tls.verify` are set for demonstration purposes. Always enable verification in production environments.

concepts/buffering.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,4 @@ Fluent Bit buffering strategies are designed to solve problems associated with b
1414

1515
These mechanisms aren't mutually exclusive. When data is ready to be processed or delivered it's always be in memory, while other data in the queue might be in the file system until is ready to be processed and moved up to memory.
1616

17-
To learn more about the buffering configuration in Fluent Bit, see [Buffering & Storage](../administration/buffering-and-storage.md).
17+
To learn more about the buffering configuration in Fluent Bit, see [Buffering and Storage](../administration/buffering-and-storage.md).

pipeline/outputs/opentelemetry.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,9 +47,9 @@ The OpenTelemetry plugin works with logs and only the metrics collected from one
4747
{% tab title="fluent-bit.yaml" %}
4848

4949
```yaml
50-
# Dummy Logs & traces with Node Exporter Metrics export using OpenTelemetry output plugin
50+
# Dummy Logs and traces with Node Exporter Metrics export using OpenTelemetry output plugin
5151
# -------------------------------------------
52-
# The following example collects host metrics on Linux and dummy logs & traces and delivers
52+
# The following example collects host metrics on Linux and dummy logs and traces and delivers
5353
# them through the OpenTelemetry plugin to a local collector :
5454
#
5555
service:
@@ -95,9 +95,9 @@ pipeline:
9595
{% tab title="fluent-bit.conf" %}
9696
9797
```text
98-
# Dummy Logs & traces with Node Exporter Metrics export using OpenTelemetry output plugin
98+
# Dummy Logs and traces with Node Exporter Metrics export using OpenTelemetry output plugin
9999
# -------------------------------------------
100-
# The following example collects host metrics on Linux and dummy logs & traces and delivers
100+
# The following example collects host metrics on Linux and dummy logs and traces and delivers
101101
# them through the OpenTelemetry plugin to a local collector :
102102
#
103103
[SERVICE]

vale-styles/FluentBit/Ampersand.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
extends: existence
22
message: 'Use "and" instead of "&" in content.'
3-
level: warning
3+
level: error
44
scope: text
55
nonword: true
66
tokens:

0 commit comments

Comments
 (0)