diff --git a/administration/memory-management.md b/administration/memory-management.md index afd6e1c71..5289bb6e1 100644 --- a/administration/memory-management.md +++ b/administration/memory-management.md @@ -1,30 +1,46 @@ -# Memory Management +# Memory management -In certain scenarios it would be ideal to estimate how much memory Fluent Bit could be using, this is very useful for containerized environments where memory limits are a must. +You might need to estimate how much memory Fluent Bit could be using in scenarios +like containerized environments where memory limits are essential. -In order to that we will assume that the input plugins have set the **Mem\_Buf\_Limit** option \(you can learn more about it in the [Backpressure](backpressure.md) section\). +To make an estimate, in-use input plugins must set the `Mem_Buf_Limit`option. +Learn more about it in [Backpressure](backpressure.md). ## Estimating -Input plugins append data independently, so in order to do an estimation, a limit should be imposed through the **Mem\_Buf\_Limit** option. If the limit was set to _10MB_ we need to estimate that in the worse case, the output plugin likely could use _20MB_. +Input plugins append data independently. To make an estimation, impose a limit with +the `Mem_Buf_Limit` option. If the limit was set to `10MB`, you can estimate that in +the worst case, the output plugin likely could use `20MB`. -Fluent Bit has an internal binary representation for the data being processed, but when this data reaches an output plugin, it will likely create its own representation in a new memory buffer for processing. -The best examples are the [InfluxDB](../pipeline/outputs/influxdb.md) and [Elasticsearch](../pipeline/outputs/elasticsearch.md) output plugins, both need to convert the binary representation to their respective custom JSON formats before it can be sent to the backend servers. +Fluent Bit has an internal binary representation for the data being processed. When +this data reaches an output plugin, it can create its own representation in a new +memory buffer for processing. The best examples are the +[InfluxDB](../pipeline/outputs/influxdb.md) and +[Elasticsearch](../pipeline/outputs/elasticsearch.md) output plugins, which need to +convert the binary representation to their respective custom JSON formats before +sending data to the backend servers. -So, if we impose a limit of _10MB_ for the input plugins and consider the worse case scenario of the output plugin consuming _20MB_ extra, as a minimum we need \(_30MB_ x 1.2\) = **36MB**. +When imposing a limit of `10MB` for the input plugins, and a worst case scenario of +the output plugin consuming `20MB`, you need to allocate a minimum (`30MB` x 1.2) = +`36MB`. -## Glibc and Memory Fragmentation +## Glibc and memory fragmentation -It is well known that in intensive environments where memory allocations happen in the orders of magnitude, the default memory allocator provided by Glibc could lead to high fragmentation, reporting a high memory usage by the service. +In intensive environments where memory allocations happen in the orders of magnitude, +the default memory allocator provided by Glibc could lead to high fragmentation, +reporting a high memory usage by the service. -It's strongly suggested that in any production environment, Fluent Bit should be built with [jemalloc](http://jemalloc.net/) enabled \(e.g. `-DFLB_JEMALLOC=On`\). Jemalloc is an alternative memory allocator that can reduce fragmentation \(among others things\) resulting in better performance. +It's strongly suggested that in any production environment, Fluent Bit should be +built with [jemalloc](http://jemalloc.net/) enabled (`-DFLB_JEMALLOC=On`). +The jemalloc implementation of malloc is an alternative memory allocator that can +reduce fragmentation, resulting in better performance. -You can check if Fluent Bit has been built with Jemalloc using the following command: +Use the following command to determine if Fluent Bit has been built with jemalloc: -```text -$ bin/fluent-bit -h | grep JEMALLOC +```bash +bin/fluent-bit -h | grep JEMALLOC ``` The output should look like: @@ -36,5 +52,4 @@ FLB_HAVE_PROXY_GO FLB_HAVE_JEMALLOC JEMALLOC_MANGLE FLB_HAVE_REGEX FLB_HAVE_C_TLS FLB_HAVE_SETJMP FLB_HAVE_ACCEPT4 FLB_HAVE_INOTIFY ``` -If the FLB\_HAVE\_JEMALLOC option is listed in _Build Flags_, everything will be fine. - +If the `FLB_HAVE_JEMALLOC` option is listed in `Build Flags`, jemalloc is enabled. diff --git a/vale-styles/FluentBit/Spelling-exceptions.txt b/vale-styles/FluentBit/Spelling-exceptions.txt index f2da97b32..55d7e92fd 100644 --- a/vale-styles/FluentBit/Spelling-exceptions.txt +++ b/vale-styles/FluentBit/Spelling-exceptions.txt @@ -57,6 +57,7 @@ Fargate Firehose FluentBit Fluentd +glibc Golang golib Grafana @@ -72,6 +73,7 @@ Hostname Ingester Keepalive Istio +jemalloc keepalive Kinesis kubectl