|
1 | 1 | # Memory Management |
2 | 2 |
|
3 | | -In certain scenarios would be ideal to estimate how much memory Fluent Bit could be using, this is very useful for containerized environments where memory limits are a must. |
| 3 | +In certain scenarios it would be ideal to estimate how much memory Fluent Bit could be using, this is very useful for containerized environments where memory limits are a must. |
4 | 4 |
|
5 | | -In order to estimate we will assume that the input plugins have set the **Mem\_Buf\_Limit** option \(you can learn more about it in the [Backpressure](backpressure.md) section\). |
| 5 | +In order to that we will assume that the input plugins have set the **Mem\_Buf\_Limit** option \(you can learn more about it in the [Backpressure](backpressure.md) section\). |
6 | 6 |
|
7 | 7 | ## Estimating |
8 | 8 |
|
9 | | -Input plugins append data independently, so in order to do an estimation a limit should be imposed through the **Mem\_Buf\_Limit** option. If the limit was set to _10MB_ we need to estimate that in the worse case, the output plugin likely could use _20MB_. |
| 9 | +Input plugins append data independently, so in order to do an estimation, a limit should be imposed through the **Mem\_Buf\_Limit** option. If the limit was set to _10MB_ we need to estimate that in the worse case, the output plugin likely could use _20MB_. |
10 | 10 |
|
11 | | -Fluent Bit has an internal binary representation for the data being processed, but when this data reach an output plugin, this one will likely create their own representation in a new memory buffer for processing. The best example are the [InfluxDB](https://github.com/fluent/fluent-bit-docs/tree/b78cfe98123e74e165f2b6669229da009258f34e/output/influxdb.md) and [Elasticsearch](https://github.com/fluent/fluent-bit-docs/tree/b78cfe98123e74e165f2b6669229da009258f34e/output/elasticsearch.md) output plugins, both needs to convert the binary representation to their respective-custom JSON formats before to talk to their backend servers. |
| 11 | +Fluent Bit has an internal binary representation for the data being processed, but when this data reaches an output plugin, it will likely create its own representation in a new memory buffer for processing. The best examples are the [InfluxDB](https://github.com/fluent/fluent-bit-docs/tree/b78cfe98123e74e165f2b6669229da009258f34e/output/influxdb.md) and [Elasticsearch](https://github.com/fluent/fluent-bit-docs/tree/b78cfe98123e74e165f2b6669229da009258f34e/output/elasticsearch.md) output plugins, both need to convert the binary representation to their respective custom JSON formats before it can be sent to the backend servers. |
12 | 12 |
|
13 | | -So, if we impose a limit of _10MB_ for the input plugins and considering the worse case scenario of the output plugin consuming _20MB_ extra, as a minimum we need \(_30MB_ x 1.2\) = **36MB**. |
| 13 | +So, if we impose a limit of _10MB_ for the input plugins and consider the worse case scenario of the output plugin consuming _20MB_ extra, as a minimum we need \(_30MB_ x 1.2\) = **36MB**. |
14 | 14 |
|
15 | 15 | ## Glibc and Memory Fragmentation |
16 | 16 |
|
17 | | -Is well known that in intensive environments where memory allocations happens in the order of magnitude, the default memory allocator provided by Glibc could lead to a high fragmentation, reporting a high memory usage by the service. |
| 17 | +It is well known that in intensive environments where memory allocations happen in the orders of magnitude, the default memory allocator provided by Glibc could lead to high fragmentation, reporting a high memory usage by the service. |
18 | 18 |
|
19 | 19 | It's strongly suggested that in any production environment, Fluent Bit should be built with [jemalloc](http://jemalloc.net/) enabled \(e.g. `-DFLB_JEMALLOC=On`\). Jemalloc is an alternative memory allocator that can reduce fragmentation \(among others things\) resulting in better performance. |
20 | 20 |
|
21 | 21 | You can check if Fluent Bit has been built with Jemalloc using the following command: |
22 | 22 |
|
23 | 23 | ```text |
24 | | -$ bin/fluent-bit -h|grep JEMALLOC |
| 24 | +$ bin/fluent-bit -h | grep JEMALLOC |
25 | 25 | ``` |
26 | 26 |
|
27 | | -The output should looks like: |
| 27 | +The output should look like: |
28 | 28 |
|
29 | 29 | ```text |
30 | 30 | Build Flags = JSMN_PARENT_LINKS JSMN_STRICT FLB_HAVE_TLS FLB_HAVE_SQLDB |
|
0 commit comments