Skip to content

Commit df63d75

Browse files
authored
Merge branch 'master' into lynettemiles/sc-113677/update-pipeline-outputs-elasticsearch-for
2 parents 0d549ad + 4e71960 commit df63d75

File tree

3 files changed

+71
-47
lines changed

3 files changed

+71
-47
lines changed

about/history.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,20 @@
11
---
2+
title: A brief history of Fluent Bit
23
description: Every project has a story
34
---
45

5-
# A Brief History of Fluent Bit
6+
# A brief history of Fluent Bit
67

7-
On 2014, the [Fluentd](https://www.fluentd.org/) team at [Treasure Data](https://www.treasuredata.com/) was forecasting the need for a lightweight log processor for constraint environments like Embedded Linux and Gateways, the project aimed to be part of the Fluentd Ecosystem; at that moment, Eduardo created [Fluent Bit](https://fluentbit.io/), a new open source solution written from scratch available under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).\
8+
In 2014, the [Fluentd](https://www.fluentd.org/) team at
9+
[Treasure Data](https://www.treasuredata.com/) was forecasting the need for a
10+
lightweight log processor for constraint environments like embedded Linux and
11+
gateways. The project aimed to be part of the Fluentd ecosystem. At that moment,
12+
Eduardo Silva created [Fluent Bit](https://fluentbit.io/), a new open source solution,
13+
written from scratch and available under the terms of the
14+
[Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).
815

9-
10-
After the project was around for some time, it got more traction for normal Linux systems, also with the new containerized world, the Cloud Native community asked to extend the project scope to support more sources, filters, and destinations. Not so long after, Fluent Bit became one of the preferred solutions to solve the logging challenges in Cloud environments.
16+
After the project matured, it gained traction for normal Linux systems. With the
17+
new containerized world, the Cloud Native community asked to extend the
18+
project scope to support more sources, filters, and destinations. Not long after,
19+
Fluent Bit became one of the preferred solutions to solve the logging challenges
20+
in Cloud environments.

about/license.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
---
2-
description: Strong Commitment to the Openness and Collaboration
2+
title: License
3+
description: Fluent Bit license description
34
---
45

56
# License
67

7-
[Fluent Bit](http://fluentbit.io), including its core, plugins and tools are distributed under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0):
8+
[Fluent Bit](http://fluentbit.io), including its core, plugins, and tools are
9+
distributed under the terms of the
10+
[Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0):
811

912
```text
1013
Apache License
@@ -184,4 +187,3 @@ description: Strong Commitment to the Openness and Collaboration
184187
185188
END OF TERMS AND CONDITIONS
186189
```
187-

pipeline/filters/record-modifier.md

Lines changed: 52 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,40 @@
11
# Record Modifier
22

3-
The _Record Modifier Filter_ plugin allows to append fields or to exclude specific fields.
3+
The _Record Modifier_ [filter](pipeline/filters.md) plugin lets you append
4+
fields to a record, or exclude specific fields.
45

5-
## Configuration Parameters
6+
## Configuration parameters
67

7-
The plugin supports the following configuration parameters: _Remove\_key_ and _Allowlist\_key_ are exclusive.
8+
The plugin supports the following configuration parameters:
89

910
| Key | Description |
1011
| :--- | :--- |
11-
| Record | Append fields. This parameter needs key and value pair. |
12-
| Remove\_key | If the key is matched, that field is removed. |
13-
| Allowlist\_key | If the key is **not** matched, that field is removed. |
14-
| Whitelist\_key | An alias of `Allowlist_key` for backwards compatibility. |
15-
| Uuid\_key| If set, the plugin appends uuid to each record. The value assigned becomes the key in the map.|
12+
| `Record` | Append fields. This parameter needs a key/value pair. |
13+
| `Remove_key` | If the key is matched, that field is removed. You can this or `Allowlist_key`.|
14+
| `Allowlist_key` | If the key isn't matched, that field is removed. You can this or `Remove_key`. |
15+
| `Whitelist_key` | An alias of `Allowlist_key` for backwards compatibility. |
16+
| `Uuid_key` | If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map. |
1617

17-
## Getting Started
18+
## Get started
1819

19-
In order to start filtering records, you can run the filter from the command line or through the configuration file.
20+
To start filtering records, run the filter from the command line or through a
21+
configuration file.
2022

21-
This is a sample in\_mem record to filter.
23+
This is a sample `in_mem` record to filter.
2224

2325
```text
2426
{"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724}
2527
```
2628

2729
### Append fields
2830

29-
The following configuration file is to append product name and hostname \(via environment variable\) to record.
31+
The following configuration file appends a product name and hostname to a record
32+
using an environment variable:
3033

3134
{% tabs %}
3235
{% tab title="fluent-bit.conf" %}
33-
```python
36+
37+
```python copy
3438
[INPUT]
3539
Name mem
3640
Tag mem.local
@@ -45,48 +49,50 @@ The following configuration file is to append product name and hostname \(via en
4549
Record hostname ${HOSTNAME}
4650
Record product Awesome_Tool
4751
```
52+
4853
{% endtab %}
4954

5055
{% tab title="fluent-bit.yaml" %}
51-
```yaml
56+
57+
```yaml copy
5258
pipeline:
5359
inputs:
5460
- name: mem
5561
tag: mem.local
5662
filters:
5763
- name: record_modifier
5864
match: '*'
59-
record:
65+
record:
6066
- hostname ${HOSTNAME}
6167
- product Awesome_Tool
6268
outputs:
6369
- name: stdout
6470
match: '*'
6571
```
72+
6673
{% endtab %}
6774
{% endtabs %}
6875
76+
You can run the filter from command line:
6977
70-
You can also run the filter from command line.
71-
72-
```text
73-
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
78+
```shell copy
79+
fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
7480
```
7581

76-
The output will be
82+
The output looks something like:
7783

78-
```python
84+
```python copy
7985
[0] mem.local: [1492436882.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724, "hostname"=>"localhost.localdomain", "product"=>"Awesome_Tool"}]
8086
```
8187

82-
### Remove fields with Remove\_key
83-
84-
The following configuration file is to remove 'Swap.\*' fields.
88+
### Remove fields with `Remove_key`
8589

90+
The following configuration file removes `Swap.*` fields:
8691

8792
{% tabs %}
8893
{% tab title="fluent-bit.conf" %}
89-
```python
94+
95+
```python copy
9096
[INPUT]
9197
Name mem
9298
Tag mem.local
@@ -102,47 +108,51 @@ The following configuration file is to remove 'Swap.\*' fields.
102108
Remove_key Swap.used
103109
Remove_key Swap.free
104110
```
111+
105112
{% endtab %}
106113

107114
{% tab title="fluent-bit.yaml" %}
108-
```yaml
115+
116+
```yaml copy
109117
pipeline:
110118
inputs:
111119
- name: mem
112120
tag: mem.local
113121
filters:
114122
- name: record_modifier
115123
match: '*'
116-
remove_key:
124+
remove_key:
117125
- Swap.total
118126
- Swap.used
119127
- Swap.free
120128
outputs:
121129
- name: stdout
122130
match: '*'
123131
```
132+
124133
{% endtab %}
125134
{% endtabs %}
126135
127136
You can also run the filter from command line.
128137
129-
```text
130-
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
138+
```shell copy
139+
fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
131140
```
132141

133-
The output will be
142+
The output looks something like:
134143

135144
```python
136145
[0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}]
137146
```
138147

139-
### Remove fields with Allowlist\_key
148+
### Retain fields with `Allowlist_key`
140149

141-
The following configuration file is to remain 'Mem.\*' fields.
150+
The following configuration file retains `Mem.*` fields.
142151

143152
{% tabs %}
144153
{% tab title="fluent-bit.conf" %}
145-
```python
154+
155+
```python copy
146156
[INPUT]
147157
Name mem
148158
Tag mem.local
@@ -158,37 +168,39 @@ The following configuration file is to remain 'Mem.\*' fields.
158168
Allowlist_key Mem.used
159169
Allowlist_key Mem.free
160170
```
171+
161172
{% endtab %}
162173

163174
{% tab title="fluent-bit.yaml" %}
164-
```yaml
175+
176+
```yaml copy
165177
pipeline:
166178
inputs:
167179
- name: mem
168180
tag: mem.local
169181
filters:
170182
- name: record_modifier
171183
match: '*'
172-
Allowlist_key:
184+
Allowlist_key:
173185
- Mem.total
174186
- Mem.used
175187
- Mem.free
176188
outputs:
177189
- name: stdout
178190
match: '*'
179191
```
192+
180193
{% endtab %}
181194
{% endtabs %}
182195
183-
You can also run the filter from command line.
196+
You can also run the filter from command line:
184197
185-
```text
186-
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
198+
```shell copy
199+
fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
187200
```
188201

189-
The output will be
202+
The output looks something like:
190203

191204
```python
192205
[0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}]
193206
```
194-

0 commit comments

Comments
 (0)