Skip to content

Commit cb79196

Browse files
esmerelesmerel
committed
Pipeline: filters: modify: style updates
Signed-off-by: Lynette Miles <[email protected]>
1 parent a4e0583 commit cb79196

File tree

1 file changed

+66
-54
lines changed

1 file changed

+66
-54
lines changed

‎pipeline/filters/modify.md

Lines changed: 66 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# Modify
22

3-
The _Modify Filter_ plugin allows you to change records using rules and conditions.
3+
The _Modify Filter_ plugin lets you change records using rules and conditions.
44

55
## Example usage
66

7-
As an example using JSON notation to,
7+
As an example, use JSON notation to:
88

9-
* Rename `Key2` to `RenamedKey`
10-
* Add a key `OtherKey` with value `Value3` if `OtherKey` does not yet exist
9+
- Rename `Key2` to `RenamedKey`
10+
- Add a key `OtherKey` with value `Value3` if `OtherKey` doesn't yet exist
1111

12-
_Example \(input\)_
12+
Example (input):
1313

1414
```text
1515
{
@@ -18,7 +18,7 @@ _Example \(input\)_
1818
}
1919
```
2020

21-
_Example \(output\)_
21+
Example (output):
2222

2323
```text
2424
{
@@ -28,56 +28,58 @@ _Example \(output\)_
2828
}
2929
```
3030

31-
## Configuration Parameters
31+
## Configuration parameters
32+
33+
The Modify filter supports multiple parameters.
3234

3335
### Rules
3436

3537
The plugin supports the following rules:
3638

3739
| Operation | Parameter 1 | Parameter 2 | Description |
3840
| :--- | :--- | :--- | :--- |
39-
| Set | STRING:KEY | STRING:VALUE | Add a key/value pair with key `KEY` and value `VALUE`. If `KEY` already exists, _this field is overwritten_ |
40-
| Add | STRING:KEY | STRING:VALUE | Add a key/value pair with key `KEY` and value `VALUE` if `KEY` does not exist |
41-
| Remove | STRING:KEY | NONE | Remove a key/value pair with key `KEY` if it exists |
42-
| Remove\_wildcard | WILDCARD:KEY | NONE | Remove all key/value pairs with key matching wildcard `KEY` |
43-
| Remove\_regex | REGEXP:KEY | NONE | Remove all key/value pairs with key matching regexp `KEY` |
44-
| Rename | STRING:KEY | STRING:RENAMED\_KEY | Rename a key/value pair with key `KEY` to `RENAMED_KEY` if `KEY` exists AND `RENAMED_KEY` _does not exist_ |
45-
| Hard\_rename | STRING:KEY | STRING:RENAMED\_KEY | Rename a key/value pair with key `KEY` to `RENAMED_KEY` if `KEY` exists. If `RENAMED_KEY` already exists, _this field is overwritten_ |
46-
| Copy | STRING:KEY | STRING:COPIED\_KEY | Copy a key/value pair with key `KEY` to `COPIED_KEY` if `KEY` exists AND `COPIED_KEY` _does not exist_ |
47-
| Hard\_copy | STRING:KEY | STRING:COPIED\_KEY | Copy a key/value pair with key `KEY` to `COPIED_KEY` if `KEY` exists. If `COPIED_KEY` already exists, _this field is overwritten_ |
48-
| Move\_to\_start | WILDCARD:KEY | NONE | Move key/value pairs with keys matching KEY to the start of the message |
49-
| Move\_to\_end | WILDCARD:KEY | NONE | Move key/value pairs with keys matching KEY to the end of the message |
50-
51-
* Rules are case insensitive, parameters are not
52-
* Any number of rules can be set in a filter instance.
53-
* Rules are applied in the order they appear, with each rule operating on the result of the previous rule.
41+
| `Set` | `STRING:KEY` | `STRING:VALUE` | Add a key/value pair with key `KEY` and value `VALUE`. If `KEY` already exists, this field is overwritten. |
42+
| `Add` | `STRING:KEY` | `STRING:VALUE` | Add a key/value pair with key `KEY` and value `VALUE` if `KEY` doesn't exist. |
43+
| `Remove` | `STRING:KEY` | _none_ | Remove a key/value pair with key `KEY` if it exists. |
44+
| `Remove_wildcard` | `WILDCARD:KEY` | _none_ | Remove all key/value pairs with key matching wildcard `KEY`. |
45+
| `Remove_regex` | `REGEXP:KEY` | _none_ | Remove all key/value pairs with key matching regexp `KEY`. |
46+
| `Rename` | `STRING:KEY` | `STRING:RENAMED_KEY` | Rename a key/value pair with key `KEY` to `RENAMED_KEY` if `KEY` exists and `RENAMED_KEY` doesn't exist. |
47+
| `Hard_rename` | `STRING:KEY` | `STRING:RENAMED_KEY` | Rename a key/value pair with key `KEY` to `RENAMED_KEY` if `KEY` exists. If `RENAMED_KEY` already exists, this field is overwritten. |
48+
| `Copy` | `STRING:KEY` | `STRING:COPIED_KEY` | Copy a key/value pair with key `KEY` to `COPIED_KEY` if `KEY` exists and `COPIED_KEY` doesn't exist. |
49+
| `Hard_copy` | `STRING:KEY` | `STRING:COPIED_KEY` | Copy a key/value pair with key `KEY` to `COPIED_KEY` if `KEY` exists. If `COPIED_KEY` already exists, this field is overwritten. |
50+
| `Move_to_start` | `WILDCARD:KEY` | _none_ | Move key/value pairs with keys matching `KEY` to the start of the message. |
51+
| `Move_to_end` | `WILDCARD:KEY` | _none_ | Move key/value pairs with keys matching `KEY` to the end of the message. |
52+
53+
- Rules are case insensitive, parameters aren't.
54+
- Any number of rules can be set in a filter instance.
55+
- Rules are applied in the order they appear, with each rule operating on the result of the previous rule.
5456

5557
### Conditions
5658

5759
The plugin supports the following conditions:
5860

5961
| Condition | Parameter | Parameter 2 | Description |
6062
| :--- | :--- | :--- | :--- |
61-
| Key\_exists | STRING:KEY | NONE | Is `true` if `KEY` exists |
62-
| Key\_does\_not\_exist | STRING:KEY | NONE | Is `true` if `KEY` does not exist |
63-
| A\_key\_matches | REGEXP:KEY | NONE | Is `true` if a key matches regex `KEY` |
64-
| No\_key\_matches | REGEXP:KEY | NONE | Is `true` if no key matches regex `KEY` |
65-
| Key\_value\_equals | STRING:KEY | STRING:VALUE | Is `true` if `KEY` exists and its value is `VALUE` |
66-
| Key\_value\_does\_not\_equal | STRING:KEY | STRING:VALUE | Is `true` if `KEY` exists and its value is not `VALUE` |
67-
| Key\_value\_matches | STRING:KEY | REGEXP:VALUE | Is `true` if key `KEY` exists and its value matches `VALUE` |
68-
| Key\_value\_does\_not\_match | STRING:KEY | REGEXP:VALUE | Is `true` if key `KEY` exists and its value does not match `VALUE` |
69-
| Matching\_keys\_have\_matching\_values | REGEXP:KEY | REGEXP:VALUE | Is `true` if all keys matching `KEY` have values that match `VALUE` |
70-
| Matching\_keys\_do\_not\_have\_matching\_values | REGEXP:KEY | REGEXP:VALUE | Is `true` if all keys matching `KEY` have values that do not match `VALUE` |
71-
72-
* Conditions are case insensitive, parameters are not
73-
* Any number of conditions can be set.
74-
* Conditions apply to the whole filter instance and all its rules. _Not_ to individual rules.
75-
* All conditions have to be `true` for the rules to be applied.
76-
* You can set [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) as `STRING:KEY` for nested key.
77-
78-
## Example \#1 - Add and Rename
79-
80-
In order to start filtering records, you can run the filter from the command line or through the configuration file. The following invokes the [Memory Usage Input Plugin](../inputs/memory-metrics.md), which outputs the following \(example\),
63+
| `Key_exists` | `STRING:KEY` | _none_ | Is `true` if `KEY` exists. |
64+
| `Key_does_not_exist` | `STRING:KEY` | _none_ | Is `true` if `KEY` doesn't exist. |
65+
| `A_key_matches` | `REGEXP:KEY` | _none_ | Is `true` if a key matches regex `KEY`. |
66+
| `No_key_matches` | `REGEXP:KEY` | _none_ | Is `true` if no key matches regex `KEY`. |
67+
| `Key_value_equals` | `STRING:KEY` | `STRING:VALUE` | Is `true` if `KEY` exists and its value is `VALUE`. |
68+
| `Key_value_does_not_equal` | `STRING:KEY` | `STRING:VALUE` | Is `true` if `KEY` exists and its value isn't `VALUE`. |
69+
| `Key_value_matches` | `STRING:KEY` | `REGEXP:VALUE` | Is `true` if key `KEY` exists and its value matches `VALUE`. |
70+
| `Key_value_does_not_match` | `STRING:KEY` | `REGEXP:VALUE` | Is `true` if key `KEY` exists and its value doesn't match `VALUE`. |
71+
| `Matching_keys_have_matching_values` | `REGEXP:KEY` | `REGEXP:VALUE` | Is `true` if all keys matching `KEY` have values that match `VALUE`. |
72+
| `Matching_keys_do_not_have_matching_values` | `REGEXP:KEY` | `REGEXP:VALUE` | Is `true` if all keys matching `KEY` have values that don't match `VALUE`. |
73+
74+
- Conditions are case insensitive, parameters aren't.
75+
- Any number of conditions can be set.
76+
- Conditions apply to the whole filter instance and all its rules. _Not_ to individual rules.
77+
- All conditions have to be `true` for the rules to be applied.
78+
- You can set [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) as `STRING:KEY` for nested key.
79+
80+
## Example 1 - add and rename
81+
82+
To start filtering records, you can run the filter from the command line or through the configuration file. The following invokes the [Memory Usage Input Plugin](../inputs/memory-metrics.md), which outputs data similar to the following:
8183

8284
```text
8385
[0] memory: [1488543156, {"Mem.total"=>1016044, "Mem.used"=>841388, "Mem.free"=>174656, "Swap.total"=>2064380, "Swap.used"=>139888, "Swap.free"=>1924492}]
@@ -86,9 +88,9 @@ In order to start filtering records, you can run the filter from the command lin
8688
[3] memory: [1488543159, {"Mem.total"=>1016044, "Mem.used"=>841420, "Mem.free"=>174624, "Swap.total"=>2064380, "Swap.used"=>139888, "Swap.free"=>1924492}]
8789
```
8890

89-
### Using command Line
91+
### Using command line
9092

91-
> Note: Using the command line mode requires quotes parse the wildcard properly. The use of a configuration file is recommended.
93+
Using the command line mode requires quotes parse the wildcard properly. The use of a configuration file is recommended.
9294

9395
```text
9496
bin/fluent-bit -i mem \
@@ -105,10 +107,11 @@ bin/fluent-bit -i mem \
105107
-o stdout
106108
```
107109

108-
### Configuration File
110+
### Configuration file
109111

110112
{% tabs %}
111113
{% tab title="fluent-bit.conf" %}
114+
112115
```python
113116
[INPUT]
114117
Name mem
@@ -129,9 +132,11 @@ bin/fluent-bit -i mem \
129132
Rename Swap.total SWAPTOTAL
130133
Add Mem.total TOTALMEM
131134
```
135+
132136
{% endtab %}
133137

134138
{% tab title="fluent-bit.yaml" %}
139+
135140
```yaml
136141
pipeline:
137142
inputs:
@@ -153,12 +158,13 @@ pipeline:
153158
- name: stdout
154159
match: '*'
155160
```
161+
156162
{% endtab %}
157163
{% endtabs %}
158164
159165
### Result
160166
161-
The output of both the command line and configuration invocations should be identical and result in the following output.
167+
The output of both the command line and configuration invocations should be identical and result in the following output:
162168
163169
```text
164170
[2018/04/06 01:35:13] [ info] [engine] started
@@ -168,12 +174,13 @@ The output of both the command line and configuration invocations should be iden
168174
[3] mem.local: [1522980613.000122671, {"Mem.total"=>4050908, "MEMUSED"=>738068, "MEMFREE"=>3312840, "SWAPTOTAL"=>1046524, "Swap.used"=>0, "Swap.free"=>1046524, "Service1"=>"SOMEVALUE", "Service3"=>"SOMEVALUE3", "Mem.total2"=>"TOTALMEM2"}]
169175
```
170176
171-
## Example \#2 - Conditionally Add and Remove
177+
## Example 2 - conditionally add and remove
172178
173-
### Configuration File
179+
### Use a configuration file
174180
175181
{% tabs %}
176182
{% tab title="fluent-bit.conf" %}
183+
177184
```python
178185
[INPUT]
179186
Name mem
@@ -211,9 +218,11 @@ The output of both the command line and configuration invocations should be iden
211218
Name stdout
212219
Match *
213220
```
221+
214222
{% endtab %}
215223

216224
{% tab title="fluent-bit.yaml" %}
225+
217226
```yaml
218227
pipeline:
219228
inputs:
@@ -242,10 +251,11 @@ pipeline:
242251
- name: stdout
243252
match: '*'
244253
```
254+
245255
{% endtab %}
246256
{% endtabs %}
247257
248-
### Result
258+
### Add and remove result
249259
250260
```text
251261
[2018/06/14 07:37:34] [ info] [engine] started (pid=1493)
@@ -255,12 +265,13 @@ pipeline:
255265
[3] mem.local: [1528925858.000152319, {"cpustats"=>"UNKNOWN", "sourcetype"=>"memstats", "cpustats_more"=>"STILL_UNKNOWN"}]
256266
```
257267
258-
## Example \#3 - Emoji
268+
## Example 3 - emoji
259269
260-
### Configuration File
270+
### Emoji configuration File
261271
262272
{% tabs %}
263273
{% tab title="fluent-bit.conf" %}
274+
264275
```python
265276
[INPUT]
266277
Name mem
@@ -283,9 +294,11 @@ pipeline:
283294
Set ❄️ is_cold
284295
Set 💦 is_wet
285296
```
297+
286298
{% endtab %}
287299

288300
{% tab title="fluent-bit.yaml" %}
301+
289302
```yaml
290303
pipeline:
291304
inputs:
@@ -310,11 +323,11 @@ pipeline:
310323
- name: stdout
311324
match: '*'
312325
```
326+
313327
{% endtab %}
314328
{% endtabs %}
315329
316-
317-
### Result
330+
### Emoji example result
318331
319332
```text
320333
[2018/06/14 07:46:11] [ info] [engine] started (pid=21875)
@@ -324,4 +337,3 @@ pipeline:
324337
[3] mem.local: [1528926375.000090841, {"This_plugin_is_on"=>"🔥", "🔥"=>"is_hot", "❄️"=>"is_cold", "💦"=>"is_wet"}]
325338
[0] mem.local: [1528926376.000610974, {"This_plugin_is_on"=>"🔥", "🔥"=>"is_hot", "❄️"=>"is_cold", "💦"=>"is_wet"}]
326339
```
327-

0 commit comments

Comments
 (0)