|
2 | 2 |
|
3 | 3 | Conditional processing allows you to selectively apply processors to log records based on field values. This feature enables you to create processing pipelines that apply processors only to records that match specific criteria. |
4 | 4 |
|
5 | | -## Configuration format |
| 5 | +## Configuration |
6 | 6 |
|
7 | | -Conditional processing is available for processors in the YAML configuration format. To apply a processor conditionally, you add a `condition` block to the processor configuration: |
| 7 | +You can turn a standard processor into a conditional processor by adding a `condition` block to the |
| 8 | +processor's YAML configuration settings. |
| 9 | + |
| 10 | +{% hint style="info" %} |
| 11 | +Conditional processing is only available for |
| 12 | +[YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md), |
| 13 | +not [classic configuration files](../../administration/configuring-fluent-bit/classic-mode/README.md). |
| 14 | +{% endhint %} |
| 15 | + |
| 16 | + |
| 17 | +These `condition` blocks use the following syntax: |
8 | 18 |
|
9 | 19 | ```yaml |
10 | | -- name: processor_name |
11 | | - # Regular processor configuration... |
12 | | - condition: |
13 | | - op: and|or |
14 | | - rules: |
15 | | - - field: "$field_name" |
16 | | - op: comparison_operator |
17 | | - value: comparison_value |
18 | | - # Additional rules... |
| 20 | +pipeline: |
| 21 | + inputs: |
| 22 | + <...> |
| 23 | + processors: |
| 24 | + logs: |
| 25 | + - name: {processor_name} |
| 26 | + <...> |
| 27 | + condition: |
| 28 | + op: {and|or} |
| 29 | + rules: |
| 30 | + - field: {field_name1} |
| 31 | + op: {comparison_operator} |
| 32 | + value: {comparison_value1} |
| 33 | + - field: {field_name2} |
| 34 | + op: {comparison_operator} |
| 35 | + value: {comparison_value2} |
| 36 | + <...> |
19 | 37 | ``` |
20 | 38 |
|
21 | | -### Condition operators |
| 39 | +Each processor can only have a single `condition` block, but can have multiple rules within that condition. |
| 40 | +These rules are stored as items in the `condition.rules` array. |
22 | 41 |
|
23 | | -The `op` field in the condition block specifies the logical operator to apply across all rules: |
| 42 | +### Condition evaluation |
24 | 43 |
|
25 | | -| Operator | Description | |
26 | | -| --- | --- | |
27 | | -| `and` | All rules must evaluate to true for the condition to be true | |
28 | | -| `or` | At least one rule must evaluate to true for the condition to be true | |
| 44 | +The `condition.op` parameter specifies the condition's evaluation logic. It has two possible values: |
| 45 | + |
| 46 | +- `and`: All rules in the `condition.rules` array must evaluate to `true` for the condition to be met. |
| 47 | +- `or`: One or more rules in the `conditions.rules` array must evaluate to `true` for the condition to be met. |
29 | 48 |
|
30 | 49 | ### Rules |
31 | 50 |
|
32 | | -Each rule consists of: |
| 51 | +Each item in the `condition.rules` array must include values for the following parameters: |
33 | 52 |
|
34 | | -- `field`: The field to evaluate (must use [record accessor syntax](/administration/configuring-fluent-bit/classic-mode/record-accessor.md) with `$` prefix) |
35 | | -- `op`: The comparison operator |
36 | | -- `value`: The value to compare against |
| 53 | +| Parameter | Description | |
| 54 | +| --- | --- | |
| 55 | +| `field` | The field within your logs to evaluate. The value of this parameter must use [the correct syntax](#field-access) to access the fields inside logs. | |
| 56 | +| `op` | The [comparison operator](#comparison-operators) to evaluate whether the rule is true. This parameter (`condition.rules.op`) is distinct from the `condition.op` parameter and has different possible values. | |
| 57 | +| `value` | The value of the specified log field to use in your comparison. Optionally, you can provide [an array that contains multiple values](#array-of-values). | |
37 | 58 |
|
38 | | -### Comparison operators |
| 59 | +Rules are evaluated against each log that passes through your data pipeline. For example, given a rule with these parameters: |
39 | 60 |
|
40 | | -The following comparison operators are supported: |
| 61 | +``` |
| 62 | +- field: "$status" |
| 63 | + op: eq |
| 64 | + value: 200 |
| 65 | +``` |
41 | 66 |
|
42 | | -| Operator | Description | |
43 | | -| --- | --- | |
44 | | -| `eq` | Equal to | |
45 | | -| `neq` | Not equal to | |
46 | | -| `gt` | Greater than | |
47 | | -| `lt` | Less than | |
48 | | -| `gte` | Greater than or equal to | |
49 | | -| `lte` | Less than or equal to | |
50 | | -| `regex` | Matches regular expression | |
51 | | -| `not_regex` | Does not match regular expression | |
52 | | -| `in` | Value is in the specified array | |
53 | | -| `not_in` | Value is not in the specified array | |
54 | | - |
55 | | -### Field access |
56 | | - |
57 | | -You can access record fields using [record accessor syntax](/administration/configuring-fluent-bit/classic-mode/record-accessor.md): |
58 | | - |
59 | | -- Simple fields: `$field` |
60 | | -- Nested fields: `$parent['child']['subchild']` |
| 67 | +This rule evaluates to `true` for a log that contains the string `'status':200`, but evaluates to `false` for a log that contains the string `'status':403`. |
| 68 | + |
| 69 | +#### Field access |
| 70 | + |
| 71 | +The `conditions.rules.field` parameter uses [record accessor syntax](/administration/configuring-fluent-bit/classic-mode/record-accessor.md) to reference fields inside logs. |
| 72 | + |
| 73 | +You can use `$field` syntax to access a top-level field, and `$field['child']['subchild']` to access nested fields. |
| 74 | + |
| 75 | +#### Comparison operators |
| 76 | + |
| 77 | +The `conditions.rules.op` parameter has the following possible values: |
| 78 | + |
| 79 | +- `eq`: equal to |
| 80 | +- `neq`: not equal to |
| 81 | +- `gt`: greater than |
| 82 | +- `lt`: less than |
| 83 | +- `gte`: greater than or equal to |
| 84 | +- `lte`: less than or equal to |
| 85 | +- `regex`: matches a regular expression |
| 86 | +- `not_regex`: does not match a regular expression |
| 87 | +- `in`: is included in the specified array |
| 88 | +- `not_in`: is not included in the specified array |
61 | 89 |
|
62 | 90 | ## Examples |
63 | 91 |
|
|
0 commit comments