You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit will add documentation for histogram mode and remove the sum mode, as it is not a specified option in the protocol.
Signed-off-by: Richard Treu <[email protected]>
Copy file name to clipboardExpand all lines: pipeline/filters/log-to-metrics.md
+90-37Lines changed: 90 additions & 37 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: Generate metrics from logs
4
4
5
5
# Log To Metrics
6
6
7
-
The _Log To Metrics Filter_ plugin allows you to generate log-derived metrics. It currently supports modes to count records, sum up field values over a record stream or provide a gauge for field values. You can also match or exclude specific records based on regular expression patterns for values or nested values. This filter plugin does not actually act as a record filter and does not change or drop records. All records will pass this filter untouched and generated metrics will be emitted into a seperate metric pipeline.
7
+
The _Log To Metrics Filter_ plugin allows you to generate log-derived metrics. It currently supports modes to count records, provide a gauge for field values or create a histogram. You can also match or exclude specific records based on regular expression patterns for values or nested values. This filter plugin does not actually act as a record filter and does not change or drop records. All records will pass this filter untouched and generated metrics will be emitted into a seperate metric pipeline.
8
8
9
9
_Please note that this plugin is an experimental feature and is not recommended for production use. Configuration parameters and plugin functionality are subject to change without notice._
10
10
@@ -16,11 +16,12 @@ The plugin supports the following configuration parameters:
16
16
| Key | Description | Mandatory | Value Format
17
17
| :--- | :--- | :--- | :---
18
18
| tag | Defines the tag for the generated metrics record| Yes ||
19
-
| metric_mode | Defines the mode for the metric. Valid values are [`counter`, `sum` or `gauge`]| Yes ||
19
+
| metric_mode | Defines the mode for the metric. Valid values are [`counter`, `gauge` or `histogram`]| Yes ||
20
20
| metric_name | Sets the name of the metric. | Yes ||
21
21
| metric_description | Sets a help text for the metric. | Yes ||
22
+
| bucket | Defines a bucket for `histogram`| Yes, for mode `histogram`| e.g. 0.75 |
22
23
| label_field | Includes a record field as label dimension in the metric. | | Name of record key. Supports [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields.
23
-
| value_field | Specify the record field that holds a numerical value to either sum up or take as most recent value | Yes, for modes [`sum` and `gauge`] | Name of record key. Supports [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields.
24
+
| value_field | Specify the record field that holds a numerical value | Yes, for modes [`gauge` and `histogram`] | Name of record key. Supports [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields.
24
25
| kubernetes_mode | If enabled, it will automatically put pod_id, pod_name, namespace_name, docker_id and container_name into the metric as labels. This option is intended to be used in combination with the [kubernetes](./kubernetes.md) filter plugin, which fills those fields. ||
25
26
| Regex | Include records in which the content of KEY matches the regular expression. | | KEY REGEX
26
27
| Exclude | Exclude records in which the content of KEY matches the regular expression. | | KEY REGEX
@@ -71,64 +72,67 @@ You can then use e.g. curl command to retrieve the generated metric:
71
72
log_metric_counter_count_all_dummy_messages 49
72
73
```
73
74
74
-
### Configuration - Sum
75
+
### Configuration - Gauge
75
76
76
-
If you want to sum up values within a record and provide the result as a metric, you have to specify a `value_field` to sum up. In this example we also add two labels via the `label_field` options:
77
+
The `gauge` mode needs a `value_field` specified, where the current metric values are generated from. In this example we also apply a regex filter and enable the `kubernetes_mode` option:
77
78
```python
78
79
[FILTER]
79
80
name log_to_metrics
80
81
match dummy.log*
81
82
tag test_metric
82
-
metric_mode sum
83
-
metric_name sum_up_durations
84
-
metric_description This metric sums up duration field values
83
+
metric_mode gauge
84
+
metric_name current_duration
85
+
metric_description This metric shows the current duration
85
86
value_field duration
87
+
kubernetes_mode on
88
+
regex message .*el.*
86
89
label_field color
87
90
label_field shape
88
91
```
89
-
90
92
You can then use e.g. curl command to retrieve the generated metric:
91
93
```text
92
94
> curl -s http://127.0.0.1:2021/metrics
93
95
94
96
95
-
# HELP log_metric_counter_sum_up_durations This metric sums up duration field values
96
-
# TYPE log_metric_counter_sum_up_durations counter
As you can see, the label sets defined by `label_field` are added to the metric. The lines in the metric represent every combination of labels. Only actually used combinations are displayed here. To see this, you can add a third `dummy` input (with "color": "blue") to your configuration:
As you can see in the output, only one line is printed, as the records from the first input plugin are ignored, as they do not match the regex.
103
+
104
+
The filter also allows to use multiple rules which are applied in order, you can have many _Regex_ and _Exclude_ entries as required (see [grep](./grep.md) filter plugin).
105
+
106
+
If you execute the above `curl` command multiple times, you see, that in this example the metric value stays at `60`, as the messages generated by the dummy plugin are not changing. In a real-world scenario the values would change and return the last processed value.
107
+
108
+
109
+
#### Metric label_values
110
+
As you can see, the label sets defined by `label_field` are added to the metric. The lines in the metric represent every combination of labels. Only actually used combinations are displayed here. To see this, you can add a dummy `dummy` input to your configuration.
109
111
110
112
The metric output would then look like:
111
113
```text
112
114
> curl -s http://127.0.0.1:2021/metrics
113
115
114
-
# HELP log_metric_counter_sum_up_durations This metric sums up duration field values
115
-
# TYPE log_metric_counter_sum_up_durations counter
You can also see, that all the kubernetes labels have been attached to the metric, accordingly.
124
+
125
+
### Configuration - Histogram
122
126
123
-
Similar to the `sum` mode, `gauge` needs a `value_field` specified, where the current metric values are generated from. In this example we also apply a regex filter and enable the `kubernetes_mode` option:
127
+
Similar to the `gauge` mode, `histogram` needs a `value_field` specified, where the current metric values are generated from. In this example we also apply a regex filter and enable the `kubernetes_mode` option:
124
128
```python
125
129
[FILTER]
126
130
name log_to_metrics
127
131
match dummy.log*
128
132
tag test_metric
129
-
metric_mode gauge
133
+
metric_mode histogram
130
134
metric_name current_duration
131
-
metric_description This metric shows the current duration
135
+
metric_description This metric shows the request duration
132
136
value_field duration
133
137
kubernetes_mode on
134
138
regex message .*el.*
@@ -140,15 +144,64 @@ You can then use e.g. curl command to retrieve the generated metric:
140
144
> curl -s http://127.0.0.1:2021/metrics
141
145
142
146
143
-
# HELP log_metric_gauge_current_duration This metric shows the current duration
As you can see in the output, only one line is printed, as the records from the first input plugin are ignored, as they do not match the regex.
179
+
As you can see in the output, there are per default the buckets `0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0` and `+Inf`, in which values are sorted into. A sum and a counter are also part of this metric. You can specify own buckets in the config, like in the following example:
149
180
150
-
The filter also allows to use multiple rules which are applied in order, you can have many _Regex_ and _Exclude_ entries as required (see [grep](./grep.md) filter plugin).
181
+
```python
182
+
[FILTER]
183
+
name log_to_metrics
184
+
match dummy.log*
185
+
tag test_metric
186
+
metric_mode histogram
187
+
metric_name current_duration
188
+
metric_description This metric shows the HTTP request duration as histogram in milliseconds
189
+
value_field duration
190
+
kubernetes_mode on
191
+
bucket 1
192
+
bucket 5
193
+
bucket 10
194
+
bucket 50
195
+
bucket 100
196
+
bucket 250
197
+
bucket 500
198
+
bucket 1000
199
+
regex message .*el.*
200
+
label_field color
201
+
label_field shape
202
+
```
203
+
204
+
Please note, that the `+Inf` bucket will always be included implicitly. The buckets in a histogram are cumulative, so a value added to one bucket will add to all larger buckets, too.
151
205
152
-
If you execute the above `curl` command multiple times, you see, that in this example the metric value stays at `60`, as the messages generated by the dummy plugin are not changing. In a real-world scenario the values would change and return the last processed value.
153
206
154
-
You can also see, that all the kubernetes labels have been attached to the metric, idential to the behavior of `label_field` described in [the previous chapter](#metric-label_values)
207
+
You can also see, that all the kubernetes labels have been attached to the metric, idential to the behavior of `label_field` described in [the previous chapter](#metric-label_values). That results in two sets for the histogram.
0 commit comments