Skip to content

Commit c824e47

Browse files
eschabellesmerel
eschabell
and
esmerel
authored
Updated log to metrics filter configuration options table to support v4.1. Fixes #2092. (#2124)
* Removing a readme file this is not standardized in classic-mode directory. Signed-off-by: Eric D. Schabell <[email protected]> * Updated to not use readme file in classic-config directory, but the correct description file. Signed-off-by: Eric D. Schabell <[email protected]> * Updated to better sort some sections and update for missing entries. Fixes #2108. Signed-off-by: Eric D. Schabell <[email protected]> * Updated configuration variables for opentelemetry output plugin, many new ones added. Signed-off-by: Eric D. Schabell <[email protected]> * Updated configuration variables for opentelemetry output plugin, many new ones added. Fixes #2083 Signed-off-by: Eric D. Schabell <[email protected]> * Fixes reqeusted by reviewers for options table for opentelemetry output plugin. Signed-off-by: Eric D. Schabell <[email protected]> * Updated TLS doc configuration options table to match v4.1 release. Fixes #2088. Signed-off-by: Eric D. Schabell <[email protected]> * Updated log to metrics filter configuration options table to support v4.1. Fixes #2092. Signed-off-by: Eric D. Schabell <[email protected]> * Fixes to review comments, removed output references and corrected bucket variable description. Applys to #2124. Signed-off-by: Eric D. Schabell <[email protected]> * Update pipeline/filters/log_to_metrics.md Grammar issue in option description fixed. Co-authored-by: Lynette Miles <[email protected]> Signed-off-by: Eric D. Schabell <[email protected]> * Update pipeline/filters/log_to_metrics.md Grammar issue in option description fixed. Co-authored-by: Lynette Miles <[email protected]> Signed-off-by: Eric D. Schabell <[email protected]> * Update pipeline/filters/log_to_metrics.md Forgot the ending period in option description! Co-authored-by: Lynette Miles <[email protected]> Signed-off-by: Eric D. Schabell <[email protected]> * Update pipeline/filters/log_to_metrics.md Better clarity in option description added. Co-authored-by: Lynette Miles <[email protected]> Signed-off-by: Eric D. Schabell <[email protected]> * Update pipeline/filters/log_to_metrics.md Grammar fix to option description. Co-authored-by: Lynette Miles <[email protected]> Signed-off-by: Eric D. Schabell <[email protected]> --------- Signed-off-by: Eric D. Schabell <[email protected]> Co-authored-by: Lynette Miles <[email protected]>
1 parent 82a66dc commit c824e47

File tree

1 file changed

+29
-18
lines changed

1 file changed

+29
-18
lines changed

‎pipeline/filters/log_to_metrics.md‎

Lines changed: 29 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,11 @@ description: Generate metrics from logs
44

55
# Logs to metrics
66

7-
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=768830f6-8d2d-4231-9e5e-259ce6797ba5" />
7+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=768830f6-8d2d-4231-9e5e-259ce6797ba5" alt="tracking"/>
88

99
The _log to metrics_ filter lets you generate log-derived metrics. It supports modes to count records, provide a gauge for field values, or create a histogram. You can also match or exclude specific records based on regular expression patterns for values or nested values.
1010

11-
This filter doesn't actually act as a record filter and therefore doesn't change or drop records. All records will pass through this filter untouched, and any generated metrics will be emitted into a separate metric pipeline.
11+
This filter doesn't actually act as a record filter and therefore does not change or drop records. All records will pass through this filter untouched, and any generated metrics will be emitted into a separate metric pipeline.
1212

1313
{% hint style="warning" %}
1414

@@ -20,21 +20,32 @@ This filter is an experimental feature and isn't recommended for production use.
2020

2121
The plugin supports the following configuration parameters:
2222

23-
| Parameter | Description | Value format |
24-
|---|---|---|
25-
| `tag` | Required. Defines the tag for the generated metrics record. | |
26-
| `metric_mode` | Required. Defines the mode for the metric. Valid values are `counter`, `gauge` or `histogram`. | |
27-
| `metric_name` | Required. Sets the name of the metric. | |
28-
| `metric_description` | Required. Sets a description for the metric. | |
29-
| `bucket` | Required for mode `histogram`. Defines a bucket for histograms. | For example, `0.75` |
30-
| `add_label` | Adds a custom label `NAME` and set the value to the value of `KEY`. | |
31-
| `label_field` | Includes a record field as label dimension in the metric. | Name of record key. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields. |
32-
| `value_field` | Required for modes `gauge` and `histogram`. Specifies the record field that holds a numerical value. | Name of record key. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields. |
33-
| `kubernetes_mode` | If enabled, adds `pod_id`, `pod_name`, `namespace_name`, `docker_id` and `container_name` to the metric as labels. This option is intended to be used in combination with the [Kubernetes](./kubernetes.md) filter plugin, which fills those fields. | |
34-
| `Regex` | Includes records in which the content of `KEY` matches the regular expression. | `KEY REGEX` |
35-
| `Exclude` | Excludes records in which the content of `KEY` matches the regular expression. | `KEY REGEX` |
36-
| `Flush_Interval_Sec` | The interval for metrics emission, in seconds. If `Flush_Interval_Sec` and `Flush_Interval_Nsec` are either both unset or both set to `0`, the filter emits metrics immediately after each filter match. Otherwise, if either parameter is set to a non-zero value, the filter emits metrics at the specified interval. Longer intervals help lower resource consumption in high-load situations. Default value: `0`. | |
37-
| `Flush_Interval_Nsec` | The interval for metrics emission, in nanoseconds. This parameter works in conjunction with `Flush_Interval_Sec`. Default value: `0`. | |
23+
| Key | Description | Default | Format |
24+
|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
25+
| `add_label` | Adds a custom label `NAME` and set the value to the value of `KEY`. | _none_ | `NAME KEY` |
26+
| `alias` | Sets an alias, useful for multiple instances of the same filter plugin. | _none_ | |
27+
| `bucket` | Optional for `metric_mode` `histogram`. If not set, default Prometheus-style buckets are used. | _none_ | For example, `0.75`. |
28+
| `discard_logs` | Flag that defines if logs should be discarded after processing. This applies for all logs, whether they have emitted metrics or not. | `false` | |
29+
| `emitter_name` | Name of the emitter (advanced users). | _none_ | |
30+
| `emitter_mem_buf_limit` | Set a buffer limit to restrict memory usage of metrics emitter. | `10M` | |
31+
| `exclude` | Excludes records in which the content of `KEY` matches the regular expression `REGEX`. | _none_ | `KEY REGEX` |
32+
| `flush_interval_sec` | The interval for metrics emission, in seconds. If `flush_interval_sec` and `flush_interval_nsec` are either both unset or both set to `0`, the filter emits metrics immediately after each filter match. Otherwise, if either parameter is set to a non-zero value, the filter emits metrics at the specified interval. Longer intervals help lower resource consumption in high-load situations. | `0` | |
33+
| `flush_interval_nsec` | The interval for metrics emission, in nanoseconds. This parameter works in conjunction with `flush_interval_sec`. | `0` | |
34+
| `kubernetes_mode` | If enabled, adds `pod_id`, `pod_name`, `namespace_name`, `docker_id` and `container_name` to the metric as labels. This option is intended to be used in combination with the [Kubernetes](./kubernetes.md) filter plugin, which fills those fields. | `false` | |
35+
| `label_field` | Includes a record field as label dimension in the metric. | _none_ | Name of record key. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields. |
36+
| `log_level` | Specifies the log level for filter plugin. If not set here, plugin uses global log level in `service` section. | `info` | |
37+
| `log_supress_interval` | Suppresses log messages from filter plugin that appear similar within a specified time interval. Setting to `0` indicates no suppression. | `0` | |
38+
| `match` | Set a tag pattern to match records that filter should process. Exact matches or wildcards. | _none_ | |
39+
| `match_regex` | Set a regular expression to match tags for filter routing. This allows more flexible matching compared to wildcards. | _none_ | |
40+
| `metric_description` | Sets a description for the metric. | _none_ | |
41+
| `metric_mode` | Defines the mode for the metric. Valid values are `counter`, `gauge` or `histogram`. | `counter` | |
42+
| `metric_name` | Sets the name of the metric. | `a` | |
43+
| `metric_namespace` | Sets the namespace of the metric. | `log_metric` | |
44+
| `metric_subsystem` | Subsystem of the metric. | _none_ | |
45+
| `regex` | Includes records in which the content of `KEY` matches the regular expression `REGEX`. | _none_ | `KEY REGEX` |
46+
| `tag` | Defines the tag for the generated metrics record. | _none_ | |
47+
| `value_field` | Required for modes `gauge` and `histogram`. Specifies the record field that holds a numerical value. | _none_ | Name of record key. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields. |
48+
3849

3950
## Examples
4051

@@ -532,4 +543,4 @@ The `+Inf` bucket will always be included regardless of the buckets you specify.
532543

533544
{% endhint %}
534545

535-
This filter also attaches Kubernetes labels to each metric, identical to the behavior of `label_field`. This results in two sets for the histogram.
546+
This filter also attaches Kubernetes labels to each metric, identical to the behavior of `label_field`. This results in two sets for the histogram.

0 commit comments

Comments
 (0)