Skip to content

Commit 2257af2

Browse files
committed
docs: update docs for loki.process with regex fields
1 parent 2e77c56 commit 2257af2

File tree

2 files changed

+17
-12
lines changed

2 files changed

+17
-12
lines changed

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ Main (unreleased)
2222

2323
- Add htpasswd file based authentication for `otelcol.auth.basic` (@pkarakal)
2424

25+
- Add a `regex` argument to the `logfmt` and `json` stages in `loki.process` to extract log fields matching a regular expression. (@timonegk)
26+
2527
### Enhancements
2628

2729
- update promtail converter to use `file_match` block for `loki.source.file` instead of going through `local.file_match`. (@kalleep)

docs/sources/reference/components/loki/loki.process.md

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -496,12 +496,15 @@ The following arguments are supported:
496496

497497
| Name | Type | Description | Default | Required |
498498
| ---------------- | ------------- | ----------------------------------------------------- | ------- | -------- |
499-
| `expressions` | `map(string)` | Key-value pairs of JMESPath expressions. | | yes |
499+
| `expressions` | `map(string)` | Key-value pairs of JMESPath expressions. | | no |
500+
| `regex` | `string` | Regular expression matched against JSON keys. | | no |
500501
| `drop_malformed` | `bool` | Drop lines whose input can't be parsed as valid JSON. | `false` | no |
501502
| `source` | `string` | Source of the data to parse as JSON. | `""` | no |
502503

503504
The `expressions` field is the set of key-value pairs of JMESPath expressions to run.
504-
The map key defines the name with which the data is extracted, while the map value is the expression used to populate the value.
505+
The map key defines the name used to extract the data, while the map value is the expression used to populate the value.
506+
507+
The `regex` field is a regular expression. All keys in the JSON source matching the regular expression are extracted.
505508

506509
When configuring a JSON stage, the `source` field defines the source of data to parse as JSON.
507510
By default, this is the log line itself, but it can also be a previously extracted value.
@@ -653,19 +656,19 @@ The `stage.logfmt` inner block configures a processing stage that reads incoming
653656

654657
The following arguments are supported:
655658

656-
| Name | Type | Description | Default | Required |
657-
| --------- | ------------- | ---------------------------------------------- | ------- | -------- |
658-
| `mapping` | `map(string)` | Key-value pairs of `logmft` fields to extract. | | yes |
659-
| `source` | `string` | Source of the data to parse as `logfmt`. | `""` | no |
659+
| Name | Type | Description | Default | Required |
660+
| --------- | ------------- | ----------------------------------------------- | ------- | -------- |
661+
| `mapping` | `map(string)` | Key-value pairs of `logmft` fields to extract. | | no |
662+
| `regex` | `string` | Regular expression matched against logfmt keys. | | no |
663+
| `source` | `string` | Source of the data to parse as `logfmt`. | `""` | no |
660664

661-
The `source` field defines the source of data to parse as `logfmt`.
662-
When `source` is missing or empty, the stage parses the log line itself, but it can also be used to parse a previously extracted value.
665+
The `mapping` field is the set of key-value pairs.
666+
The map key defines the name used to extract the data, while the map value is the logfmt field used to populate the value.
663667

664-
This stage uses the [go-logfmt][] unmarshaler, so that numeric or boolean types are unmarshalled into their correct form.
665-
The stage doesn't perform any other type conversions.
666-
If the extracted value is a complex type, it's treated as a string.
668+
The `regex` field is a regular expression. All logfmt fields matching the regular expression are extracted.
667669

668-
[go-logfmt]: https://github.com/go-logfmt/logfmt
670+
The `source` field defines the source of data to parse as `logfmt`.
671+
When `source` is missing or empty, the stage parses the log line itself, but it can also be used to parse a previously extracted value.
669672

670673
The following log line and stages demonstrates how this works.
671674

0 commit comments

Comments
 (0)