Skip to content

Commit cf78c1b

Browse files
DmitryRomanovDmitryRomanov
committed
cardinality: more examples
1 parent 23767ba commit cf78c1b

File tree

5 files changed

+108
-44
lines changed

5 files changed

+108
-44
lines changed

‎plugin/README.md‎

Lines changed: 1 addition & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -189,23 +189,7 @@ It adds field containing hostname to an event.
189189

190190
[More details...](plugin/action/add_host/README.md)
191191
## cardinality
192-
Limits the cardinality of fields on events or drops events.
193-
194-
**An example for discarding events with high cardinality field:**
195-
```yaml
196-
pipelines:
197-
example_pipeline:
198-
...
199-
- type: cardinality
200-
limit: 10
201-
action: discard
202-
metric_prefix: service_zone
203-
key:
204-
- service
205-
fields:
206-
- zone
207-
...
208-
```
192+
Limits the cardinality of fields on events, drops events or just do nothing.
209193

210194
[More details...](plugin/action/cardinality/README.md)
211195
## convert_date

‎plugin/action/README.md‎

Lines changed: 1 addition & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -10,23 +10,7 @@ It adds field containing hostname to an event.
1010

1111
[More details...](plugin/action/add_host/README.md)
1212
## cardinality
13-
Limits the cardinality of fields on events or drops events.
14-
15-
**An example for discarding events with high cardinality field:**
16-
```yaml
17-
pipelines:
18-
example_pipeline:
19-
...
20-
- type: cardinality
21-
limit: 10
22-
action: discard
23-
metric_prefix: service_zone
24-
key:
25-
- service
26-
fields:
27-
- zone
28-
...
29-
```
13+
Limits the cardinality of fields on events, drops events or just do nothing.
3014

3115
[More details...](plugin/action/cardinality/README.md)
3216
## convert_date

‎plugin/action/cardinality/README.idoc.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Cardinality limit plugin
22
@introduction
33

4+
## Examples
5+
@examples
6+
47
## Config params
58
@config-params|description

‎plugin/action/cardinality/README.md‎

Lines changed: 51 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,67 @@
11
# Cardinality limit plugin
2-
Limits the cardinality of fields on events or drops events.
2+
Limits the cardinality of fields on events, drops events or just do nothing.
33

4-
**An example for discarding events with high cardinality field:**
4+
## Examples
5+
Discarding events with high cardinality field:
56
```yaml
67
pipelines:
78
example_pipeline:
89
...
910
- type: cardinality
10-
limit: 10
11+
limit: 2
1112
action: discard
12-
metric_prefix: service_zone
13+
ttl: 1m
14+
metric_prefix: service_client
1315
key:
1416
- service
1517
fields:
16-
- zone
18+
- client_id
1719
...
1820
```
21+
The original event:
22+
```jsonl
23+
{"service": "registration", "client_id": "1"}
24+
{"service": "registration", "client_id": "1"}
25+
{"service": "registration", "client_id": "2"}
26+
{"service": "registration", "client_id": "3"} // will be discarded
27+
```
28+
The resulting event:
29+
```json
30+
{"service": "registration", "client_id": "1"}
31+
{"service": "registration", "client_id": "1"}
32+
{"service": "registration", "client_id": "2"}
33+
```
34+
---
35+
36+
Discarding events with high cardinality field:
37+
```yaml
38+
pipelines:
39+
example_pipeline:
40+
...
41+
- type: cardinality
42+
limit: 2
43+
action: remove_fields
44+
ttl: 1m
45+
metric_prefix: service_client
46+
key:
47+
- service
48+
fields:
49+
- client_id
50+
...
51+
```
52+
The original event:
53+
```jsonl
54+
{"service": "registration", "client_id": "1"}
55+
{"service": "registration", "client_id": "2"}
56+
{"service": "registration", "client_id": "3"}
57+
```
58+
The resulting event:
59+
```json
60+
{"service": "registration", "client_id": "1"}
61+
{"service": "registration", "client_id": "2"}
62+
{"service": "registration"}
63+
```
64+
---
1965

2066
## Config params
2167
**`key`** *`[]cfg.FieldSelector`* *`required`*

‎plugin/action/cardinality/cardinality.go‎

Lines changed: 52 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,23 +17,70 @@ import (
1717
)
1818

1919
/*{ introduction
20-
Limits the cardinality of fields on events or drops events.
20+
Limits the cardinality of fields on events, drops events or just do nothing.
21+
}*/
2122

22-
**An example for discarding events with high cardinality field:**
23+
/*{ examples
24+
Discarding events with high cardinality field:
2325
```yaml
2426
pipelines:
2527
example_pipeline:
2628
...
2729
- type: cardinality
28-
limit: 10
30+
limit: 2
2931
action: discard
30-
metric_prefix: service_zone
32+
ttl: 1m
33+
metric_prefix: service_client
34+
key:
35+
- service
36+
fields:
37+
- client_id
38+
...
39+
```
40+
The original event:
41+
```jsonl
42+
{"service": "registration", "client_id": "1"}
43+
{"service": "registration", "client_id": "1"}
44+
{"service": "registration", "client_id": "2"}
45+
{"service": "registration", "client_id": "3"} // will be discarded
46+
```
47+
The resulting event:
48+
```json
49+
{"service": "registration", "client_id": "1"}
50+
{"service": "registration", "client_id": "1"}
51+
{"service": "registration", "client_id": "2"}
52+
```
53+
---
54+
55+
Discarding events with high cardinality field:
56+
```yaml
57+
pipelines:
58+
example_pipeline:
59+
...
60+
- type: cardinality
61+
limit: 2
62+
action: remove_fields
63+
ttl: 1m
64+
metric_prefix: service_client
3165
key:
3266
- service
3367
fields:
34-
- zone
68+
- client_id
3569
...
3670
```
71+
The original event:
72+
```jsonl
73+
{"service": "registration", "client_id": "1"}
74+
{"service": "registration", "client_id": "2"}
75+
{"service": "registration", "client_id": "3"}
76+
```
77+
The resulting event:
78+
```json
79+
{"service": "registration", "client_id": "1"}
80+
{"service": "registration", "client_id": "2"}
81+
{"service": "registration"}
82+
```
83+
---
3784
}*/
3885

3986
type Plugin struct {

0 commit comments

Comments
 (0)