Skip to content

Commit 608c388

Browse files
Joel GerberJoel Gerber
authored andcommitted
Create Filtering-Details.md documentation page
This page takes examples given via issue #56 and puts them into more elaborate prose that hopefully gives a complete view into how one can use the filters statement to perform log filtering.
1 parent 70cec7e commit 608c388

File tree

1 file changed

+142
-0
lines changed

1 file changed

+142
-0
lines changed

docs/Filtering-Details.md

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Filtering Details
2+
3+
## Overview
4+
5+
This application takes an array of hashes passed to the `filters` parameter
6+
within a `systemd` typed source definition in your `fluent.conf` configuration
7+
file and then parses them into a format understood by `systemd`'s `journald`
8+
API. The basis behind what `journald`'s API expects can be found documented in
9+
the `journalctl` [man
10+
page](https://www.freedesktop.org/software/systemd/man/journalctl.html).
11+
12+
## Usage Information
13+
14+
In order to utilize this plugin's filtering capabilities, you will need to
15+
understand how this plugin transforms the passed array of hashes into a format
16+
that is understood by `journald`.
17+
18+
The best way to describe this process is probably by example. The following
19+
sub-sections lists out various scenarios that you might wish to perform with
20+
this plugin's filtering mechanism and describes both how to configure them,
21+
while also mapping them to examples from the `journalctl` [man
22+
page](https://www.freedesktop.org/software/systemd/man/journalctl.html).
23+
24+
### No Filters
25+
26+
You can leave the `filters` property out altogether, or include a `filters`
27+
property with an empty array (as shown below) to specify that no filtering
28+
should occur.
29+
30+
filters []
31+
32+
Which matches this part of the `journalctl` man page:
33+
34+
> Without arguments, all collected logs are shown unfiltered:
35+
>
36+
> `journalctl`
37+
38+
### Single Filter
39+
40+
You can pass a single hash map to the `filters` array with a single key/value
41+
pair specified to filter out all log entries that do not match the given
42+
field/value combination.
43+
44+
For example:
45+
46+
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service"}]
47+
48+
Which coincides with this part of the the `journalctl` man page:
49+
50+
> With one match specified, all entries with a field matching the expression are
51+
> shown:
52+
>
53+
> `journalctl _SYSTEMD_UNIT=avahi-daemon.service`
54+
55+
### Multi-Field Filters
56+
57+
You can pass a single hash map to the `filters` array with multiple key/value
58+
pairs to filter out all log entries that do not match the combination of all of
59+
the specified key/value combinations.
60+
61+
The passed key/value pairs are treated as a logical `AND`, such that all of the
62+
pairs must be true in order to allow further processing of the current log
63+
entry.
64+
65+
For Example:
66+
67+
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service", "_PID": 28097}]
68+
69+
Which coincides with this part of the the `journalctl` man page:
70+
71+
> If two different fields are matched, only entries matching both expressions at
72+
> the same time are shown:
73+
>
74+
> `journalctl _SYSTEMD_UNIT=avahi-daemon.service _PID=28097`
75+
76+
You can also perform a logical `OR` by splitting key/value pairs across multiple
77+
hashes passed to the `filters` array like so:
78+
79+
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service"}, {"_PID": 28097}]
80+
81+
You can combine both `AND` and `OR` combinations together; using a single hash
82+
map to define conditions that `AND` together and using multiple hash maps to
83+
define conditions that `OR` together like so:
84+
85+
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service", "_PID": 28097}, {"_SYSTEMD_UNIT": "dbus.service"}]
86+
87+
This can be expressed in psuedo-code like so:
88+
89+
IF (_SYSTEMD_UNIT=avahi-daemon.service AND _PID=28097) OR _SYSTEMD_UNIT=dbus.service
90+
THEN PASS
91+
ELSE DENY
92+
93+
Which coincides with this part of the `journalctl` man page:
94+
95+
> If the separator "+" is used, two expressions may be combined in a logical OR.
96+
> The following will show all messages from the Avahi service process with the
97+
> PID 28097 plus all messages from the D-Bus service (from any of its
98+
> processes):
99+
>
100+
> `journalctl _SYSTEMD_UNIT=avahi-daemon.service _PID=28097 + _SYSTEMD_UNIT=dbus.service`
101+
102+
### Multi-Value Filters
103+
104+
Fields with arrays as values are treated as a logical `OR` statement.
105+
106+
For example:
107+
108+
filters [{"_SYSTEMD_UNIT": ["avahi-daemon.service", "dbus.service"]}]
109+
110+
Which coincides with this part of the `journalctl` man page:
111+
112+
> If two matches refer to the same field, all entries matching either expression
113+
> are shown:
114+
>
115+
> `journalctl _SYSTEMD_UNIT=avahi-daemon.service _SYSTEMD_UNIT=dbus.service`
116+
117+
The above example can be equivalently broken into 2 separate hashes. This is
118+
particularly helpful when you want to create aggregate logic
119+
120+
For example:
121+
122+
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service", "_PID": 28097}, {"_SYSTEMD_UNIT": "dbus.service"}]
123+
124+
This can be expressed in psuedo-code like so:
125+
126+
IF (_SYSTEMD_UNIT=avahi-daemon.service AND _PID=28097) OR _SYSTEMD_UNIT=dbus.service
127+
THEN PASS
128+
ELSE DENY
129+
130+
Which coincides with this part of the `journalctl` man page:
131+
132+
> If the separator "+" is used, two expressions may be combined in a logical OR.
133+
> The following will show all messages from the Avahi service process with the
134+
> PID 28097 plus all messages from the D-Bus service (from any of its
135+
> processes):
136+
>
137+
> `journalctl _SYSTEMD_UNIT=avahi-daemon.service _PID=28097 + _SYSTEMD_UNIT=dbus.service`
138+
139+
### Wildcard Filters
140+
141+
`systemd`/`journald` does not presently support wild-card filtering, so neither
142+
can this plugin.

0 commit comments

Comments
 (0)