Skip to content

Commit 4a23d13

Browse files
esmerelesmerel
authored
Merge pull request #1826 from fluent/lynettemiles/sc-136204/update-fluent-bit-docs-pipeline-inputs-syslog
2 parents d6506c4 + 277df6a commit 4a23d13

File tree

1 file changed

+111
-95
lines changed

1 file changed

+111
-95
lines changed

‎pipeline/inputs/syslog.md

Lines changed: 111 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,70 @@
11
# Syslog
22

3-
_Syslog_ input plugins allows to collect Syslog messages through a Unix socket server \(UDP or TCP\) or over the network using TCP or UDP.
3+
The _Syslog_ input plugin lets you collect `syslog` messages through a Unix socket server (UDP or TCP) or over the network using TCP or UDP.
44

5-
## Configuration Parameters
5+
## Configuration parameters
66

77
The plugin supports the following configuration parameters:
88

99
| Key | Description | Default |
1010
| :--- | :--- | :--- |
11-
| Mode | Defines transport protocol mode: unix\_udp \(UDP over Unix socket\), unix\_tcp \(TCP over Unix socket\), tcp or udp | unix\_udp |
12-
| Listen | If _Mode_ is set to _tcp_ or _udp_, specify the network interface to bind. | 0.0.0.0 |
13-
| Port | If _Mode_ is set to _tcp_ or _udp_, specify the TCP port to listen for incoming connections. | 5140 |
14-
| Path | If _Mode_ is set to _unix\_tcp_ or _unix\_udp_, set the absolute path to the Unix socket file. | |
15-
| Unix\_Perm | If _Mode_ is set to _unix\_tcp_ or _unix\_udp_, set the permission of the Unix socket file. | 0644 |
16-
| Parser | Specify an alternative parser for the message. If _Mode_ is set to _tcp_ or _udp_ then the default parser is _syslog-rfc5424_ otherwise _syslog-rfc3164-local_ is used. If your syslog messages have fractional seconds set this Parser value to _syslog-rfc5424_ instead. | |
17-
| Buffer\_Chunk\_Size | By default the buffer to store the incoming Syslog messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by _Buffer\_Chunk\_Size_. If not set, _Buffer\_Chunk\_Size_ is equal to 32000 bytes \(32KB\). Read considerations below when using _udp_ or _unix\_udp_ mode. | |
18-
| Buffer\_Max\_Size | Specify the maximum buffer size to receive a Syslog message. If not set, the default size will be the value of _Buffer\_Chunk\_Size_. | |
19-
| Receive\_Buffer\_Size | Specify the maximum socket receive buffer size. If not set, the default value is OS-dependant, but generally too low to accept thousands of syslog messages per second without loss on _udp_ or _unix\_udp_ sockets. Note that on Linux the value is capped by `sysctl net.core.rmem_max`.| |
20-
| Source\_Address\_Key| Specify the key where the source address will be injected. | |
21-
| Threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
11+
| `Mode` | Defines transport protocol mode: UDP over Unix socket (`unix_udp`), TCP over Unix socket (`unix_tcp`), `tcp`, or `udp` | `unix_udp` |
12+
| `Listen` | If `Mode` is set to `tcp` or `udp`, specify the network interface to bind. | `0.0.0.0` |
13+
| `Port` | If `Mode` is set to `tcp` or `udp`, specify the TCP port to listen for incoming connections. | `5140` |
14+
| `Path` | If `Mode` is set to `unix_tcp` or `unix_udp`, set the absolute path to the Unix socket file. | _none_ |
15+
| `Unix_Perm` | If `Mode` is set to `unix_tcp` or `unix_udp`, set the permission of the Unix socket file. | `0644` |
16+
| `Parser` | Specify an alternative parser for the message. If `Mode` is set to `tcp` or `udp` then the default parser is `syslog-rfc5424`. Otherwise, `syslog-rfc3164-local` is used. If your syslog` messages have fractional seconds set this parser value to `syslog-rfc5424` instead. | _none_ |
17+
| `Buffer_Chunk_Size` | By default, the buffer to store the incoming `syslog` messages. Doesn't allocate the maximum memory allowed, instead it allocates memory when required. The rounds of allocations are set by `Buffer_Chunk_Size`. There are considerations when using `udp` or `unix_udp` mode. | `32KB` (set in code) |
18+
| `Buffer_Max_Size` | Specify the maximum buffer size to receive a `syslog` message. If not set, the default size is the value of `Buffer_Chunk_Size`. | _none_ |
19+
| `Receive_Buffer_Size` | Specify the maximum socket receive buffer size. If not set, the default value is OS-dependant, but generally too low to accept thousands of syslog messages per second without loss on `udp` or `unix_udp` sockets. For Linux, the value is capped by `sysctl net.core.rmem_max`. | _none_ |
20+
| `Source_Address_Key` | Specify the key where the source address will be injected. | _none_ |
21+
| `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
2222

2323
### Considerations
2424

25-
* When using Syslog input plugin, Fluent Bit requires access to the _parsers.conf_ file, the path to this file can be specified with the option _-R_ or through the _Parsers\_File_ key on the \[SERVICE\] section \(more details below\).
26-
* When _udp_ or _unix\_udp_ is used, the buffer size to receive messages is configurable **only** through the _Buffer\_Chunk\_Size_ option which defaults to 32kb.
25+
- When using the Syslog input plugin, Fluent Bit requires access to the `parsers.conf` file. The path to this file can be specified with the option `-R` or through the `Parsers_File` key in the `[SERVICE]` section.
26+
- When using `udp` or `unix_udp`, the buffer size to receive messages is configurable only through the `Buffer_Chunk_Size` option, which defaults to 32kb.
2727

28-
## Getting Started
28+
## Get started
2929

30-
In order to receive Syslog messages, you can run the plugin from the command line or through the configuration file:
30+
To receive `syslog` messages, you can run the plugin from the command line or through the configuration file:
3131

32-
### Command Line
32+
### Command line
3333

34-
From the command line you can let Fluent Bit listen for _Forward_ messages with the following options:
34+
From the command line you can let Fluent Bit listen for `Forward` messages with the following options:
3535

3636
```bash
37-
$ fluent-bit -R /path/to/parsers.conf -i syslog -p path=/tmp/in_syslog -o stdout
37+
fluent-bit -R /path/to/parsers.conf -i syslog -p path=/tmp/in_syslog -o stdout
3838
```
3939

40-
By default the service will create and listen for Syslog messages on the unix socket _/tmp/in\_syslog_
40+
By default the service will create and listen for Syslog messages on the Unix socket `/tmp/in_syslog`.
4141

42-
### Configuration File
42+
### Configuration file
4343

44-
In your main configuration file append the following _Input_ & _Output_ sections:
44+
In your main configuration file append the following sections:
4545

4646
{% tabs %}
47+
48+
{% tab title="fluent-bit.yaml" %}
49+
50+
```yaml
51+
service:
52+
flush: 1
53+
log_level: info
54+
parsers_file: parsers.conf
55+
pipeline:
56+
inputs:
57+
- name: syslog
58+
path: /tmp/in_syslog
59+
buffer_chunk_size: 32000
60+
buffer_max_size: 64000
61+
receive_buffer_size: 512000
62+
outputs:
63+
- name: stdout
64+
match: '*'
65+
```
66+
67+
{% endtab %}
4768
{% tab title="fluent-bit.conf" %}
4869
```python
4970
[SERVICE]
@@ -62,40 +83,28 @@ In your main configuration file append the following _Input_ & _Output_ sections
6283
Name stdout
6384
Match *
6485
```
65-
{% endtab %}
6686

67-
{% tab title="fluent-bit.yaml" %}
68-
```yaml
69-
service:
70-
flush: 1
71-
log_level: info
72-
parsers_file: parsers.conf
73-
pipeline:
74-
inputs:
75-
- name: syslog
76-
path: /tmp/in_syslog
77-
buffer_chunk_size: 32000
78-
buffer_max_size: 64000
79-
receive_buffer_size: 512000
80-
outputs:
81-
- name: stdout
82-
match: '*'
83-
```
8487
{% endtab %}
88+
8589
{% endtabs %}
8690

8791
### Testing
8892

89-
Once Fluent Bit is running, you can send some messages using the _logger_ tool:
93+
When Fluent Bit is running, you can send some messages using the logger tool:
9094

9195
```bash
92-
$ logger -u /tmp/in_syslog my_ident my_message
96+
logger -u /tmp/in_syslog my_ident my_message
9397
```
9498

95-
In [Fluent Bit](http://fluentbit.io) we should see the following output:
99+
Then run Fluent bit using the following command:
96100

97101
```bash
98-
$ bin/fluent-bit -R ../conf/parsers.conf -i syslog -p path=/tmp/in_syslog -o stdout
102+
bin/fluent-bit -R ../conf/parsers.conf -i syslog -p path=/tmp/in_syslog -o stdout
103+
```
104+
105+
You should see the following output:
106+
107+
```text
99108
Fluent Bit v1.x.x
100109
* Copyright (C) 2019-2020 The Fluent Bit Authors
101110
* Copyright (C) 2015-2018 Treasure Data
@@ -106,18 +115,39 @@ Fluent Bit v1.x.x
106115
[0] syslog.0: [1489047822, {"pri"=>"13", "host"=>"edsiper:", "ident"=>"my_ident", "pid"=>"", "message"=>"my_message"}]
107116
```
108117

109-
## Recipes
118+
## Examples
110119

111-
The following content aims to provide configuration examples for different use cases to integrate Fluent Bit and make it listen for Syslog messages from your systems.
120+
The following configuration examples cover different use cases to integrate Fluent Bit and make it listen for Syslog messages from your systems.
112121

113-
### Rsyslog to Fluent Bit: Network mode over TCP <a id="rsyslog_to_fluentbit_network"></a>
122+
### `rsyslog` to Fluent Bit: Network mode over TCP
114123

115-
#### Fluent Bit Configuration
124+
#### Fluent Bit configuration
116125

117126
Put the following content in your configuration file:
118127

119128
{% tabs %}
129+
{% tab title="fluent-bit.yaml" %}
130+
131+
```yaml
132+
service:
133+
flush: 1
134+
parsers_file: parsers.conf
135+
pipeline:
136+
inputs:
137+
- name: syslog
138+
parser: syslog-rfc3164
139+
listen: 0.0.0.0
140+
port: 5140
141+
mode: tcp
142+
outputs:
143+
- name: stdout
144+
match: '*'
145+
```
146+
147+
{% endtab %}
148+
120149
{% tab title="fluent-bit.conf" %}
150+
121151
```text
122152
[SERVICE]
123153
Flush 1
@@ -134,51 +164,54 @@ Put the following content in your configuration file:
134164
Name stdout
135165
Match *
136166
```
137-
{% endtab %}
138167

139-
{% tab title="fluent-bit.yaml" %}
140-
```yaml
141-
service:
142-
flush: 1
143-
parsers_file: parsers.conf
144-
pipeline:
145-
inputs:
146-
- name: syslog
147-
parser: syslog-rfc3164
148-
listen: 0.0.0.0
149-
port: 5140
150-
mode: tcp
151-
outputs:
152-
- name: stdout
153-
match: '*'
154-
```
155168
{% endtab %}
156169
{% endtabs %}
157170

158-
then start Fluent Bit.
171+
Then, start Fluent Bit.
159172

160-
#### RSyslog Configuration
173+
#### `rsyslog` configuration
161174

162-
Add a new file to your rsyslog config rules called _60-fluent-bit.conf_ inside the directory _/etc/rsyslog.d/_ and add the following content:
175+
Add a new file to your `rsyslog` configuration rules called `60-fluent-bit.conf` inside the directory `/etc/rsyslog.d/` and add the following content:
163176

164177
```text
165178
action(type="omfwd" Target="127.0.0.1" Port="5140" Protocol="tcp")
166179
```
167180

168-
then make sure to restart your rsyslog daemon:
181+
Then, restart your `rsyslog` daemon:
169182

170183
```bash
171-
$ sudo service rsyslog restart
184+
sudo service rsyslog restart
172185
```
173186

174-
### Rsyslog to Fluent Bit: Unix socket mode over UDP
187+
### `rsyslog` to Fluent Bit: Unix socket mode over UDP
175188

176-
#### Fluent Bit Configuration
189+
#### Fluent Bit configuration
177190

178-
Put the following content in your fluent-bit.conf file:
191+
Put the following content in your `fluent-bit.conf` file:
179192

180193
{% tabs %}
194+
{% tab title="fluent-bit.yaml" %}
195+
196+
```yaml
197+
service:
198+
flush: 1
199+
parsers_file: parsers.conf
200+
pipeline:
201+
inputs:
202+
- name: syslog
203+
parser: syslog-rfc3164
204+
path: /tmp/fluent-bit.sock
205+
mode: unix_udp
206+
unix_perm: 0644
207+
outputs:
208+
- name: stdout
209+
match: '*'
210+
```
211+
212+
{% endtab %}
181213
{% tab title="fluent-bit.conf" %}
214+
182215
```text
183216
[SERVICE]
184217
Flush 1
@@ -195,37 +228,20 @@ Put the following content in your fluent-bit.conf file:
195228
Name stdout
196229
Match *
197230
```
198-
{% endtab %}
199231

200-
{% tab title="fluent-bit.yaml" %}
201-
```yaml
202-
service:
203-
flush: 1
204-
parsers_file: parsers.conf
205-
pipeline:
206-
inputs:
207-
- name: syslog
208-
parser: syslog-rfc3164
209-
path: /tmp/fluent-bit.sock
210-
mode: unix_udp
211-
unix_perm: 0644
212-
outputs:
213-
- name: stdout
214-
match: '*'
215-
```
216232
{% endtab %}
217233
{% endtabs %}
218234

219-
then start Fluent Bit.
235+
Then, start Fluent Bit.
220236

221-
#### RSyslog Configuration
237+
#### `rsyslog` configuration
222238

223-
Add a new file to your rsyslog config rules called _60-fluent-bit.conf_ inside the directory _/etc/rsyslog.d/_ and place the following content:
239+
Add a new file to your `rsyslog` configuration rules called `60-fluent-bit.conf` inside the directory `/etc/rsyslog.d/` containing the following content:
224240

225241
```text
226242
$ModLoad omuxsock
227243
$OMUxSockSocket /tmp/fluent-bit.sock
228244
*.* :omuxsock:
229245
```
230246

231-
Make sure that the socket file is readable by rsyslog \(tweak the `Unix_Perm` option shown above\).
247+
Make sure that the socket file is readable by `rsyslog` by modifying `Unix_Perm` key.

0 commit comments

Comments
 (0)