Skip to content

Commit 345fa4d

Browse files
cosmo0920cosmo0920
authored
in_udp: Add documentation (#1165)
Also, added a description for Source_Address_Key parameter. Signed-off-by: Hiroshi Hatake <[email protected]>
1 parent 13e6460 commit 345fa4d

File tree

4 files changed

+92
-0
lines changed

4 files changed

+92
-0
lines changed

‎.gitbook.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ redirects:
3030
input/tail: ./pipeline/inputs/tail.md
3131
input/tcp: ./pipeline/inputs/tcp.md
3232
input/thermal: ./pipeline/inputs/thermal.md
33+
input/udp: ./pipeline/inputs/udp.md
3334
input/winlog: ./pipeline/inputs/windows-event-log.md
3435
input/winevtlog: ./pipeline/inputs/windows-event-log-winevtlog.md
3536

‎SUMMARY.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@
114114
* [Tail](pipeline/inputs/tail.md)
115115
* [TCP](pipeline/inputs/tcp.md)
116116
* [Thermal](pipeline/inputs/thermal.md)
117+
* [UDP](pipeline/inputs/udp.md)
117118
* [OpenTelemetry](pipeline/inputs/opentelemetry.md)
118119
* [Windows Event Log](pipeline/inputs/windows-event-log.md)
119120
* [Windows Event Log (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md)

‎installation/sources/build-and-install.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ The _input plugins_ provides certain features to gather information from a speci
149149
| [FLB\_IN\_TAIL](../../pipeline/inputs/tail.md) | Enable Tail \(follow files\) input plugin | On |
150150
| [FLB\_IN\_TCP](../../pipeline/inputs/tcp.md) | Enable TCP input plugin | On |
151151
| [FLB\_IN\_THERMAL](../../pipeline/inputs/thermal.md) | Enable system temperature\(s\) input plugin | On |
152+
| [FLB\_IN\_UDP](../../pipeline/inputs/udp.md) | Enable UDP input plugin | On |
152153
| [FLB\_IN\_WINLOG](../../pipeline/inputs/windows-event-log.md) | Enable Windows Event Log input plugin \(Windows Only\) | On |
153154
| [FLB\_IN\_WINEVTLOG](../../pipeline/inputs/windows-event-log-winevtlog.md) | Enable Windows Event Log input plugin using winevt.h API \(Windows Only\) | On |
154155

‎pipeline/inputs/udp.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# UDP
2+
3+
The **udp** input plugin allows to retrieve structured JSON or raw messages over a UDP network interface (UDP port).
4+
5+
## Configuration Parameters
6+
7+
The plugin supports the following configuration parameters:
8+
9+
| Key | Description | Default |
10+
| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
11+
| Listen | Listener network interface. | 0.0.0.0 |
12+
| Port | UDP port where listening for connections | 5170 |
13+
| Buffer\_Size | Specify the maximum buffer size in KB to receive a JSON message. If not set, the default size will be the value of _Chunk\_Size_. | |
14+
| Chunk\_Size | By default the buffer to store the incoming JSON messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by _Chunk\_Size_ in KB. If not set, _Chunk\_Size_ is equal to 32 (32KB). | 32 |
15+
| Format | Specify the expected payload format. It support the options _json_ and _none_. When using _json_, it expects JSON maps, when is set to _none_, it will split every record using the defined _Separator_ (option below). | json |
16+
| Separator | When the expected _Format_ is set to _none_, Fluent Bit needs a separator string to split the records. By default it uses the breakline character (LF or 0x10). | |
17+
| Source\_Address\_Key| Specify the key where the source address will be injected. | |
18+
19+
## Getting Started
20+
21+
In order to receive JSON messages over UDP, you can run the plugin from the command line or through the configuration file:
22+
23+
### Command Line
24+
25+
From the command line you can let Fluent Bit listen for _JSON_ messages with the following options:
26+
27+
```bash
28+
$ fluent-bit -i udp -o stdout
29+
```
30+
31+
By default the service will listen an all interfaces (0.0.0.0) through UDP port 5170, optionally you can change this directly, e.g:
32+
33+
```bash
34+
$ fluent-bit -i udp -pport=9090 -o stdout
35+
```
36+
37+
In the example the JSON messages will only arrive through network interface under 192.168.3.2 address and UDP Port 9090.
38+
39+
### Configuration File
40+
41+
In your main configuration file append the following _Input_ & _Output_ sections:
42+
43+
```python
44+
[INPUT]
45+
Name udp
46+
Listen 0.0.0.0
47+
Port 5170
48+
Chunk_Size 32
49+
Buffer_Size 64
50+
Format json
51+
52+
[OUTPUT]
53+
Name stdout
54+
Match *
55+
```
56+
57+
## Testing
58+
59+
Once Fluent Bit is running, you can send some messages using the _netcat_:
60+
61+
```bash
62+
$ echo '{"key 1": 123456789, "key 2": "abcdefg"}' | nc -u 127.0.0.1 5170
63+
```
64+
65+
In [Fluent Bit](http://fluentbit.io) we should see the following output:
66+
67+
```bash
68+
$ bin/fluent-bit -i udp -o stdout -f 1
69+
Fluent Bit v2.x.x
70+
* Copyright (C) 2015-2022 The Fluent Bit Authors
71+
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
72+
* https://fluentbit.io
73+
74+
[2023/07/21 13:01:03] [ info] [fluent bit] version=2.1.7, commit=2474ccc759, pid=9677
75+
[2023/07/21 13:01:03] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
76+
[2023/07/21 13:01:03] [ info] [cmetrics] version=0.6.3
77+
[2023/07/21 13:01:03] [ info] [ctraces ] version=0.3.1
78+
[2023/07/21 13:01:03] [ info] [input:udp:udp.0] initializing
79+
[2023/07/21 13:01:03] [ info] [input:udp:udp.0] storage_strategy='memory' (memory only)
80+
[2023/07/21 13:01:03] [ info] [output:stdout:stdout.0] worker #0 started
81+
[2023/07/21 13:01:03] [ info] [sp] stream processor started
82+
[0] udp.0: [[1689912069.078189000, {}], {"key 1"=>123456789, "key 2"=>"abcdefg"}]
83+
```
84+
85+
## Performance Considerations
86+
87+
When receiving payloads in JSON format, there are high performance penalties. Parsing JSON is a very expensive task so you could expect your CPU usage increase under high load environments.
88+
89+
To get faster data ingestion, consider to use the option `Format none` to avoid JSON parsing if not needed.

0 commit comments

Comments
 (0)