Skip to content

Pipeline: input: udp: style #1839

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 3, 2025
Merged

Pipeline: input: udp: style #1839

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f48d685
Pipeline: input: udp: style
esmerel Jul 2, 2025
cc104b8
Apply suggestions from code review
esmerel Jul 3, 2025
d36bb2d
Merge branch 'master' into lynettemiles/sc-136209/update-fluent-bit-d…
esmerel Jul 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 53 additions & 44 deletions pipeline/inputs/udp.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,48 +1,67 @@
# UDP

The **udp** input plugin allows to retrieve structured JSON or raw messages over a UDP network interface (UDP port).
The _UDP_ input plugin lets you retrieve structured JSON or raw messages over a UDP network interface (UDP port).

## Configuration Parameters
## Configuration parameters

The plugin supports the following configuration parameters:

| Key | Description | Default |
| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| Listen | Listener network interface. | 0.0.0.0 |
| Port | UDP port where listening for connections | 5170 |
| Buffer\_Size | Specify the maximum buffer size in KB to receive a JSON message. If not set, the default size will be the value of _Chunk\_Size_. | |
| Chunk\_Size | By default the buffer to store the incoming JSON messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by _Chunk\_Size_ in KB. If not set, _Chunk\_Size_ is equal to 32 (32KB). | 32 |
| Format | Specify the expected payload format. It support the options _json_ and _none_. When using _json_, it expects JSON maps, when is set to _none_, it will split every record using the defined _Separator_ (option below). | json |
| Separator | When the expected _Format_ is set to _none_, Fluent Bit needs a separator string to split the records. By default it uses the breakline character (LF or 0x10). | |
| Source\_Address\_Key| Specify the key where the source address will be injected. | |
| Threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
| Key | Description | Default |
| --- | ----------- | ------- |
| `Listen` | Listener network interface. | `0.0.0.0` |
| `Port` | UDP port used to listen for connections. | `5170` |
| `Buffer_Size `| Specify the maximum buffer size in KB to receive a JSON message. If not set, the default size will be the value of `Chunk_Size`. | `Chunk_Size` (value) |
| `Chunk_Size` | The default buffer to store incoming JSON messages. Doesn't allocate the maximum memory allowed; instead it allocates memory when required. The rounds of allocations are set by `Chunk_Size` in KB. | `32` |
| `Format` | Specify the expected payload format. Supported values: `json` and `none`. `json` expects JSON maps. `none` splits every record using the defined `Separator`. | `json` |
| `Separator` | When `Format` is set to `none`, Fluent Bit needs a separator string to split the records. | `LF` or `0x10` (break line) |
| `Source_Address_Key`| Specify the key where the source address will be injected. | _none_ |
| `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |

## Getting Started
## Get started

In order to receive JSON messages over UDP, you can run the plugin from the command line or through the configuration file:
To receive JSON messages over UDP, you can run the plugin from the command line or through the configuration file:

### Command Line
### Command line

From the command line you can let Fluent Bit listen for _JSON_ messages with the following options:
From the command line you can let Fluent Bit listen for JSON messages with the following options:

```bash
$ fluent-bit -i udp -o stdout
fluent-bit -i udp -o stdout
```

By default the service will listen an all interfaces (0.0.0.0) through UDP port 5170, optionally you can change this directly, e.g:
By default, the service listens on all interfaces (`0.0.0.0`) using UDP port `5170`. Optionally. you can change this directly.

In this example the JSON messages will only arrive through network interface at `192.168.3.2` address and UDP Port `9090`.

```bash
$ fluent-bit -i udp -pport=9090 -o stdout
fluent-bit -i udp -pport=9090 -o stdout
```

In the example the JSON messages will only arrive through network interface under 192.168.3.2 address and UDP Port 9090.

### Configuration File
### Configuration file

In your main configuration file append the following _Input_ & _Output_ sections:
In your main configuration file append the following sections:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
- name: udp
listen: 0.0.0.0
port: 5170
chunk_size: 32
buffer_size: 64
format: json
outputs:
- name: stdout
match: '*'
```

{% endtab %}

{% tab title="fluent-bit.conf" %}

```python
[INPUT]
Name udp
Expand All @@ -56,37 +75,27 @@ In your main configuration file append the following _Input_ & _Output_ sections
Name stdout
Match *
```
{% endtab %}

{% tab title="fluent-bit.yaml" %}
```yaml
pipeline:
inputs:
- name: udp
listen: 0.0.0.0
port: 5170
chunk_size: 32
buffer_size: 64
format: json
outputs:
- name: stdout
match: '*'
```
{% endtab %}
{% endtabs %}

## Testing

Once Fluent Bit is running, you can send some messages using the _netcat_:
When Fluent Bit is running, you can send some messages using `netcat`:

```bash
$ echo '{"key 1": 123456789, "key 2": "abcdefg"}' | nc -u 127.0.0.1 5170
echo '{"key 1": 123456789, "key 2": "abcdefg"}' | nc -u 127.0.0.1 5170
```

In [Fluent Bit](http://fluentbit.io) we should see the following output:
Run Fluent Bit:

```bash
$ bin/fluent-bit -i udp -o stdout -f 1
bin/fluent-bit -i udp -o stdout -f 1
```

You should see the following output:

```text
Fluent Bit v2.x.x
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
Expand All @@ -103,8 +112,8 @@ Fluent Bit v2.x.x
[0] udp.0: [[1689912069.078189000, {}], {"key 1"=>123456789, "key 2"=>"abcdefg"}]
```

## Performance Considerations
## Performance considerations

When receiving payloads in JSON format, there are high performance penalties. Parsing JSON is a very expensive task so you could expect your CPU usage increase under high load environments.

To get faster data ingestion, consider to use the option `Format none` to avoid JSON parsing if not needed.
To get faster data ingestion, consider using the option `Format none` to avoid JSON parsing if not needed.