Skip to content

Commit 17de6e5

Browse files
authored
Merge pull request #1722 from fluent/lynettemiles/sc-136176/update-fluent-bit-docs-pipeline-inputs-exec
2 parents 527823d + ac06cb3 commit 17de6e5

File tree

1 file changed

+53
-44
lines changed

1 file changed

+53
-44
lines changed

pipeline/inputs/exec.md

Lines changed: 53 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,42 +1,50 @@
11
# Exec
22

3-
The **exec** input plugin, allows to execute external program and collects event logs.
3+
The _Exec_ input plugin lets you execute external programs and collects event logs.
44

5-
**WARNING**: Because this plugin invokes commands via a shell, its inputs are
6-
subject to shell metacharacter substitution. Careless use of untrusted input in
7-
command arguments could lead to malicious command execution.
5+
{% hint style="warning" %}
6+
7+
This plugin invokes commands using a shell. Its inputs are subject to shell metacharacter substitution. Careless use of untrusted input in command arguments could lead to malicious command execution.
8+
9+
{% end hint %}
810

911
## Container support
1012

11-
This plugin will not function in all the distroless production images as it needs a functional `/bin/sh` which is not present.
12-
The debug images use the same binaries so even though they have a shell, there is no support for this plugin as it is compiled out.
13+
This plugin needs a functional `/bin/sh` and won't function in all the distroless production images.
14+
15+
The debug images use the same binaries so even though they have a shell, there is no support for this plugin as it's compiled out.
1316

14-
## Configuration Parameters
17+
## Configuration parameters
1518

1619
The plugin supports the following configuration parameters:
1720

1821
| Key | Description |
1922
| :--- | :--- |
20-
| Command | The command to execute, passed to [popen(...)](https://man7.org/linux/man-pages/man3/popen.3.html) without any additional escaping or processing. May include pipelines, redirection, command-substitution, etc. |
21-
| Parser | Specify the name of a parser to interpret the entry as a structured message. |
22-
| Interval\_Sec | Polling interval \(seconds\). |
23-
| Interval\_NSec | Polling interval \(nanosecond\). |
24-
| Buf\_Size | Size of the buffer \(check [unit sizes](../../administration/configuring-fluent-bit/unit-sizes.md) for allowed values\) |
25-
| Oneshot | Only run once at startup. This allows collection of data precedent to fluent-bit's startup (bool, default: false) |
26-
| Exit\_After\_Oneshot | Exit as soon as the one-shot command exits. This allows the exec plugin to be used as a wrapper for another command, sending the target command's output to any fluent-bit sink(s) then exiting. (bool, default: false) |
27-
| Propagate\_Exit\_Code | When exiting due to Exit\_After\_Oneshot, cause fluent-bit to exit with the exit code of the command exited by this plugin. Follows [shell conventions for exit code propagation](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html). (bool, default: false) |
28-
| Threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). Default: `false`. |
29-
30-
## Getting Started
23+
| `Command` | The command to execute, passed to [popen](https://man7.org/linux/man-pages/man3/popen.3.html) without any additional escaping or processing. Can include pipelines, redirection, command-substitution, or other information. |
24+
| `Parser` | Specify the name of a parser to interpret the entry as a structured message. |
25+
| `Interval_Sec` | Polling interval (seconds). |
26+
| `Interval_NSec` | Polling interval (nanosecond). |
27+
| `Buf_Size` | Size of the buffer. See [unit sizes](../../administration/configuring-fluent-bit/unit-sizes.md) for allowed values. |
28+
| `Oneshot` | Only run once at startup. This allows collection of data precedent to Fluent Bit startup (Boolean, default: `false`). |
29+
| `Exit_After_Oneshot` | Exit as soon as the one-shot command exits. This allows the `exec` plugin to be used as a wrapper for another command, sending the target command's output to any Fluent Bit sink, then exits. (Boolean, default: `false`). |
30+
| `Propagate_Exit_Code` | When exiting due to `Exit_After_Oneshot`, cause Fluent Bit to exit with the exit code of the command exited by this plugin. Follows [shell conventions for exit code propagation](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html). (Boolean, default: `false`). |
31+
| `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). Default: `false`. |
32+
33+
## Get started
3134

3235
You can run the plugin from the command line or through the configuration file:
3336

34-
### Command Line
37+
### Command line
3538

3639
The following example will read events from the output of _ls_.
3740

3841
```bash
39-
$ fluent-bit -i exec -p 'command=ls /var/log' -o stdout
42+
fluent-bit -i exec -p 'command=ls /var/log' -o stdout
43+
```
44+
45+
which should return something like the following:
46+
47+
```text
4048
Fluent Bit v1.x.x
4149
* Copyright (C) 2019-2020 The Fluent Bit Authors
4250
* Copyright (C) 2015-2018 Treasure Data
@@ -53,12 +61,13 @@ Fluent Bit v1.x.x
5361
[6] exec.0: [1521622010.013497225, {"exec"=>"anaconda.storage.log"}]
5462
```
5563

56-
### Configuration File
64+
### Configuration file
5765

58-
In your main configuration file append the following _Input_ & _Output_ sections:
66+
In your main configuration file append the following `Input` and `Output` sections:
5967

6068
{% tabs %}
6169
{% tab title="fluent-bit.conf" %}
70+
6271
```python
6372
[INPUT]
6473
Name exec
@@ -73,9 +82,11 @@ In your main configuration file append the following _Input_ & _Output_ sections
7382
Name stdout
7483
Match *
7584
```
85+
7686
{% endtab %}
7787

7888
{% tab title="fluent-bit.yaml" %}
89+
7990
```yaml
8091
pipeline:
8192
inputs:
@@ -91,17 +102,18 @@ pipeline:
91102
- name: stdout
92103
match: '*'
93104
```
105+
94106
{% endtab %}
95107
{% endtabs %}
96108
97109
## Use as a command wrapper
98110
99-
To use `fluent-bit` with the `exec` plugin to wrap another command, use the
100-
`Exit_After_Oneshot` and `Propagate_Exit_Code` options, e.g.:
111+
To use Fluent Bit with the `exec` plugin to wrap another command, use the `Exit_After_Oneshot` and `Propagate_Exit_Code` options:
101112

102113
{% tabs %}
103114
{% tab title="fluent-bit.conf" %}
104-
```
115+
116+
```python
105117
[INPUT]
106118
Name exec
107119
Tag exec_oneshot_demo
@@ -114,9 +126,11 @@ To use `fluent-bit` with the `exec` plugin to wrap another command, use the
114126
Name stdout
115127
Match *
116128
```
129+
117130
{% endtab %}
118131

119132
{% tab title="fluent-bit.yaml" %}
133+
120134
```yaml
121135
pipeline:
122136
inputs:
@@ -131,12 +145,13 @@ pipeline:
131145
- name: stdout
132146
match: '*'
133147
```
148+
134149
{% endtab %}
135150
{% endtabs %}
136151

137-
`fluent-bit` will output
152+
Fluent Bit will output:
138153

139-
```
154+
```text
140155
[0] exec_oneshot_demo: [[1681702172.950574027, {}], {"exec"=>"count: 1"}]
141156
[1] exec_oneshot_demo: [[1681702173.951663666, {}], {"exec"=>"count: 2"}]
142157
[2] exec_oneshot_demo: [[1681702174.953873724, {}], {"exec"=>"count: 3"}]
@@ -149,27 +164,22 @@ pipeline:
149164
[9] exec_oneshot_demo: [[1681702181.965852990, {}], {"exec"=>"count: 10"}]
150165
```
151166

152-
then exit with exit code 1.
167+
then exits with exit code 1.
153168

154-
Translation of command exit code(s) to `fluent-bit` exit code follows
155-
[the usual shell rules for exit code handling](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html).
156-
Like with a shell, there is no way to differentiate between the command exiting
157-
on a signal and the shell exiting on a signal, and no way to differentiate
158-
between normal exits with codes greater than 125 and abnormal or signal exits
159-
reported by fluent-bit or the shell. Wrapped commands should use exit codes
160-
between 0 and 125 inclusive to allow reliable identification of normal exit.
161-
If the command is a pipeline, the exit code will be the exit code of the last
162-
command in the pipeline unless overridden by shell options.
169+
Translation of command exit codes to Fluent Bit exit code follows [the usual shell rules for exit code handling](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html). Like with a shell, there is no way to differentiate between the command exiting on a signal and the shell exiting on a signal. Similarly, there is no way to differentiate between normal exits with codes greater than `125` and abnormal or signal exits reported by Fluent Bit or the shell. Wrapped commands should use exit codes between `0` and `125` inclusive to allow reliable identification of normal exit. If the command is a pipeline, the exit code will be the exit code of the last command in the pipeline unless overridden by shell options.
163170

164171
### Parsing command output
165172

166-
By default the `exec` plugin emits one message per command output line, with a
167-
single field `exec` containing the full message. Use the `Parser` directive to
168-
specify the name of a parser configuration to use to process the command input.
173+
By default the `exec` plugin emits one message per command output line, with a single field `exec` containing the full message. Use the `Parser` directive to specify the name of a parser configuration to use to process the command input.
169174

170175
### Security concerns
171176

172-
**Take great care with shell quoting and escaping when wrapping commands**.
177+
{% hint style="warning" %}
178+
179+
Take great care with shell quoting and escaping when wrapping commands**.
180+
181+
{% endhint %}
182+
173183
A script like
174184

175185
```bash
@@ -186,11 +196,10 @@ exec fluent-bit \
186196
can ruin your day if someone passes it the argument
187197
`$(rm -rf /my/important/files; echo "deleted your stuff!")'`
188198

189-
The above script would be safer if written with:
199+
The previous script would be safer if written with:
190200

191201
```bash
192202
-p command='echo '"$(printf '%q' "$@")" \
193203
```
194204

195-
... but it's generally best to avoid dynamically generating the command or
196-
handling untrusted arguments to it at all.
205+
It's generally best to avoid dynamically generating the command or handling untrusted arguments.

0 commit comments

Comments
 (0)