Skip to content

Commit 2393c88

Browse files
pwhelanpwhelan
authored
Add documentation for new trace command line. (#1166)
Document the new trace command line options introduced by fluent/fluent-bit#6726. Signed-off-by: Phillip Whelan <[email protected]>
1 parent ae337ad commit 2393c88

File tree

1 file changed

+113
-4
lines changed

1 file changed

+113
-4
lines changed

‎administration/troubleshooting.md‎

Lines changed: 113 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,128 @@
88
Tap can be used to generate events or records detailing what messages
99
pass through Fluent Bit, at what time and what filters affect them.
1010

11-
1211
### Simple example
1312

1413
First, we will make sure that the container image we are going to use actually supports Fluent Bit Tap (available in Fluent Bit 2.0+):
1514

1615
```shell
17-
$ docker run --rm -ti fluent/fluent-bit:latest --help | grep -Z
18-
-Z, --enable-chunk-trace enable chunk tracing. activating it requires using the HTTP Server API.
16+
$ docker run --rm -ti fluent/fluent-bit:latest --help | grep trace
17+
-Z, --enable-chunk-traceenable chunk tracing, it can be activated either through the http api or the command line
18+
--trace-input input to start tracing on startup.
19+
--trace-output output to use for tracing on startup.
20+
--trace-output-property set a property for output tracing on startup.
21+
--trace setup a trace pipeline on startup. Uses a single line, ie: "input=dummy.0 output=stdout output.format='json'"
1922
```
2023

2124
If the `--enable-chunk-trace` option is present it means Fluent Bit has support for Fluent Bit Tap but it is disabled by default, so remember to enable it with this option.
2225

23-
Tap support is enabled and disabled via the embedded web server, so enable it like so (or the equivalent option in the configuration file):
26+
You can start fluent-bit with tracing activated from the beginning by using the `trace-input` and `trace-output` properties, like so:
27+
28+
```bash
29+
$ fluent-bit -Z -i dummy -o stdout -f 1 --trace-input=dummy.0 --trace-output=stdout
30+
Fluent Bit v2.1.8
31+
* Copyright (C) 2015-2022 The Fluent Bit Authors
32+
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
33+
* https://fluentbit.io
34+
35+
[2023/07/21 16:27:01] [ info] [fluent bit] version=2.1.8, commit=824ba3dd08, pid=622937
36+
[2023/07/21 16:27:01] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
37+
[2023/07/21 16:27:01] [ info] [cmetrics] version=0.6.3
38+
[2023/07/21 16:27:01] [ info] [ctraces ] version=0.3.1
39+
[2023/07/21 16:27:01] [ info] [input:dummy:dummy.0] initializing
40+
[2023/07/21 16:27:01] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
41+
[2023/07/21 16:27:01] [ info] [sp] stream processor started
42+
[2023/07/21 16:27:01] [ info] [output:stdout:stdout.0] worker #0 started
43+
[2023/07/21 16:27:01] [ info] [fluent bit] version=2.1.8, commit=824ba3dd08, pid=622937
44+
[2023/07/21 16:27:01] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
45+
[2023/07/21 16:27:01] [ info] [cmetrics] version=0.6.3
46+
[2023/07/21 16:27:01] [ info] [ctraces ] version=0.3.1
47+
[2023/07/21 16:27:01] [ info] [input:emitter:trace-emitter] initializing
48+
[2023/07/21 16:27:01] [ info] [input:emitter:trace-emitter] storage_strategy='memory' (memory only)
49+
[2023/07/21 16:27:01] [ info] [sp] stream processor started
50+
[2023/07/21 16:27:01] [ info] [output:stdout:stdout.0] worker #0 started
51+
.[0] dummy.0: [[1689971222.068537501, {}], {"message"=>"dummy"}]
52+
[0] dummy.0: [[1689971223.068556121, {}], {"message"=>"dummy"}]
53+
[0] trace: [[1689971222.068677045, {}], {"type"=>1, "trace_id"=>"0", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971222, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971222, "end_time"=>1689971222}]
54+
[1] trace: [[1689971222.068735577, {}], {"type"=>3, "trace_id"=>"0", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971222, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971222, "end_time"=>1689971222}]
55+
[0] dummy.0: [[1689971224.068586317, {}], {"message"=>"dummy"}]
56+
[0] trace: [[1689971223.068626923, {}], {"type"=>1, "trace_id"=>"1", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971223, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971223, "end_time"=>1689971223}]
57+
[1] trace: [[1689971223.068675735, {}], {"type"=>3, "trace_id"=>"1", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971223, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971223, "end_time"=>1689971223}]
58+
[2] trace: [[1689971224.068689341, {}], {"type"=>1, "trace_id"=>"2", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971224, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971224, "end_time"=>1689971224}]
59+
[3] trace: [[1689971224.068747182, {}], {"type"=>3, "trace_id"=>"2", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971224, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971224, "end_time"=>1689971224}]
60+
^C[2023/07/21 16:27:05] [engine] caught signal (SIGINT)
61+
[2023/07/21 16:27:05] [ warn] [engine] service will shutdown in max 5 seconds
62+
[2023/07/21 16:27:05] [ info] [input] pausing dummy.0
63+
[0] dummy.0: [[1689971225.068568875, {}], {"message"=>"dummy"}]
64+
[2023/07/21 16:27:06] [ info] [engine] service has stopped (0 pending tasks)
65+
[2023/07/21 16:27:06] [ info] [input] pausing dummy.0
66+
[2023/07/21 16:27:06] [ warn] [engine] service will shutdown in max 1 seconds
67+
[0] trace: [[1689971225.068654038, {}], {"type"=>1, "trace_id"=>"3", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971225, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971225, "end_time"=>1689971225}]
68+
[1] trace: [[1689971225.068695829, {}], {"type"=>3, "trace_id"=>"3", "plugin_instance"=>"dummy.0", "records"=>[{"timestamp"=>1689971225, "record"=>{"message"=>"dummy"}}], "start_time"=>1689971225, "end_time"=>1689971225}]
69+
[2023/07/21 16:27:07] [ info] [engine] service has stopped (0 pending tasks)
70+
[2023/07/21 16:27:07] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
71+
[2023/07/21 16:27:07] [ info] [output:stdout:stdout.0] thread worker #0 stopped
72+
[2023/07/21 16:27:07] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
73+
[2023/07/21 16:27:07] [ info] [output:stdout:stdout.0] thread worker #0 stopped
74+
```
75+
76+
If you see the following warning then the `-Z` or `--enable-chunk-tracing` option is missing:
77+
78+
```bash
79+
[2023/07/21 16:26:42] [ warn] [chunk trace] enable chunk tracing via the configuration or command line to be able to activate tracing.
80+
```
81+
82+
Properties can be set for the output using the `--trace-output-property` option:
83+
84+
```bash
85+
$ fluent-bit -Z -i dummy -o stdout -f 1 --trace-input=dummy.0 --trace-output=stdout --trace-output-property=format=json_lines
86+
Fluent Bit v2.1.8
87+
* Copyright (C) 2015-2022 The Fluent Bit Authors
88+
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
89+
* https://fluentbit.io
90+
91+
[2023/07/21 16:28:59] [ info] [fluent bit] version=2.1.8, commit=824ba3dd08, pid=623170
92+
[2023/07/21 16:28:59] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
93+
[2023/07/21 16:28:59] [ info] [cmetrics] version=0.6.3
94+
[2023/07/21 16:28:59] [ info] [ctraces ] version=0.3.1
95+
[2023/07/21 16:28:59] [ info] [input:dummy:dummy.0] initializing
96+
[2023/07/21 16:28:59] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
97+
[2023/07/21 16:28:59] [ info] [sp] stream processor started
98+
[2023/07/21 16:28:59] [ info] [output:stdout:stdout.0] worker #0 started
99+
[2023/07/21 16:28:59] [ info] [fluent bit] version=2.1.8, commit=824ba3dd08, pid=623170
100+
[2023/07/21 16:28:59] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
101+
[2023/07/21 16:28:59] [ info] [cmetrics] version=0.6.3
102+
[2023/07/21 16:28:59] [ info] [ctraces ] version=0.3.1
103+
[2023/07/21 16:28:59] [ info] [input:emitter:trace-emitter] initializing
104+
[2023/07/21 16:28:59] [ info] [input:emitter:trace-emitter] storage_strategy='memory' (memory only)
105+
[2023/07/21 16:29:00] [ info] [sp] stream processor started
106+
[2023/07/21 16:29:00] [ info] [output:stdout:stdout.0] worker #0 started
107+
.[0] dummy.0: [[1689971340.068565891, {}], {"message"=>"dummy"}]
108+
[0] dummy.0: [[1689971341.068632477, {}], {"message"=>"dummy"}]
109+
{"date":1689971340.068745,"type":1,"trace_id":"0","plugin_instance":"dummy.0","records":[{"timestamp":1689971340,"record":{"message":"dummy"}}],"start_time":1689971340,"end_time":1689971340}
110+
{"date":1689971340.068825,"type":3,"trace_id":"0","plugin_instance":"dummy.0","records":[{"timestamp":1689971340,"record":{"message":"dummy"}}],"start_time":1689971340,"end_time":1689971340}
111+
[0] dummy.0: [[1689971342.068613646, {}], {"message"=>"dummy"}]
112+
```
113+
114+
With that options set the stdout plugin is now emitting traces in `json_lines` format:
115+
116+
```json
117+
{"date":1689971340.068745,"type":1,"trace_id":"0","plugin_instance":"dummy.0","records":[{"timestamp":1689971340,"record":{"message":"dummy"}}],"start_time":1689971340,"end_time":1689971340}
118+
```
119+
120+
All three options can also be defined using the much more flexible `--trace` option:
121+
122+
```bash
123+
$ fluent-bit -Z -i dummy -o stdout -f 1 --trace="input=dummy.0 output=stdout output.format=json_lines"
124+
```
125+
126+
We defined the entire tap pipeline using this configuration: `input=dummy.0 output=stdout output.format=json_lines` which defines the following:
127+
128+
* input: dummy.0 (listens to the tag and/or alias `dummy.0`)
129+
* output: stdout (outputs to a stdout plugin)
130+
* output.format: json_lines (sets the stdout format o `json_lines`)
131+
132+
Tap support can also be activated and deactivated via the embedded web server:
24133

25134
```shell
26135
$ docker run --rm -ti -p 2020:2020 fluent/fluent-bit:latest -Z -H -i dummy -p alias=input_dummy -o stdout -f 1

0 commit comments

Comments
 (0)