Skip to content

Commit f26e311

Browse files
committed
Adding YAML examples and layout fixes for Forward output plugin. Part of issue #1909.
Signed-off-by: Eric D. Schabell <[email protected]>
1 parent 923997b commit f26e311

File tree

1 file changed

+73
-52
lines changed

1 file changed

+73
-52
lines changed

pipeline/outputs/forward.md

Lines changed: 73 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ Before proceeding, make sure that [Fluentd](http://fluentd.org) is installed, if
5050

5151
Once [Fluentd](http://fluentd.org) is installed, create the following configuration file example that will allow us to stream data into it:
5252

53-
```
53+
```text
5454
<source>
5555
type forward
5656
bind 0.0.0.0
@@ -67,10 +67,12 @@ Then for every message with a _fluent_bit_ **TAG**, will print the message to th
6767

6868
In one terminal launch [Fluentd](http://fluentd.org) specifying the new configuration file created:
6969

70-
```bash
70+
```shell
7171
$ fluentd -c test.conf
72+
73+
...
7274
2017-03-23 11:50:43 -0600 [info]: reading config file path="test.conf"
73-
2017-03-23 11:50:43 -0600 [info]: starting fluentd-0.12.33
75+
...
7476
2017-03-23 11:50:43 -0600 [info]: gem 'fluent-mixin-config-placeholders' version '0.3.1'
7577
2017-03-23 11:50:43 -0600 [info]: gem 'fluent-plugin-docker' version '0.1.0'
7678
2017-03-23 11:50:43 -0600 [info]: gem 'fluent-plugin-elasticsearch' version '1.4.0'
@@ -98,32 +100,35 @@ $ fluentd -c test.conf
98100
</match>
99101
</ROOT>
100102
2017-03-23 11:50:43 -0600 [info]: listening fluent socket on 0.0.0.0:24224
103+
...
101104
```
102105

103106
## Fluent Bit + Forward Setup <a href="forward_setup" id="forward_setup"></a>
104107

105108
Now that [Fluentd](http://fluentd.org) is ready to receive messages, we need to specify where the **forward** output plugin will flush the information using the following format:
106109

107-
```
108-
bin/fluent-bit -i INPUT -o forward://HOST:PORT
110+
```shell
111+
fluent-bit -i INPUT -o forward://HOST:PORT
109112
```
110113

111114
If the **TAG** parameter is not set, the plugin will retain the tag.
112115
Keep in mind that **TAG** is important for routing rules inside [Fluentd](http://fluentd.org).
113116

114117
Using the [CPU](../inputs/cpu-metrics.md) input plugin as an example we will flush CPU metrics to [Fluentd](http://fluentd.org) with tag _fluent_bit_:
115118

116-
```bash
117-
bin/fluent-bit -i cpu -t fluent_bit -o forward://127.0.0.1:24224
119+
```shell
120+
fluent-bit -i cpu -t fluent_bit -o forward://127.0.0.1:24224
118121
```
119122

120123
Now on the [Fluentd](http://fluentd.org) side, you will see the CPU metrics gathered in the last seconds:
121124

122-
```bash
125+
```text
126+
...
123127
2017-03-23 11:53:06 -0600 fluent_bit: {"cpu_p":0.0,"user_p":0.0,"system_p":0.0,"cpu0.p_cpu":0.0,"cpu0.p_user":0.0,"cpu0.p_system":0.0,"cpu1.p_cpu":0.0,"cpu1.p_user":0.0,"cpu1.p_system":0.0,"cpu2.p_cpu":0.0,"cpu2.p_user":0.0,"cpu2.p_system":0.0,"cpu3.p_cpu":1.0,"cpu3.p_user":1.0,"cpu3.p_system":0.0}
124128
2017-03-23 11:53:07 -0600 fluent_bit: {"cpu_p":2.25,"user_p":2.0,"system_p":0.25,"cpu0.p_cpu":3.0,"cpu0.p_user":3.0,"cpu0.p_system":0.0,"cpu1.p_cpu":1.0,"cpu1.p_user":1.0,"cpu1.p_system":0.0,"cpu2.p_cpu":1.0,"cpu2.p_user":1.0,"cpu2.p_system":0.0,"cpu3.p_cpu":3.0,"cpu3.p_user":2.0,"cpu3.p_system":1.0}
125129
2017-03-23 11:53:08 -0600 fluent_bit: {"cpu_p":1.75,"user_p":1.0,"system_p":0.75,"cpu0.p_cpu":2.0,"cpu0.p_user":1.0,"cpu0.p_system":1.0,"cpu1.p_cpu":3.0,"cpu1.p_user":1.0,"cpu1.p_system":2.0,"cpu2.p_cpu":3.0,"cpu2.p_user":2.0,"cpu2.p_system":1.0,"cpu3.p_cpu":2.0,"cpu3.p_user":1.0,"cpu3.p_system":1.0}
126130
2017-03-23 11:53:09 -0600 fluent_bit: {"cpu_p":4.75,"user_p":3.5,"system_p":1.25,"cpu0.p_cpu":4.0,"cpu0.p_user":3.0,"cpu0.p_system":1.0,"cpu1.p_cpu":5.0,"cpu1.p_user":4.0,"cpu1.p_system":1.0,"cpu2.p_cpu":3.0,"cpu2.p_user":2.0,"cpu2.p_system":1.0,"cpu3.p_cpu":5.0,"cpu3.p_user":4.0,"cpu3.p_system":1.0}
131+
...
127132
```
128133

129134
So we gathered [CPU](../inputs/cpu-metrics.md) metrics and flushed them out to [Fluentd](http://fluentd.org) properly.
@@ -136,34 +141,65 @@ Secure Forward aims to provide a secure channel of communication with the remote
136141

137142
### Fluent Bit
138143

139-
Paste this content in a file called _flb.conf_:
140-
141-
```
144+
Paste this content in a file called `flb` :
145+
146+
{% tabs %}
147+
{% tab title="flb.yaml" %}
148+
149+
```yaml
150+
service:
151+
flush: 5
152+
daemon: off
153+
log_level: info
154+
155+
pipeline:
156+
inputs:
157+
- name: cpu
158+
tag: cpu_usage
159+
160+
outputs:
161+
- name: forward
162+
match: '*'
163+
host: 127.0.0.1
164+
port: 24284
165+
shared_key: secret
166+
self_hostname: flb.local
167+
tls: on
168+
tls.verify: off
169+
```
170+
171+
{% endtab %}
172+
{% tab title="flb.conf" %}
173+
174+
```text
142175
[SERVICE]
143-
Flush 5
144-
Daemon off
145-
Log_Level info
176+
Flush 5
177+
Daemon off
178+
Log_Level info
146179

147180
[INPUT]
148-
Name cpu
149-
Tag cpu_usage
181+
Name cpu
182+
Tag cpu_usage
150183

151184
[OUTPUT]
152-
Name forward
153-
Match *
154-
Host 127.0.0.1
155-
Port 24284
156-
Shared_Key secret
157-
Self_Hostname flb.local
158-
tls on
159-
tls.verify off
185+
Name forward
186+
Match *
187+
Host 127.0.0.1
188+
Port 24284
189+
Shared_Key secret
190+
Self_Hostname flb.local
191+
tls on
192+
tls.verify off
160193
```
161194

195+
{% endtab %}
196+
{% endtabs %}
197+
162198
### Fluentd
163199

164-
Paste this content in a file called _fld.conf_:
200+
Paste this content in a file called `fld.conf`:
165201

166-
```
202+
```text
167203
<source>
168204
@type secure_forward
169205
self_hostname myserver.local
@@ -176,45 +212,29 @@ Paste this content in a file called _fld.conf_:
176212
</match>
177213
```
178214

179-
If you're using Fluentd v1, set up it as below:
180-
181-
```
182-
<source>
183-
@type forward
184-
<transport tls>
185-
cert_path /etc/td-agent/certs/fluentd.crt
186-
private_key_path /etc/td-agent/certs/fluentd.key
187-
private_key_passphrase password
188-
</transport>
189-
<security>
190-
self_hostname myserver.local
191-
shared_key secret
192-
</security>
193-
</source>
194-
195-
<match **>
196-
@type stdout
197-
</match>
198-
```
199-
200215
### Test Communication
201216

202217
Start Fluentd:
203218

204-
```
219+
```shell
205220
fluentd -c fld.conf
206221
```
207222

208223
Start Fluent Bit:
209224

210-
```
211-
fluent-bit -c flb.conf
225+
```shell
226+
# For YAML configuration.
227+
fluent-bit --config flb.yaml
228+
229+
# For classic configuration
230+
fluent-bit --config flb.conf
212231
```
213232

214233
After five seconds, Fluent Bit will write records to Fluentd.
215234
In Fluentd output you will see a message like this:
216235

217-
```
236+
```text
237+
...
218238
2017-03-23 13:34:40 -0600 [info]: using configuration file: <ROOT>
219239
<source>
220240
@type secure_forward
@@ -230,4 +250,5 @@ In Fluentd output you will see a message like this:
230250
2017-03-23 13:34:42 -0600 cpu_usage: {"cpu_p":1.75,"user_p":1.75,"system_p":0.0,"cpu0.p_cpu":3.0,"cpu0.p_user":3.0,"cpu0.p_system":0.0,"cpu1.p_cpu":2.0,"cpu1.p_user":2.0,"cpu1.p_system":0.0,"cpu2.p_cpu":0.0,"cpu2.p_user":0.0,"cpu2.p_system":0.0,"cpu3.p_cpu":1.0,"cpu3.p_user":1.0,"cpu3.p_system":0.0}
231251
2017-03-23 13:34:43 -0600 cpu_usage: {"cpu_p":1.75,"user_p":1.25,"system_p":0.5,"cpu0.p_cpu":3.0,"cpu0.p_user":3.0,"cpu0.p_system":0.0,"cpu1.p_cpu":2.0,"cpu1.p_user":2.0,"cpu1.p_system":0.0,"cpu2.p_cpu":0.0,"cpu2.p_user":0.0,"cpu2.p_system":0.0,"cpu3.p_cpu":1.0,"cpu3.p_user":0.0,"cpu3.p_system":1.0}
232252
2017-03-23 13:34:44 -0600 cpu_usage: {"cpu_p":5.0,"user_p":3.25,"system_p":1.75,"cpu0.p_cpu":4.0,"cpu0.p_user":2.0,"cpu0.p_system":2.0,"cpu1.p_cpu":8.0,"cpu1.p_user":5.0,"cpu1.p_system":3.0,"cpu2.p_cpu":4.0,"cpu2.p_user":3.0,"cpu2.p_system":1.0,"cpu3.p_cpu":4.0,"cpu3.p_user":2.0,"cpu3.p_system":2.0}
233-
```
253+
...
254+
```

0 commit comments

Comments
 (0)