You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pipeline/outputs/forward.md
+71-66Lines changed: 71 additions & 66 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,54 +1,54 @@
1
1
# Forward
2
2
3
-
_Forward_ is the protocol used by [Fluentd](http://www.fluentd.org) to route messages between peers. The **forward** output plugin provides interoperability between [Fluent Bit](http://fluentbit.io) and [Fluentd](http://fluentd.org).
3
+
_Forward_ is the protocol used by [Fluentd](http://www.fluentd.org) to route messages between peers. The `forward` output plugin provides interoperability between [Fluent Bit](http://fluentbit.io) and [Fluentd](http://fluentd.org).
4
4
There are no configuration steps required besides specifying where [Fluentd](http://fluentd.org) is located, which can be a local or a remote destination.
5
5
6
-
This plugin offers two different transports and modes:
6
+
This plugin offers the following transports and modes:
7
7
8
-
* Forward (TCP): It uses a plain TCP connection.
9
-
* Secure Forward (TLS): when TLS is enabled, the plugin switch to Secure Forward mode.
8
+
- Forward (TCP): Uses a plain TCP connection.
9
+
- Secure Forward (TLS): When TLS is enabled, the plugin switches to Secure Forward mode.
10
10
11
-
## Configuration Parameters
11
+
## Configuration parameters
12
12
13
-
The following parameters are mandatory for either Forward for Secure Forward modes:
13
+
The following parameters are mandatory for both Forward and Secure Forward modes:
| Host| Target host where Fluent-Bit or Fluentd are listening for Forward messages. |127.0.0.1 |
18
-
| Port| TCP Port of the target service. |24224|
19
-
| Time_as_Integer| Set timestamps in integer format, it enable compatibility mode for Fluentd v0.12 series. |False|
20
-
| Upstream| If Forward will connect to an _Upstream_ instead of a simple host, this property defines the absolute path for the Upstream configuration file, for more details about this refer to the [Upstream Servers ](../../administration/configuring-fluent-bit/classic-mode/upstream-servers.md)documentation section. ||
21
-
| Unix_Path| Specify the path to unix socket to send a Forward message. If set, `Upstream` is ignored. ||
22
-
| Tag| Overwrite the tag as we transmit. This allows the receiving pipeline start fresh, or to attribute source. ||
23
-
| Send_options| Always send options (with "size"=count of messages) |False|
24
-
| Require_ack_response | Send "chunk"-option and wait for "ack" response from server. Enables at-least-once and receiving server can control rate of traffic. (Requires Fluentd v0.14.0+ server) |False|
25
-
| Compress| Set to 'gzip' to enable gzip compression. Incompatible with `Time_as_Integer=True` and tags set dynamically using the [Rewrite Tag](../filters/rewrite-tag.md) filter. Requires Fluentd server v0.14.7 or later. |_none_|
26
-
| Workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. |`2`|
15
+
| Key | Description | Default |
16
+
| --- | ------------ | --------- |
17
+
|`Host`| Target host where FluentBit or Fluentd are listening for Forward messages. |`127.0.0.1`|
18
+
|`Port`| TCP Port of the target service. |`24224`|
19
+
|`Time_as_Integer`| Set timestamps in integer format, it enables compatibility mode for Fluentd v0.12 series. |`False`|
20
+
|`Upstream`| If Forward will connect to an `Upstream` instead of a basic host, this property defines the absolute path for the Upstream configuration file, for more details about this, see [Upstream Servers ](../../administration/configuring-fluent-bit/classic-mode/upstream-servers.md). |_none_|
21
+
|`Unix_Path`| Specify the path to a Unix socket to send a Forward message. If set, `Upstream` is ignored. |_none_|
22
+
|`Tag`| Overwrite the tag as Fluent Bit transmits. This allows the receiving pipeline start fresh, or to attribute a source. |_none_|
23
+
|`Send_options`| Always send options (with `"size"=count of messages`) |`False`|
24
+
|`Require_ack_response`| Send `chunk`option and wait for an `ack` response from the server. Enables at-least-once and receiving server can control rate of traffic. Requires Fluentd v0.14.0+ or later |`False`|
25
+
|`Compress`| Set to `gzip` to enable gzip compression. Incompatible with `Time_as_Integer=True` and tags set dynamically using the [Rewrite Tag](../filters/rewrite-tag.md) filter. Requires Fluentd server v0.14.7 or later. |_none_|
26
+
|`Workers`| The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. |`2`|
27
27
28
-
## Secure Forward Mode Configuration Parameters
28
+
## Secure Forward mode configuration parameters
29
29
30
-
When using Secure Forward mode, the [TLS](../../administration/transport-security.md) mode requires to be enabled. The following additional configuration parameters are available:
30
+
When using Secure Forward mode, the [TLS](../../administration/transport-security.md) mode must be enabled. The following additional configuration parameters are available:
| Shared_Key| A key string known by the remote Fluentd used for authorization. ||
35
-
| Empty_Shared_Key | Use this option to connect to Fluentd with a zero-length secret. |False|
36
-
| Username| Specify the username to present to a Fluentd server that enables `user_auth`. ||
37
-
| Password| Specify the password corresponding to the username. ||
38
-
| Self_Hostname| Default value of the auto-generated certificate common name (CN). |localhost |
39
-
| tls| Enable or disable TLS support|Off|
40
-
| tls.verify| Force certificate validation| On |
41
-
| tls.debug| Set TLS debug verbosity level. It accept the following values: 0 (No debug), 1 (Error), 2 (State change), 3 (Informational) and 4 Verbose |1 |
42
-
| tls.ca_file| Absolute path to CA certificate file||
43
-
| tls.crt_file| Absolute path to Certificate file. ||
44
-
| tls.key_file| Absolute path to private Key file. ||
45
-
| tls.key_passwd| Optional password for tls.key_file file. ||
32
+
| Key | Description | Default |
33
+
| --- | ----------- | ------- |
34
+
|`Shared_Key`| A key string known by the remote Fluentd used for authorization. |_none_|
35
+
|`Empty_Shared_Key`| Use this option to connect to Fluentd with a zero-length secret. |`False`|
36
+
|`Username`| Specify the username to present to a Fluentd server that enables `user_auth`. |_none_|
37
+
|`Password`| Specify the password corresponding to the username. |_none_|
38
+
|`Self_Hostname`| Default value of the auto-generated certificate common name (CN). |`localhost`|
39
+
|`tls`| Enable or disable TLS support. |`Off`|
40
+
|`tls.verify`| Force certificate validation. |`On`|
41
+
|`tls.debug`| Set TLS debug verbosity level. Allowed values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational), and `4` (Verbose).|`1`|
42
+
|`tls.ca_file`| Absolute path to CA certificate file. |_none_|
43
+
|`tls.crt_file`| Absolute path to Certificate file. |_none_|
44
+
|`tls.key_file`| Absolute path to private Key file. |_none_|
45
+
|`tls.key_passwd`| Optional password for `tls.key_file`. |_none_|
46
46
47
-
## Forward Setup
47
+
## Forward setup
48
48
49
-
Before proceeding, make sure that [Fluentd](http://fluentd.org) is installed, if it's not the case please refer to the following [Fluentd Installation](http://docs.fluentd.org/v0.12/categories/installation) document and go ahead with that.
49
+
Before proceeding, ensure that [Fluentd](http://fluentd.org) is installed. If it's not, refer to the [Fluentd Installation](http://docs.fluentd.org/installation) document.
50
50
51
-
Once [Fluentd](http://fluentd.org) is installed, create the following configuration file example that will allow us to stream data into it:
51
+
After installing Fluentd, create the following configuration file example which lets you to stream data into it:
52
52
53
53
```text
54
54
<source>
@@ -62,14 +62,19 @@ Once [Fluentd](http://fluentd.org) is installed, create the following configurat
62
62
</match>
63
63
```
64
64
65
-
That configuration file specifies that it will listen for _TCP_ connections on the port _24224_ through the **forward** input type.
66
-
Then for every message with a _fluent_bit_**TAG**, will print the message to the standard output.
65
+
That configuration file specifies that it will listen for TCP connections on port `24224` through the `forward` input type.
67
66
68
-
In one terminal launch [Fluentd](http://fluentd.org) specifying the new configuration file created:
67
+
Every message with a `fluent_bit` tag will print a message to the standard output.
68
+
69
+
In one terminal, launch Fluentd while specifying the new configuration file created:
69
70
70
71
```shell
71
-
$ fluentd -c test.conf
72
+
fluentd -c test.conf
73
+
```
74
+
75
+
Which should return a response similar to the following:
## Fluent Bit + Forward Setup <ahref="forward_setup"id="forward_setup"></a>
111
+
## Fluent Bit and Forward setup
107
112
108
-
Now that [Fluentd](http://fluentd.org) is ready to receive messages, we need to specify where the **forward** output plugin will flush the information using the following format:
113
+
When Fluentd is ready to receive messages, specify where the `forward` output plugin will flush the information using the following format:
109
114
110
115
```shell
111
116
fluent-bit -i INPUT -o forward://HOST:PORT
112
117
```
113
118
114
-
If the **TAG** parameter is not set, the plugin will retain the tag.
115
-
Keep in mind that **TAG**is important for routing rules inside [Fluentd](http://fluentd.org).
119
+
If the `tag` parameter isn't set, the plugin will retain the tag.
120
+
The `tag`is important for routing rules inside Fluentd.
116
121
117
-
Using the [CPU](../inputs/cpu-metrics.md) input plugin as an example we will flush CPU metrics to [Fluentd](http://fluentd.org)with tag_fluent_bit_:
122
+
Using the [CPU](../inputs/cpu-metrics.md) input plugin as an example, you can flush `cpu` metrics with the `tag``fluent_bit` to Fluentd:
118
123
119
124
```shell
120
125
fluent-bit -i cpu -t fluent_bit -o forward://127.0.0.1:24224
121
126
```
122
127
123
-
Now on the [Fluentd](http://fluentd.org) side, you will see the CPU metrics gathered in the last seconds:
128
+
In Fluentd, you will see the CPU metrics gathered in the last seconds:
124
129
125
130
```text
126
131
...
@@ -131,13 +136,13 @@ Now on the [Fluentd](http://fluentd.org) side, you will see the CPU metrics gath
131
136
...
132
137
```
133
138
134
-
So we gathered[CPU](../inputs/cpu-metrics.md) metrics and flushed them out to [Fluentd](http://fluentd.org) properly.
139
+
This shows that[CPU](../inputs/cpu-metrics.md) metrics were gathered and flushed out to Fluentd properly.
135
140
136
-
## Fluent Bit + Secure Forward Setup <ahref="secure_forward_setup"id="secure_forward_setup"></a>
141
+
## Fluent Bit and Secure Forward setup
137
142
138
-
> DISCLAIMER: the following example does not consider the generation of certificates for best practice on production environments.
143
+
The following example doesn't consider the generation of certificates for best practice on production environments.
139
144
140
-
Secure Forward aims to provide a secure channel of communication with the remote Fluentd service using [TLS](../../administration/transport-security.md).
145
+
Secure Forward provides a secure channel of communication with the remote Fluentd service using [TLS](../../administration/transport-security.md).
141
146
142
147
### Fluent Bit
143
148
@@ -151,12 +156,12 @@ service:
151
156
flush: 5
152
157
daemon: off
153
158
log_level: info
154
-
159
+
155
160
pipeline:
156
161
inputs:
157
162
- name: cpu
158
163
tag: cpu_usage
159
-
164
+
160
165
outputs:
161
166
- name: forward
162
167
match: '*'
@@ -212,23 +217,23 @@ Paste this content in a file called `fld.conf`:
212
217
</match>
213
218
```
214
219
215
-
### Test Communication
220
+
### Test communication
216
221
217
-
Start Fluentd:
222
+
1.Start Fluentd:
218
223
219
-
```shell
220
-
fluentd -c fld.conf
221
-
```
224
+
```shell
225
+
fluentd -c fld.conf
226
+
```
222
227
223
-
Start Fluent Bit:
228
+
1.Start Fluent Bit:
224
229
225
-
```shell
226
-
# For YAML configuration.
227
-
fluent-bit --config flb.yaml
230
+
```shell
231
+
# For YAML configuration.
232
+
fluent-bit --config flb.yaml
228
233
229
-
# For classic configuration
230
-
fluent-bit --config flb.conf
231
-
```
234
+
# For classic configuration
235
+
fluent-bit --config flb.conf
236
+
```
232
237
233
238
After five seconds, Fluent Bit will write records to Fluentd.
234
239
In Fluentd output you will see a message like this:
@@ -251,4 +256,4 @@ In Fluentd output you will see a message like this:
0 commit comments