Skip to content

Commit 926a04a

Browse files
esmerelesmerel
authored
Merge pull request #1944 from fluent/lynettemiles/sc-136243/update-fluent-bit-docs-pipeline-outputs-http
2 parents 4941433 + 77be884 commit 926a04a

File tree

2 files changed

+56
-50
lines changed

2 files changed

+56
-50
lines changed

‎pipeline/outputs/http.md

Lines changed: 54 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,52 +1,54 @@
11
# HTTP
22

3-
The **http** output plugin allows to flush your records into an HTTP endpoint. For now the functionality is pretty basic, and it issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.
4-
5-
## Configuration Parameters
6-
7-
| Key | Description | default |
8-
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- |
9-
| host | IP address or hostname of the target HTTP Server | 127.0.0.1 |
10-
| http\_User | Basic Auth Username | |
11-
| http\_Passwd | Basic Auth Password. Requires HTTP\_User to be set | |
12-
| AWS\_Auth | Enable AWS SigV4 authentication | false |
13-
| AWS\_Service | Specify the AWS service code, i.e. es, xray, etc., of your service, used by SigV4 authentication. Usually can be found in the service endpoint's subdomains, `protocol://service-code.region-code.amazonaws.com` | |
14-
| AWS\_Region | Specify the AWS region of your service, used by SigV4 authentication | |
15-
| AWS\_STS\_Endpoint | Specify the custom sts endpoint to be used with STS API, used with the AWS_Role_ARN option, used by SigV4 authentication | |
16-
| AWS\_Role\_ARN | AWS IAM Role to assume, used by SigV4 authentication | |
17-
| AWS\_External\_ID | External ID for the AWS IAM Role specified with `aws_role_arn`, used by SigV4 authentication | |
18-
| port | TCP port of the target HTTP Server | 80 |
19-
| Proxy | Specify an HTTP Proxy. The expected format of this value is `http://HOST:PORT`. Note that HTTPS is **not** currently supported. It is recommended not to set this and to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | |
20-
| uri | Specify an optional HTTP URI for the target web server, e.g: /something | / |
21-
| compress | Set payload compression mechanism. Option available is 'gzip' | |
22-
| format | Specify the data format to be used in the HTTP request body, by default it uses _msgpack_. Other supported formats are _json_, _json\_stream_ and _json\_lines_ and _gelf_. | msgpack |
23-
| allow\_duplicated\_headers | Specify if duplicated headers are allowed. If a duplicated header is found, the latest key/value set is preserved. | true |
24-
| log\_response\_payload | Specify if the response payload should be logged or not. | true |
25-
| header\_tag | Specify an optional HTTP header field for the original message tag. | |
26-
| header | Add a HTTP header key/value pair. Multiple headers can be set. | |
27-
| json\_date\_key | Specify the name of the time key in the output record. To disable the time key just set the value to `false`. | date |
28-
| json\_date\_format | Specify the format of the date. Supported formats are _double_, _epoch_, _iso8601_ (eg: _2018-05-30T09:39:52.000681Z_) and _java_sql_timestamp_ (eg: _2018-05-30 09:39:52.000681_) | double |
29-
| gelf\_timestamp\_key | Specify the key to use for `timestamp` in _gelf_ format | |
30-
| gelf\_host\_key | Specify the key to use for the `host` in _gelf_ format | |
31-
| gelf\_short\_message\_key | Specify the key to use as the `short` message in _gelf_ format | |
32-
| gelf\_full\_message\_key | Specify the key to use for the `full` message in _gelf_ format | |
33-
| gelf\_level\_key | Specify the key to use for the `level` in _gelf_ format | |
34-
| body\_key | Specify the key to use as the body of the request (must prefix with "$"). The key must contain either a binary or raw string, and the content type can be specified using headers\_key (which must be passed whenever body\_key is present). When this option is present, each msgpack record will create a separate request. | |
35-
| headers\_key | Specify the key to use as the headers of the request (must prefix with "$"). The key must contain a map, which will have the contents merged on the request headers. This can be used for many purposes, such as specifying the content-type of the data contained in body\_key. | |
36-
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` |
3+
The _HTTP_ output plugin lets you flush your records into an HTTP endpoint. It issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.
4+
5+
## Configuration parameters
6+
7+
The plugin supports the following configuration parameters:
8+
9+
| Key | Description | Default |
10+
| --- | ----------- | -------- |
11+
| `host` | IP address or hostname of the target HTTP Server. | `127.0.0.1` |
12+
| `http_User` | Basic Auth username. | _none_ |
13+
| `http_Passwd` | Basic Auth password. Requires `HTTP_User` to be set. | _none_ |
14+
| `AWS_Auth` | Enable AWS SigV4 authentication. | `false` |
15+
| `AWS_Service` | Specify the AWS service code of your service, used by SigV4 authentication (for example, `es`, `xray`.) Usually found in the service endpoint's subdomains, `protocol://service-code.region-code.amazonaws.com`. | _none_ |
16+
| `AWS_Region` | Specify the AWS region of your service, used by SigV4 authentication. | _none_ |
17+
| `AWS_STS_Endpoint` | Specify the custom STS endpoint to be used by STS API. Used with the `AWS_Role_ARN option` and by SigV4 authentication. | _none_ |
18+
| `AWS_Role_ARN` | AWS IAM Role to assume, used by SigV4 authentication. | _none_ |
19+
| `AWS_External_ID` | External ID for the AWS IAM Role specified with `aws_role_arn`, used by SigV4 authentication. | _none_ |
20+
| `port` | TCP port of the target HTTP Server. | `80` |
21+
| `Proxy` | Specify an HTTP Proxy. The expected format of this value is `http://HOST:PORT`. HTTPS isn't supported. It's recommended to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | _none_ |
22+
| `uri` | Specify an optional HTTP URI for the target web server. For example, `/somepath`. | `/` |
23+
| `compress` | Set payload compression mechanism. Allowed value: `gzip`. | _none_ |
24+
| `format` | Specify the data format to be used in the HTTP request body. Supported formats: `gelf`, `json`, `json_stream`, `json_lines`, `msgpack`. | `msgpack` |
25+
| `allow_duplicated_headers` | Specify if duplicated headers are allowed. If a duplicated header is found, the latest key/value set is preserved. | `true` |
26+
| `log_response_payload` | Specify if the response payload should be logged or not. | `true` |
27+
| `header_tag` | Specify an optional HTTP header field for the original message tag. | _none_ |
28+
| `header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ |
29+
| `json_date_key` | Specify the name of the time key in the output record. To disable the time key, set the value to `false`. | `date` |
30+
| `json_date_format` | Specify the format of the date. Supported formats: `double`, `epoch`, `iso8601`, `java_sql_timestamp`. | `double` |
31+
| `gelf_timestamp_key` | Specify the key to use for `timestamp` in `gelf` format. | _none_ |
32+
| `gelf_host_key` | Specify the key to use for the `host` in `gelf` format. | _none_ |
33+
| `gelf_short_message_key` | Specify the key to use as the `short` message in `gelf` format. | _none_ |
34+
| `gelf_full_message_key` | Specify the key to use for the `full` message in `gelf` format. | _none_ |
35+
| `gelf_level_key` | Specify the key to use for the `level` in `gelf` format. | _none_ |
36+
| `body_key` | Specify the key to use as the body of the request (must prefix with `$`). The key must contain either a binary or raw string, and the content type can be specified using `headers_key`, which must be passed whenever `body_key` is present. When this option is present, each `msgpack` record will create a separate request. | _none_ |
37+
| `headers_key` | Specify the key to use as the headers of the request (must prefix with `$`). The key must contain a map, which will have the contents merged on the request headers. This can be used for many purposes, such as specifying the content type of the data contained in `body_key`.| _none_ |
38+
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` |
3739

3840
### TLS / SSL
3941

4042
The HTTP output plugin supports TLS/SSL.
4143
For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md).
4244

43-
## Getting Started
45+
## Get started
4446

45-
In order to insert records into an HTTP server, you can run the plugin from the command line or through the configuration file:
47+
To insert records into an HTTP server, you can run the plugin from the command line or through the configuration file.
4648

47-
### Command Line
49+
### Command line
4850

49-
The **http** plugin, can read the parameters from the command line in two ways, through the **-p** argument (property) or setting them directly through the service URI. The URI format is the following:
51+
The HTTP plugin can read the parameters from the command line through the `-p` argument (property) or setting them directly through the service URI. The URI format is the following:
5052

5153
```text
5254
http://host:port/something
@@ -58,7 +60,7 @@ Using the format specified, you could start Fluent Bit through:
5860
fluent-bit -i cpu -t cpu -o http://192.168.2.3:80/something -m '*'
5961
```
6062

61-
### Configuration File
63+
### Configuration file
6264

6365
In your main configuration file append the following:
6466

@@ -97,18 +99,18 @@ pipeline:
9799
{% endtab %}
98100
{% endtabs %}
99101

100-
By default, the URI becomes tag of the message, the original tag is ignored. To retain the tag, multiple configuration sections have to be made based and flush to different URIs.
102+
By default, the URI becomes tag of the message, the original tag is ignored. To retain the tag, multiple configuration sections have to be made based and flush to more than one URI.
101103

102-
Another approach we also support is the sending the original message tag in a configurable header. It's up to the receiver to do what it wants with that header field: parse it and use it as the tag for example.
104+
Another supported approach is the sending the original message tag in a configurable header. It's up to the receiver to do what it wants with that header field. For example, parse it and use it as the tag for example.
103105

104-
To configure this behaviour, add this config:
106+
To configure this behaviour, add this configuration:
105107

106108
{% tabs %}
107109
{% tab title="fluent-bit.yaml" %}
108110

109111
```yaml
110112
pipeline:
111-
113+
112114
outputs:
113115
- name: http
114116
match: '*'
@@ -155,16 +157,18 @@ Provided you are using Fluentd as data receiver, you can combine `in_http` and `
155157
</match>
156158
```
157159

158-
Notice how we override the tag, which is from URI path, with our custom header
160+
Fluent Bit overrides the tag, which is from URI path, with a custom header.
161+
162+
#### Add a header
159163

160-
#### Example : Add a header
164+
This example adds a header.
161165

162166
{% tabs %}
163167
{% tab title="fluent-bit.yaml" %}
164168

165169
```yaml
166170
pipeline:
167-
171+
168172
outputs:
169173
- name: http
170174
match: '*'
@@ -193,16 +197,16 @@ pipeline:
193197
{% endtab %}
194198
{% endtabs %}
195199

196-
#### Example : Sumo Logic HTTP Collector
200+
#### Sumo Logic HTTP collector
197201

198-
Suggested configuration for Sumo Logic using `json_lines` with `iso8601` timestamps. The `PrivateKey` is specific to a configured HTTP collector.
202+
The following is a suggested configuration for Sumo Logic using `json_lines` with `iso8601` timestamps. The `PrivateKey` is specific to a configured HTTP collector.
199203

200204
{% tabs %}
201205
{% tab title="fluent-bit.yaml" %}
202206

203207
```yaml
204208
pipeline:
205-
209+
206210
outputs:
207211
- name: http
208212
match: '*'
@@ -239,4 +243,4 @@ _sourcecategory="my_fluent_bit"
239243
| json "cpu_p" as cpu
240244
| timeslice 1m
241245
| max(cpu) as cpu group by _timeslice
242-
```
246+
```

‎vale-styles/FluentBit/Headings.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ exceptions:
5151
- Grafana
5252
- Graylog Extended Log Format
5353
- gRPC
54+
- HTTP
5455
- I
5556
- InfluxDB
5657
- Kafka
@@ -80,6 +81,7 @@ exceptions:
8081
- SSL
8182
- StatsD
8283
- Studio
84+
- Sumo Logic
8385
- Tanzu
8486
- Telemetry Pipeline
8587
- Terraform

0 commit comments

Comments
 (0)