You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The **es** output plugin lets you ingest your records into an
8
-
[Elasticsearch](http://www.elastic.co) database. To use this plugin, you must have an
9
-
operational Elasticsearch service running in your environment.
7
+
The _Elasticsearch_ (`es`) output plugin lets you ingest your records into an [Elasticsearch](http://www.elastic.co) database. To use this plugin, you must have an operational Elasticsearch service running in your environment.
10
8
11
-
## Configuration Parameters
9
+
## Configuration parameters
10
+
11
+
This plugin has the following configuration parameters:
12
12
13
13
| Key | Description | Default |
14
14
| :--- | :--- | :--- |
@@ -56,7 +56,7 @@ be compared to the `database` and `table` concepts.
56
56
57
57
### TLS / SSL
58
58
59
-
The Elasticsearch output plugin supports TLS/SSL.
59
+
The Elasticsearch output plugin supports TLS/SSL.
60
60
For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md).
61
61
62
62
### `write_operation`
@@ -78,12 +78,11 @@ The `write_operation` can be any of:
78
78
79
79
## Get started
80
80
81
-
To insert records into an Elasticsearch service, you run the plugin from the
82
-
command line or through the configuration file:
81
+
To insert records into an Elasticsearch service, run the plugin from the command line or through the configuration file.
83
82
84
-
### Command Line
83
+
### Command line
85
84
86
-
The **es** plugin can read the parameters from the command line in two ways:
85
+
The `es` plugin can read the parameters from the command line in the following ways:
87
86
88
87
- Through the `-p` argument (property).
89
88
- Setting them directly through the service URI.
@@ -108,9 +107,9 @@ fluent-bit -i cpu -t cpu -o es -p Host=192.168.2.3 -p Port=9200 \
In your main configuration file append the following `Input` and `Output`sections.
112
+
In your main configuration file append the following sections.
114
113
115
114
{% tabs %}
116
115
{% tab title="fluent-bit.yaml" %}
@@ -120,7 +119,7 @@ pipeline:
120
119
inputs:
121
120
- name: cpu
122
121
tag: cpu
123
-
122
+
124
123
outputs:
125
124
- name: es
126
125
match: '*'
@@ -151,7 +150,7 @@ pipeline:
151
150
{% endtab %}
152
151
{% endtabs %}
153
152
154
-
## About Elasticsearch field names
153
+
## Elasticsearch field names
155
154
156
155
Some input plugins can generate messages where the field names contains dots. For
157
156
Elasticsearch 2.0, this isn't allowed. The current **es** plugin replaces
@@ -173,9 +172,7 @@ Connect to Amazon OpenSearch or Elastic Cloud with the ElasticSearch plugin.
173
172
174
173
### Amazon OpenSearch Service
175
174
176
-
The Amazon OpenSearch Service adds an extra security layer where HTTP requests must
177
-
be signed with AWS Sigv4. Fluent Bit v1.5 introduced full support for Amazon
178
-
OpenSearch Service with IAM Authentication.
175
+
The Amazon OpenSearch Service adds an extra security layer where HTTP requests must be signed with AWS Sigv4. Fluent Bit v1.5 introduced full support for Amazon OpenSearch Service with IAM Authentication.
179
176
180
177
See [details](https://github.com/fluent/fluent-bit-docs/tree/43c4fe134611da471e706b0edb2f9acd7cdfdbc3/administration/aws-credentials.md) on how AWS credentials are fetched.
181
178
@@ -186,7 +183,7 @@ Example configuration:
186
183
187
184
```yaml
188
185
pipeline:
189
-
186
+
190
187
outputs:
191
188
- name: es
192
189
match: '*'
@@ -222,21 +219,15 @@ Be aware that the `Port` is set to `443`, `tls` is enabled, and `AWS_Region` is
by providing the `cloud_id` and the `cloud_auth` settings. `cloud_auth` uses the
228
-
`elastic` user and password provided when the cluster was created. For details refer
229
-
to the
230
-
[Cloud ID usage page](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html).
231
-
222
+
Fluent Bit supports connecting to [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) by providing the `cloud_id` and the `cloud_auth` settings. `cloud_auth` uses the `elastic` user and password provided when the cluster was created. For details refer to the [Cloud ID usage page](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html).
232
223
Example configuration:
233
224
234
225
{% tabs %}
235
226
{% tab title="fluent-bit.yaml" %}
236
227
237
228
```yaml
238
229
pipeline:
239
-
230
+
240
231
outputs:
241
232
- name: es
242
233
include_tag_key: true
@@ -281,9 +272,7 @@ Use the following information to help resolve errors using the ElasticSearch plu
281
272
282
273
### Using multiple types in a single index
283
274
284
-
Elasticsearch 6.0 can't create multiple types in a single index. An error message
285
-
like the following indicates you need to update your configuration to use a single
286
-
type on each index.
275
+
Elasticsearch 6.0 can't create multiple types in a single index. An error message like the following indicates you need to update your configuration to use a single type on each index.
287
276
288
277
```text
289
278
Rejecting mapping update to [products] as the final mapping would have more than 1 type:
@@ -296,7 +285,7 @@ This means that you can't set up your configuration like the following:.
296
285
297
286
```yaml
298
287
pipeline:
299
-
288
+
300
289
outputs:
301
290
- name: es
302
291
match: 'foo.*'
@@ -333,23 +322,18 @@ For details, read [the official blog post on that issue](https://www.elastic.co/
333
322
334
323
### Mapping type names can't start with underscores (`_`)
335
324
336
-
Fluent Bit v1.5 changed the default mapping type from `flb_type` to `_doc`, matching
337
-
the recommendation from Elasticsearch for version 6.2 and greater
Fluent Bit v1.5 changed the default mapping type from `flb_type` to `_doc`, matching the recommendation from Elasticsearch for version 6.2 and greater ([see commit with rationale](https://github.com/fluent/fluent-bit/commit/04ed3d8104ca8a2f491453777ae6e38e5377817e#diff-c9ae115d3acaceac5efb949edbb21196)).
340
326
341
-
This doesn't work in Elasticsearch versions 5.6 through 6.1
342
-
([discussion and fix](https://discuss.elastic.co/t/cant-use-doc-as-type-despite-it-being-declared-the-preferred-method/113837/9)).
327
+
This doesn't work in Elasticsearch versions 5.6 through 6.1 ([discussion and fix](https://discuss.elastic.co/t/cant-use-doc-as-type-despite-it-being-declared-the-preferred-method/113837/9)).
343
328
344
-
Ensure you set an explicit map such as `doc` or `flb_type` in the configuration,
345
-
as seen on the last line:
329
+
Ensure you set an explicit map such as `doc` or `flb_type` in the configuration, as seen on the last line:
346
330
347
331
{% tabs %}
348
332
{% tab title="fluent-bit.yaml" %}
349
333
350
334
```yaml
351
335
pipeline:
352
-
336
+
353
337
outputs:
354
338
- name: es
355
339
match: '*'
@@ -383,24 +367,20 @@ pipeline:
383
367
384
368
### Validation failures
385
369
386
-
In Fluent Bit v1.8.2 and greater, Fluent Bit started using `create` method (instead
387
-
of `index`) for data submission. This makes Fluent Bit compatible with `Datastream`,
388
-
introduced in Elasticsearch 7.9. You might see errors like:
370
+
In Fluent Bit v1.8.2 and greater, Fluent Bit started using `create` method (instead of `index`) for data submission. This makes Fluent Bit compatible with `Datastream`, introduced in Elasticsearch 7.9. You might see errors like:
389
371
390
372
```text
391
373
Validation Failed: 1: an id must be provided if version type or value are set
392
374
```
393
375
394
-
If you see `action_request_validation_exception` errors on your pipeline with
395
-
Fluent Bit versions greater than v1.8.2, correct them by turning on `Generate_ID`
396
-
as follows:
376
+
If you see `action_request_validation_exception` errors on your pipeline with Fluent Bit versions greater than v1.8.2, correct them by turning on `Generate_ID` as follows:
397
377
398
378
{% tabs %}
399
379
{% tab title="fluent-bit.yaml" %}
400
380
401
381
```yaml
402
382
pipeline:
403
-
383
+
404
384
outputs:
405
385
- name: es
406
386
match: '*'
@@ -424,21 +404,20 @@ pipeline:
424
404
425
405
### `Logstash_Prefix_Key`
426
406
427
-
The following snippet demonstrates using the namespace name as extracted by the
428
-
`kubernetes` filter as `logstash` prefix:
407
+
The following snippet demonstrates using the namespace name as extracted by the `kubernetes` filter as `logstash` prefix:
0 commit comments