Skip to content

Commit 8c64344

Browse files
committed
Pipeline: output: elasticsearch: style
Signed-off-by: Lynette Miles <[email protected]>
1 parent 5610226 commit 8c64344

File tree

3 files changed

+31
-51
lines changed

3 files changed

+31
-51
lines changed

pipeline/outputs/elasticsearch.md

Lines changed: 29 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,11 @@ description: Send logs to Elasticsearch (including Amazon OpenSearch Service)
44

55
# Elasticsearch
66

7-
The **es** output plugin lets you ingest your records into an
8-
[Elasticsearch](http://www.elastic.co) database. To use this plugin, you must have an
9-
operational Elasticsearch service running in your environment.
7+
The _Elasticsearch_ (`es`) output plugin lets you ingest your records into an [Elasticsearch](http://www.elastic.co) database. To use this plugin, you must have an operational Elasticsearch service running in your environment.
108

11-
## Configuration Parameters
9+
## Configuration parameters
10+
11+
This plugin has the following configuration parameters:
1212

1313
| Key | Description | Default |
1414
| :--- | :--- | :--- |
@@ -56,7 +56,7 @@ be compared to the `database` and `table` concepts.
5656

5757
### TLS / SSL
5858

59-
The Elasticsearch output plugin supports TLS/SSL.
59+
The Elasticsearch output plugin supports TLS/SSL.
6060
For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md).
6161

6262
### `write_operation`
@@ -78,12 +78,11 @@ The `write_operation` can be any of:
7878

7979
## Get started
8080

81-
To insert records into an Elasticsearch service, you run the plugin from the
82-
command line or through the configuration file:
81+
To insert records into an Elasticsearch service, run the plugin from the command line or through the configuration file.
8382

84-
### Command Line
83+
### Command line
8584

86-
The **es** plugin can read the parameters from the command line in two ways:
85+
The `es` plugin can read the parameters from the command line in the following ways:
8786

8887
- Through the `-p` argument (property).
8988
- Setting them directly through the service URI.
@@ -108,9 +107,9 @@ fluent-bit -i cpu -t cpu -o es -p Host=192.168.2.3 -p Port=9200 \
108107
-p Index=my_index -p Type=my_type -o stdout -m '*'
109108
```
110109

111-
### Configuration File
110+
### Configuration file
112111

113-
In your main configuration file append the following `Input` and `Output` sections.
112+
In your main configuration file append the following sections.
114113

115114
{% tabs %}
116115
{% tab title="fluent-bit.yaml" %}
@@ -120,7 +119,7 @@ pipeline:
120119
inputs:
121120
- name: cpu
122121
tag: cpu
123-
122+
124123
outputs:
125124
- name: es
126125
match: '*'
@@ -151,7 +150,7 @@ pipeline:
151150
{% endtab %}
152151
{% endtabs %}
153152

154-
## About Elasticsearch field names
153+
## Elasticsearch field names
155154

156155
Some input plugins can generate messages where the field names contains dots. For
157156
Elasticsearch 2.0, this isn't allowed. The current **es** plugin replaces
@@ -173,9 +172,7 @@ Connect to Amazon OpenSearch or Elastic Cloud with the ElasticSearch plugin.
173172

174173
### Amazon OpenSearch Service
175174

176-
The Amazon OpenSearch Service adds an extra security layer where HTTP requests must
177-
be signed with AWS Sigv4. Fluent Bit v1.5 introduced full support for Amazon
178-
OpenSearch Service with IAM Authentication.
175+
The Amazon OpenSearch Service adds an extra security layer where HTTP requests must be signed with AWS Sigv4. Fluent Bit v1.5 introduced full support for Amazon OpenSearch Service with IAM Authentication.
179176

180177
See [details](https://github.com/fluent/fluent-bit-docs/tree/43c4fe134611da471e706b0edb2f9acd7cdfdbc3/administration/aws-credentials.md) on how AWS credentials are fetched.
181178

@@ -186,7 +183,7 @@ Example configuration:
186183

187184
```yaml
188185
pipeline:
189-
186+
190187
outputs:
191188
- name: es
192189
match: '*'
@@ -222,21 +219,15 @@ Be aware that the `Port` is set to `443`, `tls` is enabled, and `AWS_Region` is
222219

223220
### Use Fluent Bit with Elastic Cloud
224221

225-
Fluent Bit supports connecting to
226-
[Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html)
227-
by providing the `cloud_id` and the `cloud_auth` settings. `cloud_auth` uses the
228-
`elastic` user and password provided when the cluster was created. For details refer
229-
to the
230-
[Cloud ID usage page](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html).
231-
222+
Fluent Bit supports connecting to [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) by providing the `cloud_id` and the `cloud_auth` settings. `cloud_auth` uses the `elastic` user and password provided when the cluster was created. For details refer to the [Cloud ID usage page](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html).
232223
Example configuration:
233224

234225
{% tabs %}
235226
{% tab title="fluent-bit.yaml" %}
236227

237228
```yaml
238229
pipeline:
239-
230+
240231
outputs:
241232
- name: es
242233
include_tag_key: true
@@ -281,9 +272,7 @@ Use the following information to help resolve errors using the ElasticSearch plu
281272

282273
### Using multiple types in a single index
283274

284-
Elasticsearch 6.0 can't create multiple types in a single index. An error message
285-
like the following indicates you need to update your configuration to use a single
286-
type on each index.
275+
Elasticsearch 6.0 can't create multiple types in a single index. An error message like the following indicates you need to update your configuration to use a single type on each index.
287276

288277
```text
289278
Rejecting mapping update to [products] as the final mapping would have more than 1 type:
@@ -296,7 +285,7 @@ This means that you can't set up your configuration like the following:.
296285

297286
```yaml
298287
pipeline:
299-
288+
300289
outputs:
301290
- name: es
302291
match: 'foo.*'
@@ -333,23 +322,18 @@ For details, read [the official blog post on that issue](https://www.elastic.co/
333322

334323
### Mapping type names can't start with underscores (`_`)
335324

336-
Fluent Bit v1.5 changed the default mapping type from `flb_type` to `_doc`, matching
337-
the recommendation from Elasticsearch for version 6.2 and greater
338-
([see commit with
339-
rationale](https://github.com/fluent/fluent-bit/commit/04ed3d8104ca8a2f491453777ae6e38e5377817e#diff-c9ae115d3acaceac5efb949edbb21196)).
325+
Fluent Bit v1.5 changed the default mapping type from `flb_type` to `_doc`, matching the recommendation from Elasticsearch for version 6.2 and greater ([see commit with rationale](https://github.com/fluent/fluent-bit/commit/04ed3d8104ca8a2f491453777ae6e38e5377817e#diff-c9ae115d3acaceac5efb949edbb21196)).
340326

341-
This doesn't work in Elasticsearch versions 5.6 through 6.1
342-
([discussion and fix](https://discuss.elastic.co/t/cant-use-doc-as-type-despite-it-being-declared-the-preferred-method/113837/9)).
327+
This doesn't work in Elasticsearch versions 5.6 through 6.1 ([discussion and fix](https://discuss.elastic.co/t/cant-use-doc-as-type-despite-it-being-declared-the-preferred-method/113837/9)).
343328

344-
Ensure you set an explicit map such as `doc` or `flb_type` in the configuration,
345-
as seen on the last line:
329+
Ensure you set an explicit map such as `doc` or `flb_type` in the configuration, as seen on the last line:
346330

347331
{% tabs %}
348332
{% tab title="fluent-bit.yaml" %}
349333

350334
```yaml
351335
pipeline:
352-
336+
353337
outputs:
354338
- name: es
355339
match: '*'
@@ -383,24 +367,20 @@ pipeline:
383367

384368
### Validation failures
385369

386-
In Fluent Bit v1.8.2 and greater, Fluent Bit started using `create` method (instead
387-
of `index`) for data submission. This makes Fluent Bit compatible with `Datastream`,
388-
introduced in Elasticsearch 7.9. You might see errors like:
370+
In Fluent Bit v1.8.2 and greater, Fluent Bit started using `create` method (instead of `index`) for data submission. This makes Fluent Bit compatible with `Datastream`, introduced in Elasticsearch 7.9. You might see errors like:
389371

390372
```text
391373
Validation Failed: 1: an id must be provided if version type or value are set
392374
```
393375

394-
If you see `action_request_validation_exception` errors on your pipeline with
395-
Fluent Bit versions greater than v1.8.2, correct them by turning on `Generate_ID`
396-
as follows:
376+
If you see `action_request_validation_exception` errors on your pipeline with Fluent Bit versions greater than v1.8.2, correct them by turning on `Generate_ID` as follows:
397377

398378
{% tabs %}
399379
{% tab title="fluent-bit.yaml" %}
400380

401381
```yaml
402382
pipeline:
403-
383+
404384
outputs:
405385
- name: es
406386
match: '*'
@@ -424,21 +404,20 @@ pipeline:
424404

425405
### `Logstash_Prefix_Key`
426406

427-
The following snippet demonstrates using the namespace name as extracted by the
428-
`kubernetes` filter as `logstash` prefix:
407+
The following snippet demonstrates using the namespace name as extracted by the `kubernetes` filter as `logstash` prefix:
429408

430409
{% tabs %}
431410
{% tab title="fluent-bit.yaml" %}
432411

433412
```yaml
434413
pipeline:
435-
414+
436415
outputs:
437416
- name: es
438417
match: '*'
439418
# ...
440419
logstash_prefix: logstash
441-
logstash_prefix_key: $kubernetes['namespace_name']
420+
logstash_prefix_key: $kubernetes['namespace_name']
442421
# ...
443422
```
444423

@@ -458,5 +437,4 @@ pipeline:
458437
{% endtab %}
459438
{% endtabs %}
460439

461-
For records that don't have the field `kubernetes.namespace_name`, the default prefix
462-
`logstash` will be used.
440+
For records that don't have the field `kubernetes.namespace_name`, the default prefix `logstash` will be used.

vale-styles/FluentBit/Headings.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ exceptions:
1212
- Amazon ECR Public Gallery
1313
- Amazon Kinesis Firehose
1414
- Amazon Kinesis Streams
15+
- Amazon OpenSearch Service
1516
- API
1617
- APIs
1718
- Azure

vale-styles/FluentBit/Spelling-exceptions.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,7 @@ Queryable
148148
Raspbian
149149
rdkafka
150150
Redpanda
151+
reindexed
151152
rollup
152153
Rollup
153154
rollups

0 commit comments

Comments
 (0)