Skip to content

Commit 71c885f

Browse files
author
Pat
authored
Elastic cloud updates (#751)
* windows: add 1.9 updates Signed-off-by: Patrick Stephens <[email protected]> * windows: doc tweaks Signed-off-by: Patrick Stephens <[email protected]> * elasticsearch: add some clarification Signed-off-by: Patrick Stephens <[email protected]> * elasticsearch: add Suppress_Type_Name info Signed-off-by: Patrick Stephens <[email protected]>
1 parent 0e7a54f commit 71c885f

File tree

1 file changed

+15
-2
lines changed

1 file changed

+15
-2
lines changed

pipeline/outputs/elasticsearch.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@ The **es** output plugin, allows to ingest your records into an [Elasticsearch](
2020
| AWS\_STS\_Endpoint | Specify the custom sts endpoint to be used with STS API for Amazon OpenSearch Service | |
2121
| AWS\_Role\_ARN | AWS IAM Role to assume to put records to your Amazon cluster | |
2222
| AWS\_External\_ID | External ID for the AWS IAM Role specified with `aws_role_arn` | |
23-
| Cloud\_ID | If you are using Elastic's Elasticsearch Service you can specify the cloud\_id of the cluster running | |
23+
| Cloud\_ID | If you are using Elastic's Elasticsearch Service you can specify the cloud\_id of the cluster running. The Cloud ID string has the format `<deployment_name>:<base64_info>`. Once decoded, the `base64_info` string has the format `<deployment_region>$<elasticsearch_hostname>$<kibana_hostname>`.
24+
| |
2425
| Cloud\_Auth | Specify the credentials to use to connect to Elastic's Elasticsearch Service running on Elastic Cloud | |
2526
| HTTP\_User | Optional username credential for Elastic X-Pack access | |
2627
| HTTP\_Passwd | Password for user defined in HTTP\_User | |
@@ -193,6 +194,7 @@ Notice that the `Port` is set to `443`, `tls` is enabled, and `AWS_Region` is se
193194
### Fluent Bit + Elastic Cloud
194195

195196
Fluent Bit supports connecting to [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) providing just the `cloud_id` and the `cloud_auth` settings.
197+
`cloud_auth` uses the `elastic` user and password provided when the cluster was created, for details refer to the [Cloud ID usage page](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html).
196198

197199
Example configuration:
198200

@@ -203,13 +205,15 @@ Example configuration:
203205
Tag_Key tags
204206
tls On
205207
tls.verify Off
208+
Suppress_Type_Name On
206209
cloud_id elastic-obs-deployment:ZXVybxxxxxxxxxxxg==
207210
cloud_auth elastic:2vxxxxxxxxYV
208211
```
209212

210213
### Validation Failed: 1: an id must be provided if version type or value are set
211214

212-
Since v1.8.2, Fluent Bit started using `create` method (instead of `index`) for data submission. This makes Flunt Bit compatible with Datastream introduced in Elasticsearch 7.9.
215+
Since v1.8.2, Fluent Bit started using `create` method (instead of `index`) for data submission.
216+
This makes Fluent Bit compatible with Datastream introduced in Elasticsearch 7.9.
213217

214218
If you see `action_request_validation_exception` errors on your pipeline with Fluent Bit >= v1.8.2, you can fix it up by turning on `Generate_ID` as follows:
215219

@@ -220,3 +224,12 @@ If you see `action_request_validation_exception` errors on your pipeline with Fl
220224
Host 192.168.12.1
221225
Generate_ID on
222226
```
227+
228+
### Action/metadata contains an unknown parameter type
229+
230+
Elastic Cloud is now on version 8 so the type option must be removed by setting `Suppress_Type_Name On` as indicated above.
231+
232+
Without this you will see errors like:
233+
```
234+
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}
235+
```

0 commit comments

Comments
 (0)