Skip to content

Commit 1bb5cf1

Browse files
HarshCasperHarshCasper
committed
revamp es
1 parent 4701b2c commit 1bb5cf1

File tree

1 file changed

+110
-75
lines changed
  • src/content/docs/aws/services

1 file changed

+110
-75
lines changed

‎src/content/docs/aws/services/es.md‎

Lines changed: 110 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,30 @@
11
---
22
title: "Elasticsearch Service"
3-
linkTitle: "Elasticsearch Service"
4-
description: >
5-
Get started with Amazon Elasticsearch Service (ES) on LocalStack
3+
description: Get started with Amazon Elasticsearch Service (ES) on LocalStack
64
tags: ["Free"]
75
---
86

7+
## Introduction
8+
99
The Elasticsearch Service in LocalStack lets you create one or more single-node Elasticsearch/OpenSearch cluster that behaves like the [Amazon Elasticsearch Service](https://aws.amazon.com/opensearch-service/the-elk-stack/what-is-elasticsearch/).
1010
This service is, like its AWS counterpart, heavily linked with the [OpenSearch Service](../opensearch).
1111
Any cluster created with the Elasticsearch Service will show up in the OpenSearch Service and vice versa.
1212

1313
## Creating an Elasticsearch cluster
1414

15-
You can go ahead and use [awslocal]({{< ref "aws-cli.md#localstack-aws-cli-awslocal" >}}) to create a new elasticsearch domain via the `aws es create-elasticsearch-domain` command.
15+
You can go ahead and use [`awslocal`](https://github.com/localstack/awscli-local) to create a new elasticsearch domain via the `aws es create-elasticsearch-domain` command.
1616

17-
{{< callout >}}
17+
:::note
1818
Unless you use the Elasticsearch default version, the first time you create a cluster with a specific version, the Elasticsearch binary is downloaded, which may take a while to download.
19-
{{< /callout >}}
19+
:::
20+
21+
```bash
22+
awslocal es create-elasticsearch-domain --domain-name my-domain
23+
```
2024

21-
{{< command >}}
22-
$ awslocal es create-elasticsearch-domain --domain-name my-domain
25+
The following output would be retrieved:
26+
27+
```json
2328
{
2429
"DomainStatus": {
2530
"DomainId": "000000000000/my-domain",
@@ -49,11 +54,11 @@ $ awslocal es create-elasticsearch-domain --domain-name my-domain
4954
}
5055
}
5156
}
52-
{{< / command >}}
57+
```
5358

5459
In the LocalStack log you will see something like the following, where you can see the cluster starting up in the background.
5560

56-
```plaintext
61+
```bash
5762
2021-11-08T16:29:28:INFO:localstack.services.es.cluster: starting elasticsearch: /opt/code/localstack/localstack/localstack/infra/elasticsearch/bin/elasticsearch -E http.port=57705 -E http.publish_port=57705 -E transport.port=0 -E network.host=127.0.0.1 -E http.compression=false -E path.data="/var/lib/localstack/lib//elasticsearch/arn:aws:es:us-east-1:000000000000:domain/my-domain/data" -E path.repo="/var/lib/localstack/lib//elasticsearch/arn:aws:es:us-east-1:000000000000:domain/my-domain/backup" -E xpack.ml.enabled=false with env {'ES_JAVA_OPTS': '-Xms200m -Xmx600m', 'ES_TMPDIR': '/var/lib/localstack/lib//elasticsearch/arn:aws:es:us-east-1:000000000000:domain/my-domain/tmp'}
5863
2021-11-08T16:29:28:INFO:localstack.services.es.cluster: registering an endpoint proxy for http://my-domain.us-east-1.es.localhost.localstack.cloud:4566 => http://127.0.0.1:57705
5964
2021-11-08T16:29:30:INFO:localstack.services.es.cluster: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
@@ -68,10 +73,16 @@ In the LocalStack log you will see something like the following, where you can s
6873

6974
and after some time, you should see that the `Processing` state of the domain is set to `false`:
7075

71-
{{< command >}}
72-
$ awslocal es describe-elasticsearch-domain --domain-name my-domain | jq ".DomainStatus.Processing"
76+
```bash
77+
awslocal es describe-elasticsearch-domain --domain-name my-domain | jq ".DomainStatus.Processing"
78+
```
79+
80+
The following output would be retrieved:
81+
82+
```bash
7383
false
74-
{{< / command >}}
84+
```
85+
7586

7687
## Interact with the cluster
7788

@@ -80,8 +91,13 @@ in this case `http://my-domain.us-east-1.es.localhost.localstack.cloud:4566`.
8091

8192
For example:
8293

83-
{{< command >}}
84-
$ curl http://my-domain.us-east-1.es.localhost.localstack.cloud:4566
94+
```bash
95+
curl http://my-domain.us-east-1.es.localhost.localstack.cloud:4566
96+
```
97+
98+
The following output would be retrieved:
99+
100+
```json
85101
{
86102
"name" : "localstack",
87103
"cluster_name" : "elasticsearch",
@@ -99,12 +115,17 @@ $ curl http://my-domain.us-east-1.es.localhost.localstack.cloud:4566
99115
},
100116
"tagline" : "You Know, for Search"
101117
}
102-
{{< / command >}}
118+
```
103119

104120
Or the health endpoint:
105121

106-
{{< command >}}
107-
$ curl -s http://my-domain.us-east-1.es.localhost.localstack.cloud:4566/_cluster/health | jq .
122+
```bash
123+
curl -s http://my-domain.us-east-1.es.localhost.localstack.cloud:4566/_cluster/health | jq .
124+
```
125+
126+
The following output would be retrieved:
127+
128+
```json
108129
{
109130
"cluster_name": "elasticsearch",
110131
"status": "green",
@@ -122,7 +143,7 @@ $ curl -s http://my-domain.us-east-1.es.localhost.localstack.cloud:4566/_cluster
122143
"task_max_waiting_in_queue_millis": 0,
123144
"active_shards_percent_as_number": 100
124145
}
125-
{{< / command >}}
146+
```
126147

127148
## Advanced topics
128149

@@ -134,7 +155,7 @@ There are three configurable strategies that govern how domain endpoints are cre
134155
| - | - | - |
135156
| `domain` | `<domain-name>.<region>.es.localhost.localstack.cloud:4566` | This is the default strategy that uses the `localhost.localstack.cloud` domain to route to your localhost |
136157
| `path` | `localhost:4566/es/<region>/<domain-name>` | An alternative that can be useful if you cannot resolve LocalStack's localhost domain |
137-
| `port` | `localhost:<port-from-range>` | Exposes the cluster(s) directly with ports from the [external service port range]({{< ref "external-ports" >}})|
158+
| `port` | `localhost:<port-from-range>` | Exposes the cluster(s) directly with ports from the [external service port range]()|
138159
| `off` | | *Deprecated*. This value now reverts to the `port` setting, using a port from the given range instead of `4571` |
139160

140161
Regardless of the service from which the clusters were created, the domain of the cluster always corresponds to the engine type (OpenSearch or Elasticsearch) of the cluster.
@@ -146,17 +167,17 @@ LocalStack allows you to set arbitrary custom endpoints for your clusters in the
146167
This can be used to overwrite the behavior of the endpoint strategies described above.
147168
You can also choose custom domains, however it is important to add the edge port (`80`/`443` or by default `4566`).
148169

149-
{{< command >}}
150-
$ awslocal es create-elasticsearch-domain --domain-name my-domain \
170+
```bash
171+
awslocal es create-elasticsearch-domain --domain-name my-domain \
151172
--elasticsearch-version 7.10 \
152173
--domain-endpoint-options '{ "CustomEndpoint": "http://localhost:4566/my-custom-endpoint", "CustomEndpointEnabled": true }'
153-
{{< / command >}}
174+
```
154175

155176
Once the domain processing is complete, you can access the cluster:
156177

157-
{{< command >}}
158-
$ curl http://localhost:4566/my-custom-endpoint/_cluster/health
159-
{{< / command >}}
178+
```bash
179+
curl http://localhost:4566/my-custom-endpoint/_cluster/health
180+
```
160181

161182
### Re-using a single cluster instance
162183

@@ -244,64 +265,78 @@ volumes:
244265
```
245266
246267
1. Run docker compose:
247-
{{< command >}}
248-
$ docker-compose up -d
249-
{{< /command >}}
268+
```bash
269+
docker-compose up -d
270+
```
250271

251272
2. Create the Elasticsearch domain:
252-
{{< command >}}
253-
$ awslocal es create-elasticsearch-domain \
254-
--domain-name mylogs-2 \
255-
--elasticsearch-version 7.10 \
256-
--elasticsearch-cluster-config '{ "InstanceType": "m3.xlarge.elasticsearch", "InstanceCount": 4, "DedicatedMasterEnabled": true, "ZoneAwarenessEnabled": true, "DedicatedMasterType": "m3.xlarge.elasticsearch", "DedicatedMasterCount": 3}'
257-
{
258-
"DomainStatus": {
259-
"DomainId": "000000000000/mylogs-2",
260-
"DomainName": "mylogs-2",
261-
"ARN": "arn:aws:es:us-east-1:000000000000:domain/mylogs-2",
262-
"Created": true,
263-
"Deleted": false,
264-
"Endpoint": "mylogs-2.us-east-1.es.localhost.localstack.cloud:4566",
265-
"Processing": true,
266-
"ElasticsearchVersion": "7.10",
267-
"ElasticsearchClusterConfig": {
268-
"InstanceType": "m3.xlarge.elasticsearch",
269-
"InstanceCount": 4,
270-
"DedicatedMasterEnabled": true,
271-
"ZoneAwarenessEnabled": true,
272-
"DedicatedMasterType": "m3.xlarge.elasticsearch",
273-
"DedicatedMasterCount": 3
274-
},
275-
"EBSOptions": {
276-
"EBSEnabled": true,
277-
"VolumeType": "gp2",
278-
"VolumeSize": 10,
279-
"Iops": 0
280-
},
281-
"CognitoOptions": {
282-
"Enabled": false
273+
```bash
274+
awslocal es create-elasticsearch-domain \
275+
--domain-name mylogs-2 \
276+
--elasticsearch-version 7.10 \
277+
--elasticsearch-cluster-config '{ "InstanceType": "m3.xlarge.elasticsearch", "InstanceCount": 4, "DedicatedMasterEnabled": true, "ZoneAwarenessEnabled": true, "DedicatedMasterType": "m3.xlarge.elasticsearch", "DedicatedMasterCount": 3}'
278+
```
279+
280+
The following output would be retrieved:
281+
282+
```json
283+
{
284+
"DomainStatus": {
285+
"DomainId": "000000000000/mylogs-2",
286+
"DomainName": "mylogs-2",
287+
"ARN": "arn:aws:es:us-east-1:000000000000:domain/mylogs-2",
288+
"Created": true,
289+
"Deleted": false,
290+
"Endpoint": "mylogs-2.us-east-1.es.localhost.localstack.cloud:4566",
291+
"Processing": true,
292+
"ElasticsearchVersion": "7.10",
293+
"ElasticsearchClusterConfig": {
294+
"InstanceType": "m3.xlarge.elasticsearch",
295+
"InstanceCount": 4,
296+
"DedicatedMasterEnabled": true,
297+
"ZoneAwarenessEnabled": true,
298+
"DedicatedMasterType": "m3.xlarge.elasticsearch",
299+
"DedicatedMasterCount": 3
300+
},
301+
"EBSOptions": {
302+
"EBSEnabled": true,
303+
"VolumeType": "gp2",
304+
"VolumeSize": 10,
305+
"Iops": 0
306+
},
307+
"CognitoOptions": {
308+
"Enabled": false
309+
}
283310
}
284311
}
285-
}
286-
{{< /command >}}
312+
```
287313

288-
3. If the `Processing` status is true, it means that the cluster is not yet healthy.
289-
You can run `describe-elasticsearch-domain` to receive the status:
290-
{{< command >}}
291-
$ awslocal es describe-elasticsearch-domain --domain-name mylogs-2
292-
{{< /command >}}
314+
3. If the `Processing` status is true, it means that the cluster is not yet healthy. You can run `describe-elasticsearch-domain` to receive the status:
315+
```bash
316+
awslocal es describe-elasticsearch-domain --domain-name mylogs-2
317+
```
293318

294319
4. Check the cluster health endpoint and create indices:
295-
{{< command >}}
296-
$ curl mylogs-2.us-east-1.es.localhost.localstack.cloud:4566/_cluster/health
297-
{"cluster_name":"es-docker-cluster","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}[~]
298-
{{< /command >}}
320+
```bash
321+
curl mylogs-2.us-east-1.es.localhost.localstack.cloud:4566/_cluster/health
322+
```
323+
324+
The following output would be retrieved:
325+
326+
```bash
327+
{"cluster_name":"es-docker-cluster","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}[~]
328+
```
299329

300330
5. Create an example index:
301-
{{< command >}}
302-
$ curl -X PUT mylogs-2.us-east-1.es.localhost.localstack.cloud:4566/my-index
303-
{"acknowledged":true,"shards_acknowledged":true,"index":"my-index"}
304-
{{< /command >}}
331+
```bash
332+
curl -X PUT mylogs-2.us-east-1.es.localhost.localstack.cloud:4566/my-index
333+
```
334+
335+
The following output would be retrieved:
336+
337+
```bash
338+
{"acknowledged":true,"shards_acknowledged":true,"index":"my-index"}
339+
```
305340

306341
## Differences to AWS
307342

0 commit comments

Comments
 (0)