Skip to content
This repository was archived by the owner on Aug 16, 2022. It is now read-only.

Commit c5c1c85

Browse files
authored
Merge pull request #71 from opendistro/1.0.0
Preparing for 1.0.0
2 parents b0b6854 + 050a787 commit c5c1c85

34 files changed

+814
-137
lines changed

check-links.sh

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Checks for broken link in the documentation.
2+
# Run `bundle exec jekyll serve` first.
3+
# Uses https://github.com/stevenvachon/broken-link-checker
4+
blc http://127.0.0.1:4000/for-elasticsearch-docs/ -ro

docs/alerting/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
layout: default
33
title: Alerting
4-
nav_order: 8
4+
nav_order: 30
55
has_children: true
66
---
77

docs/alerting/monitors.md

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,35 @@ Destination | A reusable location for an action, such as Amazon Chime, Slack, or
5353

5454
- Query definition gives you flexibility in terms of what you query for (using [the Elasticsearch query DSL](../../elasticsearch/full-text)) and how you evaluate the results of that query (Painless scripting).
5555

56+
You can even filter query results using `{% raw %}{{period_start}}{% endraw %}` and `{% raw %}{{period_end}}{% endraw %}`:
57+
58+
```json
59+
{
60+
"size": 0,
61+
"query": {
62+
"bool": {
63+
"filter": [{
64+
"range": {
65+
"timestamp": {
66+
"from": "{% raw %}{{period_end}}{% endraw %}||-1h",
67+
"to": "{% raw %}{{period_end}}{% endraw %}",
68+
"include_lower": true,
69+
"include_upper": true,
70+
"format": "epoch_millis",
71+
"boost": 1
72+
}
73+
}
74+
}],
75+
"adjust_pure_negative": true,
76+
"boost": 1
77+
}
78+
},
79+
"aggregations": {}
80+
}
81+
```
82+
83+
"Start" and "end" refer to the interval at which the monitor runs. See [Available variables](#available-variables).
84+
5685
1. To define a monitor visually, choose **Define using visual graph**. Then choose an aggregation (for example, `count()` or `average()`), a set of documents, and a timeframe. Visual definition works well for most monitors.
5786

5887
To use a query, choose **Define using extraction query**, add your query (using [the Elasticsearch query DSL](../../elasticsearch/full-text)), and test it using the **Run** button.
@@ -82,7 +111,7 @@ The line moves up and down as you increase and decrease the threshold. Once this
82111

83112
For **Trigger condition**, specify a Painless script that returns true or false. Painless is the default Elasticsearch scripting language and has a syntax similar to Groovy.
84113

85-
Trigger condition scripts revolve around the `ctx.results[0]` variable, which corresponds to the extraction query response. For example, your script might reference `ctx.results[0].hits.total` or `ctx.results[0].hits.hits[i]._source.error_code`.
114+
Trigger condition scripts revolve around the `ctx.results[0]` variable, which corresponds to the extraction query response. For example, your script might reference `ctx.results[0].hits.total.value` or `ctx.results[0].hits.hits[i]._source.error_code`.
86115

87116
A return value of true means the trigger condition has been met, and the trigger should execute its actions. Test your script using the **Run** button.
88117

@@ -98,7 +127,7 @@ These scripts are Painless, not Groovy, but calling them Groovy in Jekyll gets u
98127

99128
```groovy
100129
// Evaluates to true if the query returned any documents
101-
ctx.results[0].hits.total > 0
130+
ctx.results[0].hits.total.value > 0
102131
```
103132

104133
```groovy

docs/elasticsearch/index-data.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,15 @@ For situations in which new data arrives incrementally (for example, customer or
1717
A request to the index API looks like the following:
1818

1919
```json
20-
PUT elasticsearch_domain/<index>/_doc/<id>
20+
PUT cluster_endpoint/<index>/_doc/<id>
2121
{ "A JSON": "document" }
2222
```
2323

2424
A request to the `_bulk` API looks a little different, because you specify the index and ID in the bulk data:
2525

2626
```json
27-
POST elasticsearch_domain/_bulk
28-
{ "index": { "_index" : "<index>", "_type" : "_doc", "_id" : "<id>" } }
27+
POST cluster_endpoint/_bulk
28+
{ "index": { "_index" : "<index>", "_id" : "<id>" } }
2929
{ "A JSON": "document" }
3030

3131
```
@@ -46,21 +46,21 @@ The document is optional, because `delete` actions do not require a document. Th
4646
Elasticsearch features automatic index creation when you add a document to an index that doesn't already exist. It also features automatic ID generation if you don't specify an ID in the request. This simple example automatically creates the movies index, indexes the document, and assigns it a unique ID:
4747

4848
```json
49-
POST elasticsearch_domain/movies/_doc
49+
POST cluster_endpoint/movies/_doc
5050
{"title": "Spirited Away"}
5151
```
5252

5353
Automatic ID generation has a clear downside: because the indexing request didn't specify a document ID, you can't easily update the document at a later time. To specify an ID of 1, use the following request, and note the use of PUT instead of POST:
5454

5555
```json
56-
PUT elasticsearch_domain/movies/_doc/1
56+
PUT cluster_endpoint/movies/_doc/1
5757
{"title": "Spirited Away"}
5858
```
5959

60-
Indices default to five primary shards and one replica. If you want to specify non-default settings, create the index before adding documents:
60+
Indices default to one primary shard and one replica. If you want to specify non-default settings, create the index before adding documents:
6161

6262
```json
63-
PUT elasticsearch_domain/more-movies
63+
PUT cluster_endpoint/more-movies
6464
{"settings": {"number_of_shards": 6, "number_of_replicas": 2}}
6565
```
6666

@@ -73,4 +73,4 @@ Elasticsearch indices have the following naming restrictions:
7373
- Index names can't begin with `_` (underscore) or `-` (hyphen).
7474
- Index names can't contain spaces, commas, or the following characters:
7575

76-
`"`, `*`, `+`, `/`, `\`, `|`, `?`, `#`, `>`, or `<`
76+
`:`, `"`, `*`, `+`, `/`, `\`, `|`, `?`, `#`, `>`, or `<`

docs/elasticsearch/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
layout: default
33
title: Elasticsearch
4-
nav_order: 3
4+
nav_order: 10
55
has_children: true
66
has_toc: false
77
---

docs/elasticsearch/snapshot-restore.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ readonly | Whether the repository is read-only. Useful when migrating from one c
101101
If you're using the Docker installation, see [Run with custom plugins](../../install/docker/#run-with-custom-plugins). Your `Dockerfile` should look something like this:
102102

103103
```
104-
FROM amazon/opendistro-for-elasticsearch:0.9.0
104+
FROM amazon/opendistro-for-elasticsearch:1.0.0
105105

106106
ENV AWS_ACCESS_KEY_ID <access-key>
107107
ENV AWS_SECRET_ACCESS_KEY <secret-key>

docs/install/docker-security.md

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -16,17 +16,23 @@ Before deploying to a production environment, you should replace the demo securi
1616
version: '3'
1717
services:
1818
odfe-node1:
19-
image: amazon/opendistro-for-elasticsearch:0.9.0
19+
image: amazon/opendistro-for-elasticsearch:1.0.0
2020
container_name: odfe-node1
2121
environment:
2222
- cluster.name=odfe-cluster
23+
- node.name=odfe-node1
24+
- discovery.seed_hosts=odfe-node1,odfe-node2
25+
- cluster.initial_master_nodes=odfe-node1,odfe-node2
2326
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
2427
- "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
25-
- network.host=0.0.0.0 # required if not using the demo Security configuration
28+
- network.host=0.0.0.0
2629
ulimits:
2730
memlock:
2831
soft: -1
2932
hard: -1
33+
nofile:
34+
soft: 65536 # maximum number of open files for the Elasticsearch user, set to at least 65536 on modern systems
35+
hard: 65536
3036
volumes:
3137
- odfe-data1:/usr/share/elasticsearch/data
3238
- ./root-ca.pem:/usr/share/elasticsearch/config/root-ca.pem
@@ -36,24 +42,33 @@ services:
3642
- ./kirk-key.pem:/usr/share/elasticsearch/config/kirk-key.pem
3743
- ./custom-elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
3844
- ./internal_users.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/internal_users.yml
45+
- ./roles_mapping.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles_mapping.yml
46+
- ./tenants.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/tenants.yml
47+
- ./roles.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles.yml
48+
- ./action_groups.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/action_groups.yml
3949
ports:
4050
- 9200:9200
4151
- 9600:9600 # required for Performance Analyzer
4252
networks:
4353
- odfe-net
4454
odfe-node2:
45-
image: amazon/opendistro-for-elasticsearch:0.9.0
55+
image: amazon/opendistro-for-elasticsearch:1.0.0
4656
container_name: odfe-node2
4757
environment:
4858
- cluster.name=odfe-cluster
59+
- node.name=odfe-node2
60+
- discovery.seed_hosts=odfe-node1,odfe-node2
61+
- cluster.initial_master_nodes=odfe-node1,odfe-node2
4962
- bootstrap.memory_lock=true
5063
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
51-
- discovery.zen.ping.unicast.hosts=odfe-node1
5264
- network.host=0.0.0.0
5365
ulimits:
5466
memlock:
5567
soft: -1
5668
hard: -1
69+
nofile:
70+
soft: 65536
71+
hard: 65536
5772
volumes:
5873
- odfe-data2:/usr/share/elasticsearch/data
5974
- ./root-ca.pem:/usr/share/elasticsearch/config/root-ca.pem
@@ -63,10 +78,14 @@ services:
6378
- ./kirk-key.pem:/usr/share/elasticsearch/config/kirk-key.pem
6479
- ./custom-elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
6580
- ./internal_users.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/internal_users.yml
81+
- ./roles_mapping.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles_mapping.yml
82+
- ./tenants.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/tenants.yml
83+
- ./roles.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles.yml
84+
- ./action_groups.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/action_groups.yml
6685
networks:
6786
- odfe-net
6887
kibana:
69-
image: amazon/opendistro-for-elasticsearch-kibana:0.9.0
88+
image: amazon/opendistro-for-elasticsearch-kibana:1.0.0
7089
container_name: odfe-kibana
7190
ports:
7291
- 5601:5601
@@ -121,7 +140,7 @@ If you encounter any `File /usr/share/elasticsearch/config/elasticsearch.yml has
121140

122141
## Change passwords for read-only users
123142

124-
After the cluster starts, change the passwords for the [read-only user accounts](../../security-configuration/api/#read-only-and-hidden-resources): `admin` and `kibanaserver`.
143+
After the cluster starts, change the passwords for the [read-only user accounts](../../security-access-control/api/#read-only-and-hidden-resources): `admin` and `kibanaserver`.
125144

126145
- The `admin` user has full privileges on the cluster.
127146
- `kibanaserver` user has certain permissions to the `.kibana` index that let it perform management tasks like setting index patterns and retrieving visualizations. This user, or one just like it, is required for Kibana to work properly with the Security plugin. We recommend just using `kibanaserver`.

docs/install/docker.md

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@ nav_order: 1
1010
You can pull the Open Distro for Elasticsearch Docker image just like any other image:
1111

1212
```bash
13-
docker pull amazon/opendistro-for-elasticsearch:0.9.0
14-
docker pull amazon/opendistro-for-elasticsearch-kibana:0.9.0
13+
docker pull amazon/opendistro-for-elasticsearch:1.0.0
14+
docker pull amazon/opendistro-for-elasticsearch-kibana:1.0.0
1515
```
1616

1717
Open Distro for Elasticsearch images use `centos:7` as the base image.
@@ -31,7 +31,7 @@ Open Distro for Elasticsearch images use `centos:7` as the base image.
3131
To run the image for local development:
3232

3333
```bash
34-
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" amazon/opendistro-for-elasticsearch:0.9.0
34+
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" amazon/opendistro-for-elasticsearch:1.0.0
3535
```
3636

3737
Then send requests to the server to verify that Elasticsearch is up and running:
@@ -84,16 +84,22 @@ This sample file starts two data nodes and Kibana. If you're running Docker loca
8484
version: '3'
8585
services:
8686
odfe-node1:
87-
image: amazon/opendistro-for-elasticsearch:0.9.0
87+
image: amazon/opendistro-for-elasticsearch:1.0.0
8888
container_name: odfe-node1
8989
environment:
9090
- cluster.name=odfe-cluster
91+
- node.name=odfe-node1
92+
- discovery.seed_hosts=odfe-node1,odfe-node2
93+
- cluster.initial_master_nodes=odfe-node1,odfe-node2
9194
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
9295
- "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
9396
ulimits:
9497
memlock:
9598
soft: -1
9699
hard: -1
100+
nofile:
101+
soft: 65536 # maximum number of open files for the Elasticsearch user, set to at least 65536 on modern systems
102+
hard: 65536
97103
volumes:
98104
- odfe-data1:/usr/share/elasticsearch/data
99105
ports:
@@ -102,23 +108,28 @@ services:
102108
networks:
103109
- odfe-net
104110
odfe-node2:
105-
image: amazon/opendistro-for-elasticsearch:0.9.0
111+
image: amazon/opendistro-for-elasticsearch:1.0.0
106112
container_name: odfe-node2
107113
environment:
108114
- cluster.name=odfe-cluster
115+
- node.name=odfe-node2
116+
- discovery.seed_hosts=odfe-node1,odfe-node2
117+
- cluster.initial_master_nodes=odfe-node1,odfe-node2
109118
- bootstrap.memory_lock=true
110119
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
111-
- discovery.zen.ping.unicast.hosts=odfe-node1
112120
ulimits:
113121
memlock:
114122
soft: -1
115123
hard: -1
124+
nofile:
125+
soft: 65536
126+
hard: 65536
116127
volumes:
117128
- odfe-data2:/usr/share/elasticsearch/data
118129
networks:
119130
- odfe-net
120131
kibana:
121-
image: amazon/opendistro-for-elasticsearch-kibana:0.9.0
132+
image: amazon/opendistro-for-elasticsearch-kibana:1.0.0
122133
container_name: odfe-kibana
123134
ports:
124135
- 5601:5601
@@ -151,7 +162,7 @@ docker run \
151162
-p 9200:9200 -p 9600:9600 \
152163
-e "discovery.type=single-node" \
153164
-v /<full-path-to>/custom-elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
154-
amazon/opendistro-for-elasticsearch:0.9.0
165+
amazon/opendistro-for-elasticsearch:1.0.0
155166
```
156167

157168
You can perform the same operation in `docker-compose.yml` using a relative path:
@@ -199,15 +210,15 @@ vm.max_map_count=262144
199210

200211
Then run `sudo sysctl -p` to reload.
201212

202-
The `docker-compose.yml` file above also contains several key settings: `bootstrap.memory_lock=true`, `ES_JAVA_OPTS=-Xms512m -Xmx512m`, and `9600:9600`. Respectively, these settings disable memory swapping (along with `memlock`), set the size of the Java heap (we recommend half of system RAM), and allow you to access Performance Analyzer on port 9600.
213+
The `docker-compose.yml` file above also contains several key settings: `bootstrap.memory_lock=true`, `ES_JAVA_OPTS=-Xms512m -Xmx512m`, `nofile 65536` and `port 9600`. Respectively, these settings disable memory swapping (along with `memlock`), set the size of the Java heap (we recommend half of system RAM), set a limit of 65536 open files for the Elasticsearch user, and allow you to access Performance Analyzer on port 9600.
203214

204215

205216
## Run with custom plugins
206217

207218
To run the image with a custom plugin, first create a [`Dockerfile`](https://docs.docker.com/engine/reference/builder/):
208219

209220
```
210-
FROM amazon/opendistro-for-elasticsearch:0.9.0
221+
FROM amazon/opendistro-for-elasticsearch:1.0.0
211222
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch <plugin-name-or-url>
212223
```
213224

@@ -221,7 +232,7 @@ docker run -p 9200:9200 -p 9600:9600 -v /usr/share/elasticsearch/data odfe-custo
221232
You can also use a `Dockerfile` to pass your own certificates for use with the [Security](../../security-configuration/) plugin, similar to the `-v` argument in [Configure Elasticsearch](#configure-elasticsearch):
222233

223234
```
224-
FROM amazon/opendistro-for-elasticsearch:0.9.0
235+
FROM amazon/opendistro-for-elasticsearch:1.0.0
225236
COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/
226237
COPY --chown=elasticsearch:elasticsearch my-key-file.pem /usr/share/elasticsearch/config/
227238
COPY --chown=elasticsearch:elasticsearch my-certificate-chain.pem /usr/share/elasticsearch/config/

docs/install/other-components.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,4 @@ Open Distro for Elasticsearch has a number of other components that you might wa
1212
- [Java Database Connectivity (JDBC) driver](../../sql/jdbc)
1313
- [PerfTop client for Performance Analyzer](../../pa/)
1414
- [Job Scheduler plugin](https://github.com/opendistro-for-elasticsearch/job-scheduler), an extensible plugin for running periodic jobs
15+
- [Alerting CLI](https://github.com/mihirsoni/odfe-monitor-cli), a command line interface that lets you use YAML files to manage your Open Distro for Elasticsearch monitors

docs/install/plugins.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Navigate to the Elasticsearch home directory (likely `/usr/share/elasticsearch`)
1818
#### Security
1919

2020
```bash
21-
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-security/opendistro_security-0.9.0.0.zip
21+
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-security/opendistro_security-1.0.0.0.zip
2222
```
2323

2424
After installing the Security plugin, you can run `sudo sh /usr/share/elasticsearch/plugins/opendistro_security/tools/install_demo_configuration.sh` to quickly get started with demo certificates. Otherwise, you must configure it manually.
@@ -27,22 +27,22 @@ After installing the Security plugin, you can run `sudo sh /usr/share/elasticsea
2727
#### Alerting
2828

2929
```bash
30-
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-alerting/opendistro_alerting-0.9.0.0.zip
30+
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-alerting/opendistro_alerting-1.0.0.0.zip
3131
```
3232

3333

3434
#### SQL
3535

3636
```bash
37-
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-sql/opendistro_sql-0.9.0.0.zip
37+
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-sql/opendistro_sql-1.0.0.0.zip
3838
```
3939

4040

4141
{% comment %}
4242
#### Performance Analyzer
4343

4444
```bash
45-
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/performance-analyzer/opendistro_performance_analyzer-0.9.0.0.zip
45+
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/performance-analyzer/opendistro_performance_analyzer-1.0.0.0.zip
4646
```
4747

4848
Performance Analyzer requires so many additional configuration steps that we don't recommend installing it as a standalone plugin. After installing the plugin:

0 commit comments

Comments
 (0)