You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: deploy-manage/deploy/cloud-enterprise/add-custom-bundles-plugins.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -184,7 +184,7 @@ In this example, we assume the Identity Provider does not publish its SAML metad
184
184
185
185
## Example: Custom JVM trust store bundle [ece-add-custom-bundle-example-cacerts]
186
186
187
-
If you are using SSL certificates signed by non-public certificate authorities, {{es}} is not able to communicate with the services using those certificates unless you import a custom JVM trust store containing the certificates of your signing authority into your {{ece}} installation. You’ll need the trust store to access snapshot repositories like Minio, for your {{ece}} proxy, or to reindex from remote.
187
+
If you are using SSL certificates signed by non-public certificate authorities, {{es}} is not able to communicate with the services using those certificates unless you import a custom JVM trust store containing the certificates of your signing authority into your {{ece}} installation. You’ll need the trust store to access snapshot repositories like MinIO, for your {{ece}} proxy, or to reindex from remote.
If you are installing ECE without internet access (commonly called an offline or air-gapped installation), you will need to use an on-premise storage service. We suggest that you use [Minio](https://www.minio.io/). For our installation notes, check [Snapshotting to Minio On-Premise Storage](minio-on-premise-repository.md).
39
+
If you are installing ECE without internet access (commonly called an offline or air-gapped installation), you will need to use an on-premise storage service. We suggest that you use [MinIO](https://www.minio.io/). For our installation notes, check [Snapshotting to MinIO On-Premise Storage](minio-on-premise-repository.md).
40
40
::::
41
41
42
42
@@ -47,7 +47,7 @@ The following guides provide instructions on adding a snapshot repository in ECE
Minio is a popular, open-source distributed object storage server compatible with the Amazon AWS S3 API. You can use it with {{ece}} installations when you want to store your {{es}} snapshots locally.
13
+
[MinIO](https://min.io/docs/minio/container/index.html) is a popular, open-source object storage server compatible with the Amazon AWS S3 API. As an [S3 compatible service](/deploy-manage/tools/snapshot-and-restore/s3-repository.md#repository-s3-compatible-services), MinIO is supported for use as a snapshot repository in {{ece}} (ECE).
14
14
15
+
This guide walks you through integrating MinIO with ECE to store your {{es}} snapshots.
15
16
16
-
## Create a test environment [ece-minio-test]
17
+
::::{important}
18
+
Avoid running MinIO directly on ECE hosts. Sharing infrastructure can lead to resource contention, especially disk I/O, and may affect the performance and stability of your Elastic workloads. It also complicates upgrades, troubleshooting, and supportability.
17
19
18
-
We recommend following the [Minio Quickstart Guide Docker Container instructions](https://docs.minio.io/docs/minio-docker-quickstart-guide) to create a simple Minio standalone installation for your initial evaluation and development.
19
-
20
-
Be sure to use the `docker -v` option to map persistent storage to the container.
21
-
22
-
23
-
## Production environment prerequisites [ece-minio-requirements]
24
-
25
-
Installing Minio for production requires a high-availability configuration where Minio is running in [Distributed mode](https://docs.minio.io/docs/distributed-minio-quickstart-guide).
26
-
27
-
As mentioned in the Minio documentation, you will need to have 4-16 Minio drive mounts. There is no hard limit on the number of Minio nodes. It might be convenient to place the Minio node containers on your ECE hosts to ensure you have a suitable level of availability, but those can not be located on the same hosts as ECE proxies since they both listen on the same port.
28
-
29
-
The following illustration is a sample architecture for a [large ECE installation](../../deploy/cloud-enterprise/deploy-large-installation.md). Note that there is at least one MinIO container in *each* availability zone.
30
-
31
-
There are a number of different ways of orchestrating the Minio deployment (Docker Compose, Kubernetes, and so on). We suggest you use the method most familiar to you.
32
-
33
-
We recommend:
34
-
35
-
* Using a single Minio endpoint with the {{ece}} installation, to simplify repository management.
36
-
* Securing access to the Minio endpoint with TLS.
20
+
If you're evaluating MinIO in a test system, do not place MinIO containers on the same hosts as ECE proxies, as both services use the same port.
This section provides guidance and recommendations for deploying MinIO. It does not include installation steps. As MinIO is a third-party product, its deployment, configuration, and maintenance are outside the scope of Elastic support.
43
26
44
-
## Create an offline installation [ece-minio-offline-installation]
27
+
For installation instructions, refer to the official [MinIO documentation](https://min.io/docs/).
45
28
46
-
If you are installing MinIO offline, the process is very similar to the [offline installation of {{ece}}](../../deploy/cloud-enterprise/air-gapped-install.md). There are two options:
29
+
The performance and reliability of MinIO depend on its configuration and the underlying infrastructure. Consider the following best practices:
47
30
48
-
* Use a private Docker repository and [install the Minio images in the private repository](https://docs.docker.com/registry/deploying/).
49
-
* Download the Minio images from an internet-connected machine, then use docker save to bundle the images into tar files. Copy the TAR files to the target hosts and use `docker load` to install.
31
+
* For production use, deploy MinIO in [distributed mode](https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#minio-mnmd).
32
+
* Use a single MinIO endpoint with the ECE installation, to simplify repository configuration.
33
+
* Secure access to the MinIO endpoint with TLS.
50
34
51
-
Gather the following after your installation:
35
+
After deployment, make sure you collect the following values:
52
36
53
-
*Minio AccessKey
54
-
*Minio SecretKey
55
-
*Endpoint URL
37
+
*MinIO Access Key
38
+
*MinIO Secret Key
39
+
*MinIO endpoint URL
56
40
57
41
::::{tip}
58
-
Minio might report various Endpoint URLs, be sure to choose the one that will be routable from your {{es}} Docker containers.
42
+
MinIO may report multiple endpoint URLs. Be sure to select the one reachable from your {{es}} containers running on ECE allocator hosts.
59
43
::::
60
44
45
+
### Testing and evaluation
61
46
47
+
Use the [MinIO Quickstart Guide](https://charts.min.io/) or the [container deployment guide](https://min.io/docs/minio/container/index.html) to spin up a simple standalone MinIO container. Use `-v` to map persistent storage when using the `docker` or `podman` options.
62
48
63
-
## Create the S3 bucket [ece-minio-create-s3-bucket]
64
-
65
-
How you create the AWS S3 bucket depends on what version of {{es}} you are using:
49
+
### Production environments
66
50
67
-
* For version 7.x:
51
+
Set up MinIO in distributed mode across multiple nodes and drives. You can use Docker Compose, Kubernetes, or another orchestration tool of your choice.
68
52
69
-
1. Using the Minio browser or an S3 client application, create an S3 bucket to store your snapshots.
70
-
2.[Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md) and [add the S3 repository plugin](elasticsearch://reference/elasticsearch-plugins/plugin-management.md) to your cluster.
53
+
## Create the S3 bucket [ece-minio-create-s3-bucket]
71
54
72
-
* For versions 8.0 and later, {{es}} has built-in support for AWS S3 repositories; no repository plugin is needed. Use the Minio browser or an S3 client application to create an S3 bucket to store your snapshots.
55
+
After installing MinIO you will need to create a bucket to store your deployments' snapshots. Use the MinIO browser or an S3 client application to create an S3 bucket to store your snapshots.
73
56
74
57
::::{tip}
75
58
Don’t forget to make the bucket name DNS-friendly, for example no underscores or uppercase letters. For more details, read the [bucket restrictions](https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html).
76
59
::::
77
60
61
+
## {{ece}} configuration [ece-install-with-minio]
78
62
63
+
This section describes the configuration changes required to use MinIO storage within ECE to make periodic snapshots of your deployments. The required steps include:
79
64
80
-
## {{ece}} configuration [ece-install-with-minio]
65
+
* Configuring the repository at ECE level
66
+
* Associating it with your deployments
67
+
* Applying specific YAML settings to the deployments
81
68
82
-
You can configure existing deployments, or create new ones, with the following changes to use Minio storage.
69
+
### Prerequisites
83
70
71
+
Before integrating ECE with MinIO, ensure you have the following details from your MinIO deployment:
72
+
73
+
* MinIO Access Key
74
+
* MinIO Secret Key
75
+
* MinIO endpoint URL
76
+
* S3 bucket name
84
77
85
78
### Add the repository to {{ece}} [ece-add-repository]
86
79
87
-
You must add the new repository to {{ece}} before it can be used with your {{es}} clusters.
80
+
You must add the new repository at ECE platform level before it can be used by your {{es}} deployments.
88
81
89
82
1.[Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
90
83
2. From the **Platform** menu, select **Repositories**.
@@ -97,76 +90,50 @@ You must add the new repository to {{ece}} before it can be used with your {{es}
Once the MinIO repository is created at the ECE platform level, you can associate it with your {{es}} deployments in two ways:
122
107
108
+
* For new deployments, select the repository from the **Snapshot repository** drop-down list while [creating the deployment](/deploy-manage/deploy/cloud-enterprise/create-deployment.md).
123
109
124
-
### Additional settings for 6.x clusters [ece-6.x-settings]
110
+
* For existing deployments, associate the repository by following the instructions in [Manage {{es}} clusters repositories](/deploy-manage/tools/snapshot-and-restore/cloud-enterprise.md#ece-manage-repositories-clusters).
125
111
126
-
For {{es}} versions 6.0 and later, after selecting the repository, you also need to set your **User Settings** YAML to specify the endpoint and protocol. For example:
112
+
### Additional settings for {{es}} [ece-6.x-settings]
113
+
114
+
After selecting the repository, you also need to configure your [{{es}} user settings YAML](/deploy-manage/deploy/cloud-enterprise/edit-stack-settings-elasticsearch.md) to specify the endpoint and protocol. For example:
Check the [{{es}} S3 plugin details](https://www.elastic.co/guide/en/elasticsearch/plugins/6.8/repository-s3-client.html) for more information.
133
120
121
+
Refer to the [{{es}} S3 plugin details](/deploy-manage/tools/snapshot-and-restore/s3-repository.md) for more information.
134
122
135
-
## Upgrade from 5.x to 6.x {{es}} clusters [ece-upgrade-minio]
123
+
#### Add S3 repository plugin (only for {{es}} 7.x)
136
124
137
-
The configuration options for the {{es}} S3 repository plugin have changed from 5.x to 6.x versions and you must copy the endpoint and protocol values from your repository configuration to your **User Settings** YAML before you upgrade.
125
+
For {{es}} clusters in version 7.x you must add the S3 repository plugin to your cluster. Refer to [Managing plugins for ECE](elasticsearch://reference/elasticsearch-plugins/plugin-management.md#managing-plugins-for-ece) for more details.
138
126
127
+
::::{note}
128
+
For versions 8.0 and later, {{es}} has built-in support for AWS S3 repositories; no repository plugin is needed.
129
+
::::
139
130
140
131
## Verify snapshots [ece-minio-verify-snapshot]
141
132
142
-
The cluster should make a snapshot when the repository is set up. You can check that by going to the **Elasticsearch** and then the **Snapshots** page.
143
-
144
-
As an extra verification step, you can restore a cluster using the snapshots that have been taken.
145
-
146
-
1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
147
-
2. Get the plan from your test cluster.
148
-
149
-
1. From the **Deployments** page, select your deployment.
150
-
151
-
Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
152
-
153
-
2. From your deployment menu, go to the **Edit** page then go to the bottom of the page and select **advanced {{es}} configuration**.
154
-
3. Copy the JSON format under the **Deployment configuration** heading.
155
-
156
-
3. Create a new {{es}} cluster as your target.
157
-
4. On the new cluster, open the advanced cluster configuration editor. In the transient section, add the `restore_snapshot` settings to the plan.
158
-
159
-
```json
160
-
...
161
-
"transient": {
162
-
"restore_snapshot": {
163
-
"repository_name": "<Minio repository name>",
164
-
"snapshot_name": "latest_success"
165
-
}
166
-
}
167
-
```
133
+
The cluster should make periodic snapshots when the repository is set up and associated to it. You can check this in the **Elasticsearch > Snapshots** section of the deployment page in the [Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
168
134
169
-
5. Select **Save** to restore from the snapshot. When the plan update is complete, you can check the restored indexes in your target cluster.
135
+
As an extra verification step, you can [restore snapshots across clusters](/deploy-manage/tools/snapshot-and-restore/ece-restore-across-clusters.md).
170
136
171
-
More details are available to [work with snapshots](../snapshot-and-restore.md).
137
+
Refer to [work with snapshots](../snapshot-and-restore.md) for more information around {{es}} snapshot and restore.
172
138
139
+
For additional considerations on performance, reliability, and troubleshooting when using MinIO as a snapshot repository, refer to [Using MinIO with {{es}}](/deploy-manage/tools/snapshot-and-restore/s3-repository.md#using-minio-with-elasticsearch).
Copy file name to clipboardExpand all lines: deploy-manage/tools/snapshot-and-restore/s3-repository.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -384,6 +384,7 @@ There are many systems, including some from very well-known storage vendors, whi
384
384
385
385
You can perform some basic checks of the suitability of your storage system using the [repository analysis API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-snapshot-repository-analyze). If this API does not complete successfully, or indicates poor performance, then your storage system is not fully compatible with AWS S3 and therefore unsuitable for use as a snapshot repository. However, these checks do not guarantee full compatibility.
386
386
387
+
$$$using-minio-with-elasticsearch$$$
387
388
::::{admonition} Using MinIO with {{es}}
388
389
[MinIO](https://minio.io) is an example of a storage system that provides an S3-compatible API. The `s3` repository type allows {{es}} to work with MinIO-backed repositories as well as repositories stored on AWS S3. The {{es}} test suite includes some checks which aim to detect deviations in behavior between MinIO and AWS S3. Elastic will report directly to the MinIO project any deviations in behavior found by these checks. If you are running a version of MinIO whose behavior deviates from that of AWS S3 then you must upgrade your MinIO installation. If in doubt, please contact the MinIO support team for further information.
0 commit comments