diff --git a/deploy-manage/security/secure-settings.md b/deploy-manage/security/secure-settings.md index b1aae0c58a..61ddf5d025 100644 --- a/deploy-manage/security/secure-settings.md +++ b/deploy-manage/security/secure-settings.md @@ -7,7 +7,6 @@ applies_to: self: ga mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-configuring-keystore.html - - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-restful-api-examples-configuring-keystore.html - https://www.elastic.co/guide/en/cloud/current/ec-configuring-keystore.html - https://www.elastic.co/guide/en/cloud-heroku/current/ech-configuring-keystore.html - https://www.elastic.co/guide/en/elasticsearch/reference/current/secure-settings.html @@ -16,222 +15,168 @@ mapped_pages: # Secure your settings -$$$reloadable-secure-settings$$$ +Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. For this use case, {{es}} and {{kib}} provide secure keystores to store sensitive configuration values such as passwords, API keys, and tokens. -$$$ec-add-secret-values$$$ +Secure settings are often referred to as **keystore settings**, since they must be added to the product-specific keystore rather than the standard `elasticsearch.yml` or `kibana.yml` files. Unlike regular settings, they are encrypted and protected at rest, and they cannot be read or modified through the usual configuration files or environment variables. -$$$change-password$$$ +Keystore settings must be handled using a specific tool or method depending on the deployment type. The following table summarizes how {{es}} and {{kib}} keystores are supported and managed across different deployment models: -$$$creating-keystore$$$ +| Deployment Type | {{es}} keystore configuration | {{kib}} keystore configuration | +|----------------------|--------------------------------------------|----------------------------------------| +| {{serverless-full}} | Not available | Not available | +| {{ech}} | Cloud UI | Not available | +| {{ece}} | Cloud UI or API | Not available | +| {{eck}} | Kubernetes secrets | Kubernetes secrets | +| Self-managed | Manual configuration with [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) | Manual configuration with `kibana-keystore` | -Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. Depending on the settings you need to protect, you can use: +This section describes how to configure and manage secure settings in each keystore depending on the deployment model: +* [{{es}} secure settings](./secure-settings.md#elasticsearch) +* [{{kib}} secure settings](./secure-settings.md#kibana) -- [The {{es}} keystore](secure-settings.md#the-es-keystore) and the [`elasticsearch-keystore` tool](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) to manage {{es}} settings. -- [The {{kib}} keystore](secure-settings.md#the-kib-keystore) and the `kibana-keystore` tool to manage {{kib}} settings. -- [Kubernetes secrets](k8s-secure-settings.md), if you are using {{eck}}. +## {{es}} secure settings [elasticsearch] +The {{es}} keystore has some important characteristics and limitations to be aware of: -:::{important} -Only some settings are designed to be read from the keystore. However, the keystore has no validation to block unsupported settings. Adding unsupported settings to the keystore causes [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to fail and if not addressed, {{es}} will fail to start. To check whether a setting is supported in the keystore, look for a "Secure" qualifier in the [setting reference](/reference/index.md). -::: +* **Only specific settings are allowed**: The keystore accepts only settings marked as *secure* in the [{{es}} configuration reference](elasticsearch://reference/elasticsearch/configuration-reference/index.md). Adding unsupported settings to the keystore causes the validation in the [`reload_secure_settings` API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to fail and can also prevent {{es}} from starting. +* **Mandatory for secure settings**: Settings marked as secure must be added to the keystore. They cannot be placed in `elasticsearch.yml` or set using environment variables. This differs from the {{kib}} keystore, which supports all settings. +* **Changes require a restart**: Most secure settings take effect only after restarting the nodes. However, some are marked as *reloadable* and can be updated without a restart using the [`reload_secure_settings`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API. Refer to [Reloadable settings](#reloadable-secure-settings) for more information. +* **Keystore is per-node**: Each node in the cluster has its own keystore file. Secure settings must be specified individually on every node and must have the same values across the cluster. Orchestrated deployments, such as ECH, ECE, and ECK, handle this automatically when configuring secure settings. -## The {{es}} keystore +::::{tip} +To check whether a setting can be added to the keystore and supports reloading without a restart, look for the `Secure` and `Reloadable` qualifiers in the [{{es}} configuration reference](elasticsearch://reference/elasticsearch/configuration-reference/index.md). + +For example, you can search for `secure_bind_password` in the [security settings document](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md) to see how these qualifiers appear in context. +:::: -With the {{es}} keystore, you can add a key and its secret value, then use the key in place of the secret value when you configure your sensitive settings. +The instructions below cover how to manage {{es}} keystore settings for each deployment type. :::::{tab-set} :group: deployment-type -::::{tab-item} {{ecloud}} +::::{tab-item} ECH and ECE :sync: cloud +% ### ECE and ECH +You can manage {{es}} secure settings in the **Security > {{es}} keystore** section of your deployment page in the Elastic Cloud Console or ECE Cloud UI. -There are three types of secrets that you can use: +:::{note} +{{ece}} also supports managing {{es}} keystore of your deployments through its [RESTful API](https://www.elastic.co/docs/api/doc/cloud-enterprise/). Refer to [Configure {{es}} keystore through ECE API](cloud://reference/cloud-enterprise/ece-restful-api-examples-configuring-keystore.md) for an example. +::: -* **Single string** - Associate a secret value to a setting. -* **Multiple strings** - Associate multiple keys to multiple secret values. -* **JSON block/file** - Associate multiple keys to multiple secret values in JSON format. +There are three input formats you can use for secure setting values: +* **Single string**: Associate a secret value to a setting. +* **Multiple strings**: A group of key-value pairs, each stored as part of the setting's value. +* **JSON block/file**: A structured JSON object containing multiple key-value pairs, typically used for more complex secrets like service account credentials. -**Add secret values** +#### Add secure settings [ec-add-secret-values] -Add keys and secret values to the keystore. +Add settings and secret values to the keystore. -1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page or on the **Hosted deployments** page, then select **Manage** to access its settings menus. +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body) or [ECE Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +2. Find your deployment and select **Manage**. - On the **Hosted deployments** page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + On the **Deployments** page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. 3. From your deployment menu, select **Security**. 4. Locate **{{es}} keystore** and select **Add settings**. 5. On the **Create setting** window, select the secret **Type**. 6. Configure the settings, then select **Save**. -7. All the modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. + :::{important} + All modifications to the non-reloadable settings take effect only after restarting {{es}}. [Reloadable](#reloadable-secure-settings) keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. Adding unsupported settings to the keystore will cause {{es}} to fail to start. + ::: -**Delete secret values** +#### Remove secure settings -When your keys and secret values are no longer needed, delete them from the keystore. +When your secure settings are no longer needed, delete them from the keystore. -1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page or on the **Hosted deployments** page, then select **Manage** to access its settings menus. +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body) or [ECE Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +2. Find your deployment and select **Manage**. - On the **Hosted deployments** page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + On the **Deployments** page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. 3. From your deployment menu, select **Security**. -4. From the **Existing keystores** list, use the delete icon next to the **Setting Name** that you want to delete. +4. From the **Existing keystores** list, use the delete icon next to setting that you want to delete. 5. On the **Confirm to delete** window, select **Confirm**. -6. All modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. - :::: -::::{tab-item} {{ece}} -:sync: ece - -There are three types of secrets that you can use: - -* **Single string** - Associate a secret value to a setting. -* **Multiple strings** - Associate multiple keys to multiple secret values. -* **JSON block/file** - Associate multiple keys to multiple secret values in JSON format. +::::{tab-item} ECK +:sync: eck +% ### ECK +In ECK, the operator simplifies secure settings configuration by relying on Kubernetes secrets. +Refer to [Configure secure settings on ECK](./k8s-secure-settings.md) for details and examples. +:::: -**Add secret values** - -Add keys and secret values to the keystore. - -1. [Log into the Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the **Deployments** page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. From your deployment menu, select **Security**. -4. Locate **{{es}} keystore** and select **Add settings**. -5. On the **Create setting** window, select the secret **Type**. -6. Configure the settings, then select **Save**. -7. All the modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. - - - -**Delete secret values** - -When your keys and secret values are no longer needed, delete them from the keystore. - -1. [Log into the Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the **Deployments** page, select your deployment. +::::{tab-item} Self-managed +:sync: self-managed +% ### Self-managed +In self-managed deployments, you're responsible for configuring and maintaining the Elasticsearch keystore on each node individually. - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. +{{es}} provides the [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) command-line tool to help with this task. It allows you to list, add, remove, and update secure settings, as well as protect the keystore with a password if wanted. -3. From your deployment menu, select **Security**. -4. From the **Existing keystores** list, use the delete icon next to the **Setting Name** that you want to delete. -5. On the **Confirm to delete** window, select **Confirm**. -6. All modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. +:::{important} +Secure settings must be specified on every node, and must have the same values across the cluster to ensure consistent behavior. -:::{dropdown} Using the API +Changes to the keystore take effect only after restarting {{es}}, except for [reloadable settings](#reloadable-secure-settings) that can be refreshed using the API. +::: -**Steps** +#### Create the keystore [creating-keystore] -Create the keystore: +To create the Elasticsearch keystore, use the `create` command: ```sh -curl -k -X PATCH -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/deployments/$DEPLOYMENT_ID/elasticsearch/$REF_ID/keystore \ -{ - "secrets": { - "s3.client.CLIENT_NAME.access_key": { - "as_file": false - "value": "ACCESS_KEY_VALUE" - } - "s3.client.CLIENT_NAME.secret_key": { - "value": "SECRET_KEY_VALUE" - } - } -} +bin/elasticsearch-keystore create -p ``` -`ELASTICSEARCH_CLUSTER_ID` -: The {{es}} cluster ID as shown in the Cloud UI or obtained through the API +You are prompted to enter a keystore password, but setting one is optional. If you press Enter without typing a password, the keystore file will be created without password protection. -List the keys defined in the keystore: +The command creates a file named `elasticsearch.keystore` alongside the `elasticsearch.yml` file. -```sh -{ - "secrets": { - "s3.client.CLIENT_NAME.access_key": { - "as_file": false - }, - "s3.client.CLIENT_NAME.secret_key": { - "as_file": false - } - } -} -``` +#### List settings in the keystore -Create the credentials for an S3 or Minio repository: +To list the settings in the keystore, use the `list` command. ```sh -curl -k -X PUT -H "Authorization: ApiKey $ECE_API_KEY" https://$COODINATOR_HOST:12443/api/v1/clusters/elasticsearch/$ELASTICSEARCH_CLUSTER_ID/_snapshot/s3-repo -{ - "type": "s3", - "settings": { - "bucket": "s3_REPOSITORY_NAME", - "client": "CLIENT_NAME", - "base_path": "PATH_NAME" - } -} +bin/elasticsearch-keystore list ``` +If the {{es}} keystore is password protected, you are prompted to enter the password. + +#### Add secure settings to the keystore -Create the credentials for a GCS repository: +Sensitive string settings, like authentication credentials for Cloud plugins, can be added with the `add` command: ```sh -curl -k -X PUT -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/clusters/elasticsearch/ELASTICSEARCH_CLUSTER_ID/_snapshot/s3-repo -{ - "type": "gcs", - "settings": { - "bucket": "BUCKET_NAME", - "base_path": "BASE_PATH_NAME", - "client": "CLIENT_NAME" - } -} +bin/elasticsearch-keystore add the.setting.name.to.set ``` -To use GCS snapshots, the cluster must have the `repository-gcs` plugin enabled. +You are prompted to enter the value of the setting. If the {{es}} keystore is password protected, you are also prompted to enter the password. +#### Remove secure settings from the keystore -Remove keys that are defined in the keystore: +To remove a setting from the keystore, use the `remove` command: ```sh -curl -k -X PATCH -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/deployments/$DEPLOYMENT_ID/elasticsearch/$REF_ID/keystore \ -{ - "secrets": { - "KEY_TO_REMOVE": {} - } -} +bin/elasticsearch-keystore remove the.setting.name.to.remove ``` +If the {{es}} keystore is password protected, you are prompted to enter the password. -**Verify your credentials** - -If your credentials are invalid, an administrator can verify that they are correct by checking the `keystore` field in the cluster metadata. +#### Other examples [change-password] -If the credential values are correct, but do not work, the keystore file could be out of sync on one or more nodes. To sync the keystore file, update the value for the key by using the patch API to delete the key from keystore, then add it back again. - -::: +For a full command reference and additional examples, such as displaying stored values or adding entire files as setting values, refer to the [`elasticsearch-keystore` tool documentation](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md). :::: -::::{tab-item} Self-managed -:sync: self-managed - -All the modifications to the keystore take effect only after restarting {{es}}. - -These settings, just like the regular ones in the `elasticsearch.yml` config file, need to be specified on each node in the cluster. Currently, all secure settings are node-specific settings that must have the same value on every node. - +::::: -**Reloadable secure settings** +### Reloadable secure settings [reloadable-secure-settings] -Just like the settings values in `elasticsearch.yml`, changes to the keystore contents are not automatically applied to the running {{es}} node. Re-reading settings requires a node restart. However, certain secure settings are marked as **reloadable**. Such settings can be re-read and applied on a running node. +Just like the settings values in `elasticsearch.yml`, changes to the keystore contents are not automatically applied to the running {{es}} node. Re-reading settings requires a node restart. -You can define these settings before the node is started, or call the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) after the settings are defined to apply them to a running node. +However, certain secure settings are marked as `Reloadable` in [{{es}} reference documentation](elasticsearch://reference/elasticsearch/configuration-reference/index.md). Such settings can be re-read and applied on a running node by using the [reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings). -The values of all secure settings, **reloadable** or not, must be identical across all cluster nodes. After making the desired secure settings changes, using the `bin/elasticsearch-keystore add` command, call: +The values of all secure settings, whether reloadable or not, must be identical across all nodes. After making the desired changes, call the following endpoint. If your keystore is password-protected, include the `secure_settings_password` parameter: ```console POST _nodes/reload_secure_settings @@ -239,44 +184,35 @@ POST _nodes/reload_secure_settings "secure_settings_password": "keystore-password" <1> } ``` +1. Only required if the {{es}} keystore is password-protected. -1. The password that the {{es}} keystore is encrypted with. - - -This API decrypts, re-reads the entire keystore and validates all settings on every cluster node, but only the **reloadable** secure settings are applied. Changes to other settings do not go into effect until the next restart. Once the call returns, the reload has been completed, meaning that all internal data structures dependent on these settings have been changed. Everything should look as if the settings had the new value from the start. - -When changing multiple **reloadable** secure settings, modify all of them on each cluster node, then issue a [`reload_secure_settings`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) call instead of reloading after each modification. - -There are reloadable secure settings for: - -* [The Azure repository plugin](/deploy-manage/tools/snapshot-and-restore/azure-repository.md) -* [The EC2 discovery plugin](elasticsearch://reference/elasticsearch-plugins/discovery-ec2-usage.md#_configuring_ec2_discovery) -* [The GCS repository plugin](/deploy-manage/tools/snapshot-and-restore/google-cloud-storage-repository.md) -* [The S3 repository plugin](/deploy-manage/tools/snapshot-and-restore/s3-repository.md) -* [Monitoring settings](elasticsearch://reference/elasticsearch/configuration-reference/monitoring-settings.md) -* [{{watcher}} settings](elasticsearch://reference/elasticsearch/configuration-reference/watcher-settings.md) -* [JWT realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-jwt-settings) -* [Active Directory realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-ad-settings) -* [LDAP realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-ldap-settings) -* [Remote cluster credentials for the API key based security model](/deploy-manage/remote-clusters/remote-clusters-settings.md#remote-cluster-credentials-setting) - +This API decrypts, re-reads the entire keystore and validates all settings on every cluster node, but only the reloadable secure settings are applied. Changes to other settings do not go into effect until the next restart. Once the call returns, the reload has been completed, meaning that all internal data structures dependent on these settings have been changed. Everything should look as if the settings had the new value from the start. +::::{tip} +When changing multiple reloadable secure settings, modify all of them on each cluster node, then issue a [`reload_secure_settings`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) call instead of reloading after each modification. :::: -::::: +Reloadable secure settings are available for snapshot repository plugins, watcher, monitoring, and certain authentication realms. Refer to the relevant documentation for each feature to see if secure settings can be reloaded. -## The {{kib}} keystore +## {{kib}} secure settings [kibana] ```{applies_to} deployment: self: ga + eck: ga ``` -Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. For this use case, {{kib}} provides a keystore, and the `kibana-keystore` tool to manage the settings in the keystore. +{{kib}} supports secure settings through its own keystore, similar to the {{es}} keystore. It provides a way to protect sensitive configuration values, such as authentication credentials or encryption keys, by storing them securely outside of `kibana.yml`. ::::{note} -* Run all commands as the user who runs {{kib}}. -* Any valid {{kib}} setting can be stored in the keystore securely. Unsupported, extraneous or invalid settings will cause {{kib}} to fail to start up. +Unlike with {{es}} keystore, any valid {{kib}} setting can be stored securely in the keystore. However, as with {{es}}, adding invalid, unsupported, or extraneous settings will cause Kibana to fail to start. Always ensure the setting exists and is properly formatted in the [configuration reference](kibana://reference/configuration-reference.md) before adding it to the keystore. +:::: + +This section provides examples of {{kib}} secure settings handling using the `kibana-keystore` command-line tool for self-managed deployments. + +For ECK deployments, the configuration process is similar to {{es}} and is documented in the [{{kib}} secure settings section for ECK](./k8s-secure-settings.md#k8s-kibana-secure-settings). +::::{important} +In the following examples, run all commands as the user who runs {{kib}}. :::: ### Create the keystore [creating-keystore] @@ -289,7 +225,7 @@ bin/kibana-keystore create The file `kibana.keystore` will be created in the `config` directory defined by the environment variable `KBN_PATH_CONF`. -To create a password protected keystore use the `--password` flag. +To create a password protected keystore use the `--password` flag. Setting a password is optional. ### List settings in the keystore [list-settings]