From 118c544b6e01fc6800e18207df96e9639226a10d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 3 Apr 2025 15:00:25 +0200 Subject: [PATCH 1/5] secure settings preliminary changes --- deploy-manage/security/secure-settings.md | 245 +++++++++------------- 1 file changed, 95 insertions(+), 150 deletions(-) diff --git a/deploy-manage/security/secure-settings.md b/deploy-manage/security/secure-settings.md index 2550d7bae7..b6bfb0a3d2 100644 --- a/deploy-manage/security/secure-settings.md +++ b/deploy-manage/security/secure-settings.md @@ -7,7 +7,6 @@ applies_to: self: ga mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-configuring-keystore.html - - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-restful-api-examples-configuring-keystore.html - https://www.elastic.co/guide/en/cloud/current/ec-configuring-keystore.html - https://www.elastic.co/guide/en/cloud-heroku/current/ech-configuring-keystore.html - https://www.elastic.co/guide/en/elasticsearch/reference/current/secure-settings.html @@ -16,48 +15,68 @@ mapped_pages: # Secure your settings -$$$reloadable-secure-settings$$$ - $$$ec-add-secret-values$$$ $$$change-password$$$ $$$creating-keystore$$$ -Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. Depending on the settings you need to protect, you can use: +Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. For this use case, {{es}} and {{kib}} provide secure keystores to store sensitive configuration values such as passwords, API keys, and tokens. -- [The {{es}} keystore](secure-settings.md#the-es-keystore) and the [`elasticsearch-keystore` tool](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) to manage {{es}} settings. -- [The {{kib}} keystore](secure-settings.md#the-kib-keystore) and the `kibana-keystore` tool to manage {{kib}} settings. -- [Kubernetes secrets](k8s-secure-settings.md), if you are using {{eck}}. +Secure settings are often referred to as **keystore settings**, since they must be added to the product-specific keystore rather than the standard `elasticsearch.yml` or `kibana.yml` files. Unlike regular settings, they are encrypted and protected at rest, and they cannot be read or modified through the usual configuration files or environment variables. +Keystore settings must be handled using a specific tool or method depending on the deployment type. The following table summarizes how {{es}} and {{kib}} keystores are supported and managed across different deployment models: -:::{important} -Only some settings are designed to be read from the keystore. However, the keystore has no validation to block unsupported settings. Adding unsupported settings to the keystore causes [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to fail and if not addressed, {{es}} will fail to start. To check whether a setting is supported in the keystore, look for a "Secure" qualifier in the [setting reference](/reference/index.md). -::: +| Deployment Type | {{es}} keystore configuration | {{kib}} keystore configuration | +|----------------------|--------------------------------------------|----------------------------------------| +| {{ech}} | Cloud UI | Not available | +| {{ece}} | Cloud UI or API | Not available | +| {{eck}} | Kubernetes `secrets` | Kubernetes `secrets` | +| Self-managed | Manual configuration with [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) | Manual configuration with `kibana-keystore` | + +This section describes how to configure and manage secure settings in each keystore depending on the deployment model: +* [{{es}} secure settings](./secure-settings.md#elasticsearch) +* [{{kib}} secure settings](./secure-settings.md#kibana) + +## {{es}} secure settings [elasticsearch] -## The {{es}} keystore +The {{es}} keystore has some important characteristics and limitations to be aware of: -With the {{es}} keystore, you can add a key and its secret value, then use the key in place of the secret value when you configure your sensitive settings. +* **Only specific settings are allowed**: The keystore accepts only settings marked as *secure* in the [{{es}} configuration reference](elasticsearch://reference/elasticsearch/configuration-reference/index.md). Adding unsupported settings to the keystore causes the validation in the [_nodes/reload_secure_settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to fail and can also prevent {{es}} from starting. +* **Mandatory for secure settings**: Settings marked as secure must be added to the keystore. They cannot be placed in `elasticsearch.yml` or set via environment variables. This differs from the {{kib}} keystore, which supports all settings. +* **Changes require a restart**: Most secure settings take effect only after restarting the nodes. However, some are marked as *reloadable* and can be updated without a restart using the [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API. Refer to [Reloadable settings](#reload-secure-settings) for more information. +* **Keystore is per-node**: Each node in the cluster has its own keystore file. Secure settings must be specified individually on every node and must have the same values across the cluster. Orchestrated deployments, such as ECH, ECE, and ECK, handle this automatically when configuring secure settings. + +::::{tip} +To verify whether a setting can be added to the keystore and whether it supports reloading without a restart, look for the `Secure` and `Reloadable` qualifiers in the [{{es}} security settings reference](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md). For example, you can search for `secure_bind_password` on that page. +:::: + +The instructions below cover how to manage {{es}} keystore settings for each deployment type. :::::{tab-set} :group: deployment-type -::::{tab-item} {{ecloud}} +::::{tab-item} ECH and ECE :sync: cloud -There are three types of secrets that you can use: +You can manage {{es}} secure settings in the **Security > {{es}} keystore** section of your deployment page in the Elastic Cloud Console or ECE Cloud UI. -* **Single string** - Associate a secret value to a setting. -* **Multiple strings** - Associate multiple keys to multiple secret values. -* **JSON block/file** - Associate multiple keys to multiple secret values in JSON format. +:::{note} +Additionally, {{ece}} supports managing {{es}} keystore of your deployments through its [RESTful API](https://www.elastic.co/docs/api/doc/cloud-enterprise/). Refer to [Configure {{es}} keystore through ECE API](cloud://reference/cloud-enterprise/ece-restful-api-examples-configuring-keystore.md) for an example. +::: + +There are three input formats you can use for secure setting values: +* **Single string** - Associate a secret value to a setting. +* **Multiple strings** - A group of key-value pairs, each stored as part of the setting's value. +* **JSON block/file** - A structured JSON object containing multiple key-value pairs, typically used for more complex secrets like service account credentials. -**Add secret values** +**Add secure settings** -Add keys and secret values to the keystore. +Add settings and secret values to the keystore. -1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the **Hosted deployments** card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the **Deployments** page to view all of your deployments. +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body) or [ECE Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +2. Find your deployment and select **Manage**. On the **Deployments** page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. @@ -65,173 +84,100 @@ Add keys and secret values to the keystore. 4. Locate **{{es}} keystore** and select **Add settings**. 5. On the **Create setting** window, select the secret **Type**. 6. Configure the settings, then select **Save**. -7. All the modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. + :::{important} + All modifications to the non-reloadable settings take effect only after restarting {{es}}. [Reloadable](#reloadable-secure-settings) keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. Adding unsupported settings to the keystore will cause {{es}} to fail to start. + ::: -**Delete secret values** +**Delete secure settings** -When your keys and secret values are no longer needed, delete them from the keystore. +When your secure settings are no longer needed, delete them from the keystore. -1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the **Hosted deployments** card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the **Deployments** page to view all of your deployments. +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body) or [ECE Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +2. Find your deployment and select **Manage**. On the **Deployments** page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. 3. From your deployment menu, select **Security**. 4. From the **Existing keystores** list, use the delete icon next to the **Setting Name** that you want to delete. 5. On the **Confirm to delete** window, select **Confirm**. -6. All modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. - :::: -::::{tab-item} {{ece}} -:sync: ece - -There are three types of secrets that you can use: - -* **Single string** - Associate a secret value to a setting. -* **Multiple strings** - Associate multiple keys to multiple secret values. -* **JSON block/file** - Associate multiple keys to multiple secret values in JSON format. - +::::{tab-item} ECK +:sync: eck -**Add secret values** +In ECK, the operator simplifies secure settings configuration by relying on Kubernetes `Secrets`. -Add keys and secret values to the keystore. - -1. [Log into the Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the **Deployments** page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. From your deployment menu, select **Security**. -4. Locate **{{es}} keystore** and select **Add settings**. -5. On the **Create setting** window, select the secret **Type**. -6. Configure the settings, then select **Save**. -7. All the modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. - - - -**Delete secret values** +Refer to [Configure secure settings on ECK](./k8s-secure-settings.md) for details and examples. +:::: -When your keys and secret values are no longer needed, delete them from the keystore. +::::{tab-item} Self-managed +:sync: self-managed -1. [Log into the Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the **Deployments** page, select your deployment. +In self-managed deployments, you're responsible for configuring and maintaining the Elasticsearch keystore on each node individually. - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. +{{es}} provides the [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) command-line tool to help with this task. It allows you to list, add, remove, and update secure settings, as well as protect the keystore with a password if wanted. -3. From your deployment menu, select **Security**. -4. From the **Existing keystores** list, use the delete icon next to the **Setting Name** that you want to delete. -5. On the **Confirm to delete** window, select **Confirm**. -6. All modifications to the non-reloadable keystore take effect only after restarting {{es}}. Reloadable keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. +:::{important} +Secure settings must be specified on every node, and must have the same values across the cluster to ensure consistent behavior. -:::{dropdown} Using the API +Changes to the keystore take effect only after restarting {{es}}, except for [reloadable settings](#reloadable-secure-settings) that can be refreshed using the API. +::: -**Steps** +**Create the keystore** -Create the keystore: +To create the `elasticsearch.keystore`, use the `create` command: ```sh -curl -k -X PATCH -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/deployments/$DEPLOYMENT_ID/elasticsearch/$REF_ID/keystore \ -{ - "secrets": { - "s3.client.CLIENT_NAME.access_key": { - "as_file": false - "value": "ACCESS_KEY_VALUE" - } - "s3.client.CLIENT_NAME.secret_key": { - "value": "SECRET_KEY_VALUE" - } - } -} +bin/elasticsearch-keystore create -p ``` -`ELASTICSEARCH_CLUSTER_ID` -: The {{es}} cluster ID as shown in the Cloud UI or obtained through the API +You are prompted to enter the keystore password. A password-protected `elasticsearch.keystore` file is created alongside the `elasticsearch.yml` file. -List the keys defined in the keystore: +**List settings in the keystore** -```sh -{ - "secrets": { - "s3.client.CLIENT_NAME.access_key": { - "as_file": false - }, - "s3.client.CLIENT_NAME.secret_key": { - "as_file": false - } - } -} -``` - -Create the credentials for an S3 or Minio repository: +To list the settings in the keystore, use the `list` command. ```sh -curl -k -X PUT -H "Authorization: ApiKey $ECE_API_KEY" https://$COODINATOR_HOST:12443/api/v1/clusters/elasticsearch/$ELASTICSEARCH_CLUSTER_ID/_snapshot/s3-repo -{ - "type": "s3", - "settings": { - "bucket": "s3_REPOSITORY_NAME", - "client": "CLIENT_NAME", - "base_path": "PATH_NAME" - } -} +bin/elasticsearch-keystore list ``` +If the {{es}} keystore is password protected, you are prompted to enter the password. + +**Add secure settings to the keystore** -Create the credentials for a GCS repository: +Sensitive string settings, like authentication credentials for Cloud plugins, can be added with the `add` command: ```sh -curl -k -X PUT -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/clusters/elasticsearch/ELASTICSEARCH_CLUSTER_ID/_snapshot/s3-repo -{ - "type": "gcs", - "settings": { - "bucket": "BUCKET_NAME", - "base_path": "BASE_PATH_NAME", - "client": "CLIENT_NAME" - } -} +bin/elasticsearch-keystore add the.setting.name.to.set ``` -To use GCS snapshots, the cluster must have the `repository-gcs` plugin enabled. +You are prompted to enter the value of the setting. If the {{es}} keystore is password protected, you are also prompted to enter the password. +**Remove secure settings from the keystore** -Remove keys that are defined in the keystore: +To remove a setting from the keystore, use the `remove` command: ```sh -curl -k -X PATCH -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/deployments/$DEPLOYMENT_ID/elasticsearch/$REF_ID/keystore \ -{ - "secrets": { - "KEY_TO_REMOVE": {} - } -} +bin/elasticsearch-keystore remove the.setting.name.to.remove ``` +If the {{es}} keystore is password protected, you are prompted to enter the password. -**Verify your credentials** - -If your credentials are invalid, an administrator can verify that they are correct by checking the `keystore` field in the cluster metadata. +**Other examples** -If the credential values are correct, but do not work, the keystore file could be out of sync on one or more nodes. To sync the keystore file, update the value for the key by using the patch API to delete the key from keystore, then add it back again. - -::: +For a full command reference and additional examples, such as displaying stored values or adding entire files as setting values, refer to the [`elasticsearch-keystore` tool documentation](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md). :::: -::::{tab-item} Self-managed -:sync: self-managed - -All the modifications to the keystore take effect only after restarting {{es}}. - -These settings, just like the regular ones in the `elasticsearch.yml` config file, need to be specified on each node in the cluster. Currently, all secure settings are node-specific settings that must have the same value on every node. - +::::: -**Reloadable secure settings** +### Reloadable settings [reloadable-secure-settings] -Just like the settings values in `elasticsearch.yml`, changes to the keystore contents are not automatically applied to the running {{es}} node. Re-reading settings requires a node restart. However, certain secure settings are marked as **reloadable**. Such settings can be re-read and applied on a running node. +Just like the settings values in `elasticsearch.yml`, changes to the keystore contents are not automatically applied to the running {{es}} node. Re-reading settings requires a node restart. -You can define these settings before the node is started, or call the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) after the settings are defined to apply them to a running node. +However, certain secure settings are marked as **reloadable**. Such settings can be re-read and applied on a running node by using the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings). -The values of all secure settings, **reloadable** or not, must be identical across all cluster nodes. After making the desired secure settings changes, using the `bin/elasticsearch-keystore add` command, call: +This API should be called after having properly defined the keystore settings in all nodes of the cluster. The values of all secure settings, **reloadable** or not, must be identical across all cluster nodes. After making the desired secure settings changes, call: ```console POST _nodes/reload_secure_settings @@ -239,9 +185,7 @@ POST _nodes/reload_secure_settings "secure_settings_password": "keystore-password" <1> } ``` - -1. The password that the {{es}} keystore is encrypted with. - +1. If the {{es}} keystore is password-protected, include the `secure_settings_password` parameter in the request body with the keystore password. This API decrypts, re-reads the entire keystore and validates all settings on every cluster node, but only the **reloadable** secure settings are applied. Changes to other settings do not go into effect until the next restart. Once the call returns, the reload has been completed, meaning that all internal data structures dependent on these settings have been changed. Everything should look as if the settings had the new value from the start. @@ -260,23 +204,24 @@ There are reloadable secure settings for: * [LDAP realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-ldap-settings) * [Remote cluster credentials for the API key based security model](/deploy-manage/remote-clusters/remote-clusters-settings.md#remote-cluster-credentials-setting) - -:::: - -::::: - -## The {{kib}} keystore +## {{kib}} secure settings ```{applies_to} deployment: self: ga + eck: ga ``` -Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. For this use case, {{kib}} provides a keystore, and the `kibana-keystore` tool to manage the settings in the keystore. +{{kib}} supports secure settings through its own keystore, similar in nature to the {{es}} keystore. It provides a way to protect sensitive configuration values, such as authentication credentials or encryption keys, by storing them securely outside of `kibana.yml`. ::::{note} -* Run all commands as the user who runs {{kib}}. -* Any valid {{kib}} setting can be stored in the keystore securely. Unsupported, extraneous or invalid settings will cause {{kib}} to fail to start up. +Unlike with {{es}} keystore, any valid {{kib}} setting can be stored securely in the keystore. However, as with {{es}}, adding invalid, unsupported, or extraneous settings will cause Kibana to fail to start. Always ensure the setting exists and is properly formatted in the [configuration reference](kibana://reference/configuration-reference.md) before adding it to the keystore. +:::: + +This section provides examples of {{kib}} secure settings handling using the `kibana-keystore` command-line tool. +For ECK deployments, the configuration process is similar to {{es}} and is documented in the [{{kib}} secure settings section for ECK](./k8s-secure-settings.md#k8s-kibana-secure-settings). +::::{important} +In the following examples, run all commands as the user who runs {{kib}}. :::: ### Create the keystore [creating-keystore] From cad2956a67bad3223a744814264a3d87055a6b5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 3 Apr 2025 15:13:14 +0200 Subject: [PATCH 2/5] kibana anchor added --- deploy-manage/security/secure-settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/security/secure-settings.md b/deploy-manage/security/secure-settings.md index b6bfb0a3d2..6d6c587387 100644 --- a/deploy-manage/security/secure-settings.md +++ b/deploy-manage/security/secure-settings.md @@ -204,7 +204,7 @@ There are reloadable secure settings for: * [LDAP realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-ldap-settings) * [Remote cluster credentials for the API key based security model](/deploy-manage/remote-clusters/remote-clusters-settings.md#remote-cluster-credentials-setting) -## {{kib}} secure settings +## {{kib}} secure settings [kibana] ```{applies_to} deployment: self: ga From 8dcd11100e2a25b247baccda6e7c22eca2a1a200 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 3 Apr 2025 18:25:18 +0200 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> --- deploy-manage/security/secure-settings.md | 29 ++++++++++++----------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/deploy-manage/security/secure-settings.md b/deploy-manage/security/secure-settings.md index 6d6c587387..f2e35e1753 100644 --- a/deploy-manage/security/secure-settings.md +++ b/deploy-manage/security/secure-settings.md @@ -31,7 +31,7 @@ Keystore settings must be handled using a specific tool or method depending on t |----------------------|--------------------------------------------|----------------------------------------| | {{ech}} | Cloud UI | Not available | | {{ece}} | Cloud UI or API | Not available | -| {{eck}} | Kubernetes `secrets` | Kubernetes `secrets` | +| {{eck}} | Kubernetes secrets | Kubernetes secrets | | Self-managed | Manual configuration with [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) | Manual configuration with `kibana-keystore` | This section describes how to configure and manage secure settings in each keystore depending on the deployment model: @@ -42,9 +42,9 @@ This section describes how to configure and manage secure settings in each keyst The {{es}} keystore has some important characteristics and limitations to be aware of: -* **Only specific settings are allowed**: The keystore accepts only settings marked as *secure* in the [{{es}} configuration reference](elasticsearch://reference/elasticsearch/configuration-reference/index.md). Adding unsupported settings to the keystore causes the validation in the [_nodes/reload_secure_settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to fail and can also prevent {{es}} from starting. -* **Mandatory for secure settings**: Settings marked as secure must be added to the keystore. They cannot be placed in `elasticsearch.yml` or set via environment variables. This differs from the {{kib}} keystore, which supports all settings. -* **Changes require a restart**: Most secure settings take effect only after restarting the nodes. However, some are marked as *reloadable* and can be updated without a restart using the [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API. Refer to [Reloadable settings](#reload-secure-settings) for more information. +* **Only specific settings are allowed**: The keystore accepts only settings marked as *secure* in the [{{es}} configuration reference](elasticsearch://reference/elasticsearch/configuration-reference/index.md). Adding unsupported settings to the keystore causes the validation in the [`reload_secure_settings` API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to fail and can also prevent {{es}} from starting. +* **Mandatory for secure settings**: Settings marked as secure must be added to the keystore. They cannot be placed in `elasticsearch.yml` or set using environment variables. This differs from the {{kib}} keystore, which supports all settings. +* **Changes require a restart**: Most secure settings take effect only after restarting the nodes. However, some are marked as *reloadable* and can be updated without a restart using the [`reload_secure_settings`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API. Refer to [Reloadable settings](#reloadable-secure-settings) for more information. * **Keystore is per-node**: Each node in the cluster has its own keystore file. Secure settings must be specified individually on every node and must have the same values across the cluster. Orchestrated deployments, such as ECH, ECE, and ECK, handle this automatically when configuring secure settings. ::::{tip} @@ -62,14 +62,14 @@ The instructions below cover how to manage {{es}} keystore settings for each dep You can manage {{es}} secure settings in the **Security > {{es}} keystore** section of your deployment page in the Elastic Cloud Console or ECE Cloud UI. :::{note} -Additionally, {{ece}} supports managing {{es}} keystore of your deployments through its [RESTful API](https://www.elastic.co/docs/api/doc/cloud-enterprise/). Refer to [Configure {{es}} keystore through ECE API](cloud://reference/cloud-enterprise/ece-restful-api-examples-configuring-keystore.md) for an example. +{{ece}} also supports managing {{es}} keystore of your deployments through its [RESTful API](https://www.elastic.co/docs/api/doc/cloud-enterprise/). Refer to [Configure {{es}} keystore through ECE API](cloud://reference/cloud-enterprise/ece-restful-api-examples-configuring-keystore.md) for an example. ::: There are three input formats you can use for secure setting values: -* **Single string** - Associate a secret value to a setting. -* **Multiple strings** - A group of key-value pairs, each stored as part of the setting's value. -* **JSON block/file** - A structured JSON object containing multiple key-value pairs, typically used for more complex secrets like service account credentials. +* **Single string**: Associate a secret value to a setting. +* **Multiple strings**: A group of key-value pairs, each stored as part of the setting's value. +* **JSON block/file**: A structured JSON object containing multiple key-value pairs, typically used for more complex secrets like service account credentials. **Add secure settings** @@ -99,14 +99,14 @@ When your secure settings are no longer needed, delete them from the keystore. On the **Deployments** page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. 3. From your deployment menu, select **Security**. -4. From the **Existing keystores** list, use the delete icon next to the **Setting Name** that you want to delete. +4. From the **Existing keystores** list, use the delete icon next to setting that you want to delete. 5. On the **Confirm to delete** window, select **Confirm**. :::: ::::{tab-item} ECK :sync: eck -In ECK, the operator simplifies secure settings configuration by relying on Kubernetes `Secrets`. +In ECK, the operator simplifies secure settings configuration by relying on Kubernetes secrets. Refer to [Configure secure settings on ECK](./k8s-secure-settings.md) for details and examples. :::: @@ -126,7 +126,7 @@ Changes to the keystore take effect only after restarting {{es}}, except for [re **Create the keystore** -To create the `elasticsearch.keystore`, use the `create` command: +To create the Elasticsearch keystore, use the `create` command: ```sh bin/elasticsearch-keystore create -p @@ -177,7 +177,7 @@ Just like the settings values in `elasticsearch.yml`, changes to the keystore co However, certain secure settings are marked as **reloadable**. Such settings can be re-read and applied on a running node by using the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings). -This API should be called after having properly defined the keystore settings in all nodes of the cluster. The values of all secure settings, **reloadable** or not, must be identical across all cluster nodes. After making the desired secure settings changes, call: +This API should be called after having properly defined the keystore settings in all nodes of the cluster. The values of all secure settings, reloadable or not, must be identical across all cluster nodes. After making the desired secure settings changes, call the following endpoint: ```console POST _nodes/reload_secure_settings @@ -211,13 +211,14 @@ deployment: eck: ga ``` -{{kib}} supports secure settings through its own keystore, similar in nature to the {{es}} keystore. It provides a way to protect sensitive configuration values, such as authentication credentials or encryption keys, by storing them securely outside of `kibana.yml`. +{{kib}} supports secure settings through its own keystore, similar to the {{es}} keystore. It provides a way to protect sensitive configuration values, such as authentication credentials or encryption keys, by storing them securely outside of `kibana.yml`. ::::{note} Unlike with {{es}} keystore, any valid {{kib}} setting can be stored securely in the keystore. However, as with {{es}}, adding invalid, unsupported, or extraneous settings will cause Kibana to fail to start. Always ensure the setting exists and is properly formatted in the [configuration reference](kibana://reference/configuration-reference.md) before adding it to the keystore. :::: -This section provides examples of {{kib}} secure settings handling using the `kibana-keystore` command-line tool. +This section provides examples of {{kib}} secure settings handling using the `kibana-keystore` command-line tool for self-managed deployments. + For ECK deployments, the configuration process is similar to {{es}} and is documented in the [{{kib}} secure settings section for ECK](./k8s-secure-settings.md#k8s-kibana-secure-settings). ::::{important} From df7e18e3359c5e6e2c1f75891347b2e914e5c8ba Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 4 Apr 2025 12:10:18 +0200 Subject: [PATCH 4/5] reviews applied --- deploy-manage/security/secure-settings.md | 68 +++++++++++------------ 1 file changed, 31 insertions(+), 37 deletions(-) diff --git a/deploy-manage/security/secure-settings.md b/deploy-manage/security/secure-settings.md index f2e35e1753..f0e7ff1105 100644 --- a/deploy-manage/security/secure-settings.md +++ b/deploy-manage/security/secure-settings.md @@ -15,12 +15,6 @@ mapped_pages: # Secure your settings -$$$ec-add-secret-values$$$ - -$$$change-password$$$ - -$$$creating-keystore$$$ - Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. For this use case, {{es}} and {{kib}} provide secure keystores to store sensitive configuration values such as passwords, API keys, and tokens. Secure settings are often referred to as **keystore settings**, since they must be added to the product-specific keystore rather than the standard `elasticsearch.yml` or `kibana.yml` files. Unlike regular settings, they are encrypted and protected at rest, and they cannot be read or modified through the usual configuration files or environment variables. @@ -29,11 +23,16 @@ Keystore settings must be handled using a specific tool or method depending on t | Deployment Type | {{es}} keystore configuration | {{kib}} keystore configuration | |----------------------|--------------------------------------------|----------------------------------------| +| {{serverless-full}} | Not available | Not available | | {{ech}} | Cloud UI | Not available | | {{ece}} | Cloud UI or API | Not available | | {{eck}} | Kubernetes secrets | Kubernetes secrets | | Self-managed | Manual configuration with [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) | Manual configuration with `kibana-keystore` | +::::{note} +In {{serverless-full}} and {{ech}} all data is encrypted at rest already. +:::: + This section describes how to configure and manage secure settings in each keystore depending on the deployment model: * [{{es}} secure settings](./secure-settings.md#elasticsearch) * [{{kib}} secure settings](./secure-settings.md#kibana) @@ -48,7 +47,9 @@ The {{es}} keystore has some important characteristics and limitations to be awa * **Keystore is per-node**: Each node in the cluster has its own keystore file. Secure settings must be specified individually on every node and must have the same values across the cluster. Orchestrated deployments, such as ECH, ECE, and ECK, handle this automatically when configuring secure settings. ::::{tip} -To verify whether a setting can be added to the keystore and whether it supports reloading without a restart, look for the `Secure` and `Reloadable` qualifiers in the [{{es}} security settings reference](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md). For example, you can search for `secure_bind_password` on that page. +To check whether a setting can be added to the keystore and supports reloading without a restart, look for the `Secure` and `Reloadable` qualifiers in the [{{es}} configuration reference](elasticsearch://reference/elasticsearch/configuration-reference/index.md). + +For example, you can search for `secure_bind_password` in the [security settings document](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md) to see how these qualifiers appear in context. :::: The instructions below cover how to manage {{es}} keystore settings for each deployment type. @@ -58,7 +59,7 @@ The instructions below cover how to manage {{es}} keystore settings for each dep ::::{tab-item} ECH and ECE :sync: cloud - +% ### ECE and ECH You can manage {{es}} secure settings in the **Security > {{es}} keystore** section of your deployment page in the Elastic Cloud Console or ECE Cloud UI. :::{note} @@ -71,7 +72,7 @@ There are three input formats you can use for secure setting values: * **Multiple strings**: A group of key-value pairs, each stored as part of the setting's value. * **JSON block/file**: A structured JSON object containing multiple key-value pairs, typically used for more complex secrets like service account credentials. -**Add secure settings** +### Add secure settings [ec-add-secret-values] Add settings and secret values to the keystore. @@ -89,7 +90,7 @@ Add settings and secret values to the keystore. All modifications to the non-reloadable settings take effect only after restarting {{es}}. [Reloadable](#reloadable-secure-settings) keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. Adding unsupported settings to the keystore will cause {{es}} to fail to start. ::: -**Delete secure settings** +### Remove secure settings When your secure settings are no longer needed, delete them from the keystore. @@ -105,7 +106,7 @@ When your secure settings are no longer needed, delete them from the keystore. ::::{tab-item} ECK :sync: eck - +% ### ECK In ECK, the operator simplifies secure settings configuration by relying on Kubernetes secrets. Refer to [Configure secure settings on ECK](./k8s-secure-settings.md) for details and examples. @@ -113,7 +114,7 @@ Refer to [Configure secure settings on ECK](./k8s-secure-settings.md) for detail ::::{tab-item} Self-managed :sync: self-managed - +% ### Self-managed In self-managed deployments, you're responsible for configuring and maintaining the Elasticsearch keystore on each node individually. {{es}} provides the [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) command-line tool to help with this task. It allows you to list, add, remove, and update secure settings, as well as protect the keystore with a password if wanted. @@ -124,7 +125,7 @@ Secure settings must be specified on every node, and must have the same values a Changes to the keystore take effect only after restarting {{es}}, except for [reloadable settings](#reloadable-secure-settings) that can be refreshed using the API. ::: -**Create the keystore** +### Create the keystore [creating-keystore] To create the Elasticsearch keystore, use the `create` command: @@ -132,9 +133,11 @@ To create the Elasticsearch keystore, use the `create` command: bin/elasticsearch-keystore create -p ``` -You are prompted to enter the keystore password. A password-protected `elasticsearch.keystore` file is created alongside the `elasticsearch.yml` file. +You are prompted to enter a keystore password, but setting one is optional. If you press Enter without typing a password, the keystore file will be created without password protection. + +The command creates a file named `elasticsearch.keystore` alongside the `elasticsearch.yml` file. -**List settings in the keystore** +### List settings in the keystore To list the settings in the keystore, use the `list` command. @@ -143,7 +146,7 @@ bin/elasticsearch-keystore list ``` If the {{es}} keystore is password protected, you are prompted to enter the password. -**Add secure settings to the keystore** +### Add secure settings to the keystore Sensitive string settings, like authentication credentials for Cloud plugins, can be added with the `add` command: @@ -153,7 +156,7 @@ bin/elasticsearch-keystore add the.setting.name.to.set You are prompted to enter the value of the setting. If the {{es}} keystore is password protected, you are also prompted to enter the password. -**Remove secure settings from the keystore** +### Remove secure settings from the keystore To remove a setting from the keystore, use the `remove` command: @@ -163,7 +166,7 @@ bin/elasticsearch-keystore remove the.setting.name.to.remove If the {{es}} keystore is password protected, you are prompted to enter the password. -**Other examples** +### Other examples [change-password] For a full command reference and additional examples, such as displaying stored values or adding entire files as setting values, refer to the [`elasticsearch-keystore` tool documentation](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md). @@ -171,13 +174,13 @@ For a full command reference and additional examples, such as displaying stored ::::: -### Reloadable settings [reloadable-secure-settings] +### Reloadable secure settings [reloadable-secure-settings] Just like the settings values in `elasticsearch.yml`, changes to the keystore contents are not automatically applied to the running {{es}} node. Re-reading settings requires a node restart. -However, certain secure settings are marked as **reloadable**. Such settings can be re-read and applied on a running node by using the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings). +However, certain secure settings are marked as `Reloadable` in [{{es}} reference documentation](elasticsearch://reference/elasticsearch/configuration-reference/index.md). Such settings can be re-read and applied on a running node by using the [reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings). -This API should be called after having properly defined the keystore settings in all nodes of the cluster. The values of all secure settings, reloadable or not, must be identical across all cluster nodes. After making the desired secure settings changes, call the following endpoint: +The values of all secure settings, whether reloadable or not, must be identical across all nodes. After making the desired changes, call the following endpoint. If your keystore is password-protected, include the `secure_settings_password` parameter: ```console POST _nodes/reload_secure_settings @@ -185,24 +188,15 @@ POST _nodes/reload_secure_settings "secure_settings_password": "keystore-password" <1> } ``` -1. If the {{es}} keystore is password-protected, include the `secure_settings_password` parameter in the request body with the keystore password. +1. Only required if the {{es}} keystore is password-protected. -This API decrypts, re-reads the entire keystore and validates all settings on every cluster node, but only the **reloadable** secure settings are applied. Changes to other settings do not go into effect until the next restart. Once the call returns, the reload has been completed, meaning that all internal data structures dependent on these settings have been changed. Everything should look as if the settings had the new value from the start. +This API decrypts, re-reads the entire keystore and validates all settings on every cluster node, but only the reloadable secure settings are applied. Changes to other settings do not go into effect until the next restart. Once the call returns, the reload has been completed, meaning that all internal data structures dependent on these settings have been changed. Everything should look as if the settings had the new value from the start. -When changing multiple **reloadable** secure settings, modify all of them on each cluster node, then issue a [`reload_secure_settings`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) call instead of reloading after each modification. - -There are reloadable secure settings for: +::::{tip} +When changing multiple reloadable secure settings, modify all of them on each cluster node, then issue a [`reload_secure_settings`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) call instead of reloading after each modification. +:::: -* [The Azure repository plugin](/deploy-manage/tools/snapshot-and-restore/azure-repository.md) -* [The EC2 discovery plugin](elasticsearch://reference/elasticsearch-plugins/discovery-ec2-usage.md#_configuring_ec2_discovery) -* [The GCS repository plugin](/deploy-manage/tools/snapshot-and-restore/google-cloud-storage-repository.md) -* [The S3 repository plugin](/deploy-manage/tools/snapshot-and-restore/s3-repository.md) -* [Monitoring settings](elasticsearch://reference/elasticsearch/configuration-reference/monitoring-settings.md) -* [{{watcher}} settings](elasticsearch://reference/elasticsearch/configuration-reference/watcher-settings.md) -* [JWT realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-jwt-settings) -* [Active Directory realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-ad-settings) -* [LDAP realm](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#ref-ldap-settings) -* [Remote cluster credentials for the API key based security model](/deploy-manage/remote-clusters/remote-clusters-settings.md#remote-cluster-credentials-setting) +Reloadable secure settings are available for snapshot repository plugins, watcher, monitoring, and certain authentication realms. Refer to the relevant documentation for each feature to see if secure settings can be reloaded. ## {{kib}} secure settings [kibana] ```{applies_to} @@ -235,7 +229,7 @@ bin/kibana-keystore create The file `kibana.keystore` will be created in the `config` directory defined by the environment variable `KBN_PATH_CONF`. -To create a password protected keystore use the `--password` flag. +To create a password protected keystore use the `--password` flag. Setting a password is optional. ### List settings in the keystore [list-settings] From cec3dcfa658f782196cb4b1bc9cdbb8fb12812a8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 4 Apr 2025 17:35:08 +0200 Subject: [PATCH 5/5] headings within tabs updated --- deploy-manage/security/secure-settings.md | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/deploy-manage/security/secure-settings.md b/deploy-manage/security/secure-settings.md index f0e7ff1105..61ddf5d025 100644 --- a/deploy-manage/security/secure-settings.md +++ b/deploy-manage/security/secure-settings.md @@ -29,10 +29,6 @@ Keystore settings must be handled using a specific tool or method depending on t | {{eck}} | Kubernetes secrets | Kubernetes secrets | | Self-managed | Manual configuration with [`elasticsearch-keystore`](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md) | Manual configuration with `kibana-keystore` | -::::{note} -In {{serverless-full}} and {{ech}} all data is encrypted at rest already. -:::: - This section describes how to configure and manage secure settings in each keystore depending on the deployment model: * [{{es}} secure settings](./secure-settings.md#elasticsearch) * [{{kib}} secure settings](./secure-settings.md#kibana) @@ -72,7 +68,7 @@ There are three input formats you can use for secure setting values: * **Multiple strings**: A group of key-value pairs, each stored as part of the setting's value. * **JSON block/file**: A structured JSON object containing multiple key-value pairs, typically used for more complex secrets like service account credentials. -### Add secure settings [ec-add-secret-values] +#### Add secure settings [ec-add-secret-values] Add settings and secret values to the keystore. @@ -90,7 +86,7 @@ Add settings and secret values to the keystore. All modifications to the non-reloadable settings take effect only after restarting {{es}}. [Reloadable](#reloadable-secure-settings) keystore changes take effect after issuing a [reload_secure_settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API request. Adding unsupported settings to the keystore will cause {{es}} to fail to start. ::: -### Remove secure settings +#### Remove secure settings When your secure settings are no longer needed, delete them from the keystore. @@ -125,7 +121,7 @@ Secure settings must be specified on every node, and must have the same values a Changes to the keystore take effect only after restarting {{es}}, except for [reloadable settings](#reloadable-secure-settings) that can be refreshed using the API. ::: -### Create the keystore [creating-keystore] +#### Create the keystore [creating-keystore] To create the Elasticsearch keystore, use the `create` command: @@ -137,7 +133,7 @@ You are prompted to enter a keystore password, but setting one is optional. If y The command creates a file named `elasticsearch.keystore` alongside the `elasticsearch.yml` file. -### List settings in the keystore +#### List settings in the keystore To list the settings in the keystore, use the `list` command. @@ -146,7 +142,7 @@ bin/elasticsearch-keystore list ``` If the {{es}} keystore is password protected, you are prompted to enter the password. -### Add secure settings to the keystore +#### Add secure settings to the keystore Sensitive string settings, like authentication credentials for Cloud plugins, can be added with the `add` command: @@ -156,7 +152,7 @@ bin/elasticsearch-keystore add the.setting.name.to.set You are prompted to enter the value of the setting. If the {{es}} keystore is password protected, you are also prompted to enter the password. -### Remove secure settings from the keystore +#### Remove secure settings from the keystore To remove a setting from the keystore, use the `remove` command: @@ -166,7 +162,7 @@ bin/elasticsearch-keystore remove the.setting.name.to.remove If the {{es}} keystore is password protected, you are prompted to enter the password. -### Other examples [change-password] +#### Other examples [change-password] For a full command reference and additional examples, such as displaying stored values or adding entire files as setting values, refer to the [`elasticsearch-keystore` tool documentation](elasticsearch://reference/elasticsearch/command-line-tools/elasticsearch-keystore.md).