ECE and ECH to ECK remote clusters setup updated for API keys (#3849)

eedugon · shainaraskas · barkbay · web-flow · commit 826287426bf7 · 2025-11-26T14:08:00.000+01:00
This PR introduces the Remote Clusters -> API Key security model configuration for the following use cases: * ECH deployment to ECK-managed cluster * ECE deployment to ECE-managed cluster TLS certificate-based authentication method (deprecated) has been left untouched, except of a minor step to expose the transport service of the destination ECK cluster, which was missing. The procedure has been fully tested. All based on snippets as the content of both pages is almost identital. Relates to #1645 --------- Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> Co-authored-by: Michael Morello <michael.morello@elastic.co>
diff --git a/deploy-manage/remote-clusters/_snippets/apikeys-local-ece-remote-private.md b/deploy-manage/remote-clusters/_snippets/apikeys-local-ece-remote-private.md
@@ -2,6 +2,7 @@
 This snippet is in use in the following locations:
 - ece-remote-cluster-self-managed.md
 - ece-remote-cluster-other-ece.md
+- ece-enable-ccs-for-eck.md
 
 It requires remote_type substitution to be defined
 -->
@@ -10,7 +11,7 @@ It requires remote_type substitution to be defined
 
     Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
 
-3. Access the **Security** page of the deployment.
+3. From the navigation menu, select **Security**.
 4. Select **Remote Connections > Add trusted environment** and choose **{{remote_type}}**. Then click **Next**.
 5. Select **API keys** as authentication mechanism and click **Next**.
 6. When asked whether the Certificate Authority (CA) of the remote environment’s proxy or load-balancing infrastructure is public, select **No, it is private**.
@@ -21,13 +22,13 @@ It requires remote_type substitution to be defined
         * For the **Remote cluster name**, enter the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores.
         * For the **Cross-cluster API key**, paste the encoded cross-cluster API key.
 
-    2. Click **Add** to save the API key to the keystore.
+    2. Click **Add** to save the API key.
     3. Repeat these steps for each API key you want to add. For example, if you want to use several clusters of the remote environment for CCR or CCS.
 
 8. Add the CA certificate of the remote environment.
 9. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment's **Security** page.
 10. Select **Create trust** to complete the configuration.
-11. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart {{es}}**.
+11. Restart the local deployment to reload the new settings. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart {{es}}**.
 
     ::::{note}
     If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys.
diff --git a/deploy-manage/remote-clusters/_snippets/apikeys-local-ece-remote-public.md b/deploy-manage/remote-clusters/_snippets/apikeys-local-ece-remote-public.md
@@ -4,23 +4,24 @@ This snippet is in use in the following locations:
 - ece-remote-cluster-same-ece.md
 - ece-remote-cluster-other-ece.md
 - ece-remote-cluster-ece-ess.md
+- ece-enable-ccs-for-eck.md
 -->
 1. [Log into the Cloud UI](/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md).
 2. On the **Deployments** page, select your deployment.
 
     Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
 
-3. From the deployment menu, select **Security**.
+3. From the navigation menu, select **Security**.
 4. Locate **Remote Connections > Trust management > Connections using API keys** and select **Add API key**.
 
     1. Fill both fields.
 
-        * For the **Remote cluster name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores.
+        * For the **Remote cluster name**, enter the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores.
         * For the **Cross-cluster API key**, paste the encoded cross-cluster API key.
 
-    2. Click **Add** to save the API key to the keystore.
+    2. Click **Add** to save the API key.
 
-5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment's main page (named after your deployment's name), locate the **Actions** menu, and select **Restart {{es}}**.
+5. Restart the local deployment to reload the new setting. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart {{es}}**.
 
     ::::{note}
     If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys.
diff --git a/deploy-manage/remote-clusters/_snippets/apikeys-local-ech-remote-private.md b/deploy-manage/remote-clusters/_snippets/apikeys-local-ech-remote-private.md
@@ -2,6 +2,7 @@
 This snippet is in use in the following locations:
 - ec-remote-cluster-self-managed.md
 - ec-remote-cluster-ece.md
+- ec-enable-ccs-for-eck.md
 
 It requires remote_type substitution to be defined
 -->
@@ -21,13 +22,13 @@ It requires remote_type substitution to be defined
         * For the **Remote cluster name**, enter the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores.
         * For the **Cross-cluster API key**, paste the encoded cross-cluster API key.
 
-    2. Click **Add** to save the API key to the keystore.
+    2. Click **Add** to save the API key.
     3. Repeat these steps for each API key you want to add. For example, if you want to use several clusters of the remote environment for CCR or CCS.
 
 8. Add the CA certificate of the remote environment.
 9. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment's **Security** page.
 10. Select **Create trust** to complete the configuration.
-11. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart {{es}}**.
+11. Restart the local deployment to reload the new settings. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart {{es}}**.
 
     ::::{note}
     If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys.
diff --git a/deploy-manage/remote-clusters/_snippets/apikeys-local-ech-remote-public.md b/deploy-manage/remote-clusters/_snippets/apikeys-local-ech-remote-public.md
@@ -4,6 +4,8 @@ This snippet is in use in the following locations:
 - ec-remote-cluster-same-ess.md
 - ec-remote-cluster-other-ess.md
 - ec-remote-cluster-ece.md
+- ec-enable-ccs-for-eck.md
+
 -->
 1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body).
 2. On the home page, find your hosted deployment and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the **Hosted deployments** page to view all of your deployments.
@@ -15,12 +17,12 @@ This snippet is in use in the following locations:
 
     1. Fill both fields.
 
-        * For the **Remote cluster name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores.
+        * For the **Remote cluster name**, enter the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores.
         * For the **Cross-cluster API key**, paste the encoded cross-cluster API key.
 
     2. Click **Add** to save the API key.
 
-5. Restart the local deployment to reload the new setting. To do that, go to the deployment's main page (named after your deployment's name), locate the **Actions** menu, and select **Restart {{es}}**.
+5. Restart the local deployment to reload the new setting. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart {{es}}**.
 
     ::::{note}
     If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys.
diff --git a/deploy-manage/remote-clusters/_snippets/eck_expose_transport.md b/deploy-manage/remote-clusters/_snippets/eck_expose_transport.md
@@ -0,0 +1,15 @@
+Expose the transport service (defaults to port `9300`) of your ECK cluster to allow external {{es}} clusters to connect:
+
+```yaml
+apiVersion: elasticsearch.k8s.elastic.co/v1
+kind: Elasticsearch
+metadata:
+  name: <cluster-name>
+spec:
+  transport:
+    service:
+      spec:
+        type: LoadBalancer <1>
+```
+
+1. On cloud providers which support external load balancers, setting the type field to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `<cluster-name>-es-transport` through one of the Kubernetes Ingress controllers that support TCP services.
diff --git a/deploy-manage/remote-clusters/_snippets/eck_rcs_connect_intro.md b/deploy-manage/remote-clusters/_snippets/eck_rcs_connect_intro.md
@@ -0,0 +1,5 @@
+On the local deployment, add the remote ECK cluster using {{kib}} or the {{es}} API with the following connection settings:
+
+* **Remote address**: Use the FQDN or IP address of the LoadBalancer service, or similar resource, you created to expose the remote cluster server interface (for API key-based authentication) or the transport interface (for TLS certificate-based authentication).
+
+* **TLS server name**: You can try leaving this field empty first. If the connection fails, and your environment is presenting the ECK-managed certificates during the TLS handshake, use `<cluster-name>-es-remote-cluster.<namespace>.svc` as the server name. For example, for a cluster named `quickstart` in the `default` namespace, use `quickstart-es-remote-cluster.default.svc`.
diff --git a/deploy-manage/remote-clusters/_snippets/eck_rcs_enable.md b/deploy-manage/remote-clusters/_snippets/eck_rcs_enable.md
@@ -0,0 +1,23 @@
+By default, the remote cluster server interface is deactivated on ECK-managed clusters. To use the API key–based security model for cross-cluster connections, you must first enable it on the remote {{es}} cluster:
+
+```yaml subs=true
+apiVersion: elasticsearch.k8s.elastic.co/v1
+kind: Elasticsearch
+metadata:
+  name: <cluster-name>
+  namespace: <namespace>
+spec:
+  version: {{version.stack}}
+  remoteClusterServer:
+    enabled: true
+  nodeSets:
+    - name: default
+      count: 3
+      ...
+      ...
+```
+
+::::{note}
+Enabling the remote cluster server triggers a restart of the {{es}} cluster.
+::::
+
diff --git a/deploy-manage/remote-clusters/_snippets/eck_rcs_expose.md b/deploy-manage/remote-clusters/_snippets/eck_rcs_expose.md
@@ -0,0 +1,52 @@
+When the remote cluster server is enabled, ECK automatically creates a Kubernetes service named `<cluster-name>-es-remote-cluster` that exposes the server internally on port `9443`.
+
+To allow clusters running outside your Kubernetes environment to connect to this {{es}} cluster, you must expose this service externally. The way to expose this service depends on your ECK version.
+
+:::::{applies-switch}
+
+::::{applies-item} eck: ga 3.3
+You can customize how the remote cluster service is exposed by overriding its service specification directly under `spec.remoteClusterServer.service` in the {{es}} resource. By default, this service listens on port 9443.
+
+```yaml
+apiVersion: elasticsearch.k8s.elastic.co/v1
+kind: Elasticsearch
+metadata:
+  name: <cluster-name>
+  namespace: <namespace>
+spec:
+  version: 9.2.1
+  remoteClusterServer:
+    enabled: true
+    service:
+      spec:
+        type: LoadBalancer <1>
+  nodeSets:
+    - name: default
+      count: 3
+      ...
+      ...
+```
+1. On cloud providers that support external load balancers, setting the type to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `<cluster-name>-es-remote-cluster` through one of the Kubernetes Ingress controllers that support TCP services.
+::::
+
+::::{applies-item} eck: ga 3.0
+
+In ECK 3.2 and earlier, you can't customize the service that ECK generates for the remote cluster interface, but you can create your own `LoadBalancer` service, `Ingress` object, or use another method available in your environment.
+
+For example, for a cluster named `quickstart`, the following command creates a separate `LoadBalancer` service named `quickstart-es-remote-cluster-lb`, pointing to the ECK-managed service `quickstart-es-remote-cluster`:
+
+```sh
+kubectl expose service quickstart-es-remote-cluster \
+  --name=quickstart-es-remote-cluster-lb \
+  --type=LoadBalancer \ <1>
+  --port=9443 --target-port=9443
+```
+1. On cloud providers that support external load balancers, setting the type to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `<cluster-name>-es-remote-cluster` through one of the Kubernetes Ingress controllers that support TCP services.
+
+::::
+:::::
+
+:::{warning}
+If you change the service’s `port`, set `targetPort` explicitly to `9443`, which is the default remote cluster server listening port. Otherwise, Kubernetes uses the same value for both fields, resulting in failed connections.
+:::
+
diff --git a/deploy-manage/remote-clusters/_snippets/eck_rcs_retrieve_ca.md b/deploy-manage/remote-clusters/_snippets/eck_rcs_retrieve_ca.md
@@ -0,0 +1,17 @@
+The certificate authority (CA) used by ECK to issue certificates for the remote cluster server interface is stored in the `ca.crt` key of the secret named `<cluster_name>-es-transport-certs-public`.
+
+If the external connections reach the {{es}} Pods on port `9443` without any intermediate TLS termination, you need to retrieve this CA because it is required in the local cluster configuration to establish trust.
+
+If TLS is terminated by any intermediate component and the certificate presented is not the ECK-managed one, use the CA associated with that component, or omit the CA entirely if it uses a publicly trusted certificate.
+
+To save the transport CA certificate of a cluster named `quickstart` into a local file, run the following command:
+
+```sh
+kubectl get secret quickstart-es-transport-certs-public \
+-o go-template='{{index .data "ca.crt" | base64decode}}' > eck_transport_ca.crt
+```
+
+::::{important}
+ECK-managed CA certificates are automatically rotated after one year by default, but you can [configure](/deploy-manage/deploy/cloud-on-k8s/configure-eck.md) a different validity period. When the CA certificate is rotated, ensure that this CA is updated in all environments where it's used to preserve trust.
+::::
+
diff --git a/deploy-manage/remote-clusters/_snippets/rcs-elasticsearch-api-snippet-self.md b/deploy-manage/remote-clusters/_snippets/rcs-elasticsearch-api-snippet-self.md
@@ -2,6 +2,8 @@
 This snippet is in use in the following locations:
 - ece-remote-cluster-self-managed.md
 - ec-remote-cluster-self-managed.md
+- ece-enable-ccs-for-eck.md
+- ec-enable-ccs-for-eck.md
 -->
 To add a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields:
 
diff --git a/deploy-manage/remote-clusters/_snippets/rcs-kibana-api-snippet-self.md b/deploy-manage/remote-clusters/_snippets/rcs-kibana-api-snippet-self.md
@@ -2,6 +2,8 @@
 This snippet is in use in the following locations:
 - ece-remote-cluster-self-managed.md
 - ec-remote-cluster-self-managed.md
+- ece-enable-ccs-for-eck.md
+- ec-enable-ccs-for-eck.md
 -->
 1. Go to the **Remote Clusters** management page in the navigation menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).
 2. Select **Add a remote cluster**.
diff --git a/deploy-manage/remote-clusters/ec-enable-ccs-for-eck.md b/deploy-manage/remote-clusters/ec-enable-ccs-for-eck.md
diff --git a/deploy-manage/remote-clusters/ece-enable-ccs-for-eck.md b/deploy-manage/remote-clusters/ece-enable-ccs-for-eck.md
