[Remote Clusters] Connection modes and Security models presented in their own pages for all deployment types (#3151)

This PR presents RCS connection modes (proxy & sniff) and security
models (TLS certificates & API key) properly within Remote Clusters
sections, as currently the detailed information was only available in
the self-managed configuration guide for remote clusters.

Closes #2064

---------

Co-authored-by: shainaraskas <[email protected]>

Author: eedugon

deploy-manage/remote-clusters.md

@@ -24,6 +24,17 @@ Remote clusters are especially useful in two cases:
 In the case of remote clusters, the {{es}} cluster or deployment initiating the connection and requests is often referred to as the **local cluster**, while the {{es}} cluster or deployment receiving the requests is referred to as the **remote cluster**.
 ::::
 
+## Security models and connection modes
+
+When configuring remote clusters, you can choose between two security models and two connection modes. Both security models are compatible with either connection mode.
+
+- [Security models](./remote-clusters/security-models.md): API key–based authentication (recommended) or TLS certificate–based authentication (deprecated).
+- [Connection modes](./remote-clusters/connection-modes.md): Sniff mode (direct connections to {{es}} nodes) or proxy mode (connections through a reverse proxy or load balancer endpoint).
+
+::::{note}
+In managed or orchestrated environments, such as {{ech}}, {{ece}}, and {{eck}}, you can select the security model, but the connection mode is effectively limited to *proxy*. This is because sniff mode requires {{es}} nodes publish addresses to be directly reachable across clusters, which is generally not practical in containerized deployments.
+::::
+
 ## Setup
 
 Depending on the environment the local and remote clusters are deployed on and the security model you wish to use, the exact details needed to add a remote cluster vary but generally follow the same path:

deploy-manage/remote-clusters/connection-modes.md

@@ -0,0 +1,44 @@
+---
+navigation_title: Connection modes
+applies_to:
+  stack: ga
+  serverless: unavailable
+products:
+  - id: elasticsearch
+---
+# Remote cluster connection modes
+
+When you configure a remote cluster, the local cluster needs a way to connect to the nodes of the remote cluster. {{es}} supports two connection modes to handle different network architectures:
+
+- **Proxy mode**: The local cluster connects through a reverse proxy or load balancer, which forwards traffic to the appropriate nodes in the remote cluster. You can configure this mode using either the {{kib}} UI or the {{es}} API.
+- **Sniff mode**: The local cluster discovers the remote cluster’s gateway nodes and connects to them directly. This mode can only be configured using the {{es}} API.
+
+::::{note}
+Connection modes work independently of [security models](./security-models.md). Both connection modes are compatible with either security model.
+::::
+
+The choice between proxy and sniff mode depends on your network architecture and deployment type.  
+
+- **Self-managed clusters:** If direct connections on the publish addresses between {{es}} nodes in both clusters are possible, you can use sniff mode. If direct connectivity is difficult to implement—for example, when clusters are separated by NAT, firewalls, or containerized environments—you can place a reverse proxy or load balancer in front of the remote cluster and use proxy mode instead.  
+
+- **Managed environments ({{ece}}, {{ech}}, {{eck}}):** Direct node-to-node connectivity is generally not feasible, so these deployments always rely on the proxy connection mode.
+
+The following sections describe each method in more detail.
+
+## Proxy mode
+
+In proxy mode, a cluster alias is registered with a name of your choosing and the address of a TCP (layer 4) reverse proxy specified with the `cluster.remote.<cluster_alias>.proxy_address` setting. You must configure this proxy to route connections to one or more nodes of the remote cluster. The service port to forward traffic to depends on the [security model](./security-models.md) in use, as each model uses a different service port.
+
+When you register a remote cluster using proxy mode, {{es}} opens several TCP connections to the proxy address and uses these connections to communicate with the remote cluster. In proxy mode, {{es}} disregards the publish addresses of the remote cluster nodes, which means that the publish addresses of the remote cluster nodes do not need to be accessible to the local cluster.
+
+Proxy mode is not the default connection mode when adding remotes using the {{es}} API, so you must set `cluster.remote.<cluster_alias>.mode: proxy` to use it. Refer to [Proxy mode remote cluster settings](elasticsearch://reference/elasticsearch/configuration-reference/remote-clusters.md#remote-cluster-proxy-settings) for more information about configuring proxy mode.
+
+::::{note}
+Remote clusters configured through **{{kib}}** support only proxy mode. You can’t select a connection mode or configure sniff mode from the UI.
+::::
+
+## Sniff mode
+
+In sniff mode, a cluster alias is registered with a name of your choosing and a list of addresses of *seed* nodes specified with the `cluster.remote.<cluster_alias>.seeds` setting. When you register a remote cluster using sniff mode, {{es}} retrieves from one of the seed nodes the addresses of up to three *gateway nodes*. Each `remote_cluster_client` node in the local {{es}} cluster then opens several TCP connections to the publish addresses of the gateway nodes. This mode therefore requires that the gateway nodes' publish addresses are accessible to nodes in the local cluster.
+
+Sniff mode is the default connection mode when adding a remote cluster through the {{es}} API. Refer to [Sniff mode remote cluster settings](elasticsearch://reference/elasticsearch/configuration-reference/remote-clusters.md#remote-cluster-sniff-settings) for more information about configuring sniff mode.

deploy-manage/remote-clusters/security-models.md

@@ -0,0 +1,70 @@
+---
+navigation_title: Security models
+applies_to:
+  stack: ga
+  serverless: unavailable
+products:
+  - id: elasticsearch
+---
+# Remote cluster security models
+
+Remote cluster security models determine how authentication and authorization works between clusters. {{es}} has evolved from a TLS certificate–based model, relying on mutual TLS authentication over the transport interface and duplicated roles across clusters, to a more flexible API key–based model that uses a dedicated service endpoint and supports fine-grained authorization on both local and remote clusters.
+
+TLS certificate–based authentication is now deprecated, and users are encouraged to migrate to the API key–based model.
+
+The following sections describe each model in more detail.
+
+::::{tip}
+Security models work independently of [connection modes](./connection-modes.md). Both security models are compatible with either connection mode.
+::::
+
+## API key authentication [api-key]
+
+API key authentication enables a local cluster to authenticate itself with a remote cluster using a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key, and grants access based on the API key’s privileges.
+
+### Authorization
+
+With this security model, authorization is enforced jointly by the local and remote cluster, as follows:
+
+* All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key.
+
+* On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. It is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key.
+
+  ::::{tip}
+  To configure fine-grained authorization for remote resources, use the `remote_indices` and `remote_clusters` fields in [role definitions](/deploy-manage/users-roles/cluster-or-deployment-auth/role-structure.md).
+  ::::
+
+### Connection details
+
+In this model, cross-cluster operations use [a dedicated server port](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md#remote_cluster.port), referred to as the remote cluster interface, for communication between clusters. The default port is `9443`. The remote cluster must enable this port for local clusters to connect.
+
+From a TLS perspective, the local cluster must trust the remote cluster on the remote cluster interface. This means the local cluster must trust the certificate authority (CA) that signs the server certificate used by the remote cluster interface. When establishing a connection, all nodes from the local cluster that participate in cross-cluster communication verify certificates from nodes on the other side, based on the TLS trust configuration.
+
+Mutual TLS is not required in this model.
+
+### Setup
+
+Refer to [Remote cluster setup](../remote-clusters.md#setup) for configuration guidance across all deployment types.
+
+## TLS certificate authentication
+```{applies_to}
+stack: deprecated 9.0
+```
+
+TLS certificate authentication requires all connected clusters to trust one another and be mutually authenticated with TLS on the {{es}} transport interface (default port `9300` in self-managed, `9400` in {{ece}} and {{ech}}). This means that the local cluster trusts the certificate authority (CA) of the remote cluster, and the remote cluster trusts the CA of the local cluster. When establishing a connection, all nodes will verify certificates from nodes on the other side. This mutual trust is required to securely connect a remote cluster, because all connected nodes effectively form a single security domain.
+
+### Authorization
+
+With this security model, user authentication is performed on the local cluster, and the user and user’s roles names are passed to the remote cluster. A remote cluster checks the user’s role names against its local role definitions to determine which indices the user is allowed to access. This requires at least a role existing in the remote cluster with the same name as the local cluster for the user to gain privileges.
+
+::::{warning}
+In this model, a superuser on the local cluster gains total read access to the remote cluster, so it is only suitable for clusters that are in the same security domain.
+::::
+
+### Connection details
+
+The local cluster uses the [transport interface](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md) to establish communication with remote clusters. The coordinating nodes in the local cluster establish [long-lived](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md#long-lived-connections) TCP connections with specific nodes in the remote cluster. These connections must remain open, even if the connections are idle for an extended period.
+
+### Setup
+
+Refer to [Remote cluster setup](../remote-clusters.md#setup) for configuration guidance across all deployment types.

deploy-manage/toc.yml

@@ -633,6 +633,8 @@ toc:
   - file: manage-connectors.md
   - file: remote-clusters.md
     children:
+      - file: remote-clusters/security-models.md
+      - file: remote-clusters/connection-modes.md
       - file: remote-clusters/ec-enable-ccs.md
         children:
           - file: remote-clusters/ec-remote-cluster-same-ess.md