diff --git a/reference/fleet/remote-elasticsearch-output.md b/reference/fleet/remote-elasticsearch-output.md index 2e7bbf10f4..cc22d9955b 100644 --- a/reference/fleet/remote-elasticsearch-output.md +++ b/reference/fleet/remote-elasticsearch-output.md @@ -5,7 +5,7 @@ mapped_pages: # Remote Elasticsearch output [remote-elasticsearch-output] -Beginning in version 8.12.0, you can send {{agent}} data to a remote {{es}} cluster. This is especially useful for data that you want to keep separate and independent from the deployment where you use {{fleet}} to manage the agents. +Remote {{es}} outputs allow you to send {{agent}} data to a remote {{es}} cluster. This is especially useful for data that you want to keep separate and independent from the deployment where you use {{fleet}} to manage the agents. A remote {{es}} cluster supports the same [output settings](/reference/fleet/es-output-settings.md) as your main {{es}} cluster. @@ -18,6 +18,7 @@ A bug has been found that causes {{elastic-defend}} response actions to stop wor Using a remote {{es}} output with a target cluster that has [traffic filters](/deploy-manage/security/traffic-filtering.md) enabled is not currently supported. :::: +## Configuration To configure a remote {{es}} cluster for your {{agent}} data: @@ -33,7 +34,7 @@ To configure a remote {{es}} cluster for your {{agent}} data: 5. Create a service token to access the remote cluster. 1. Below the **Service Token** field, copy the API request. - 2. In the remote cluster, open the {{kib}} menu and go to **Management → Dev Tools**. + 2. In the remote cluster, open the {{kib}} menu and go to **Management > Dev Tools**. 3. Run the API request. 4. Copy the value for the generated token. 5. Back in your main cluster, paste the value you copied into the output **Service Token** field. @@ -42,12 +43,13 @@ To configure a remote {{es}} cluster for your {{agent}} data: To prevent unauthorized access the {{es}} Service Token is stored as a secret value. While secret storage is recommended, you can choose to override this setting and store the password as plain text in the agent policy definition. Secret storage requires {{fleet-server}} version 8.12 or higher. This setting can also be stored as a secret value or as plain text for preconfigured outputs. See [Preconfiguration settings](kibana://reference/configuration-reference/fleet-settings.md#_preconfiguration_settings_for_advanced_use_cases) in the {{kib}} Guide to learn more. :::: -6. Choose whether or not the remote output should be the default for agent integrations or for agent monitoring data. When set, {{agent}}s use this output to send data if no other output is set in the [agent policy](/reference/fleet/agent-policy.md). -7. Select which [performance tuning settings](/reference/fleet/es-output-settings.md#es-output-settings-performance-tuning-settings) you’d prefer in order to optimize {{agent}} for throughput, scale, or latency, or leave the default `balanced` setting. -8. Add any [advanced YAML configuration settings](/reference/fleet/es-output-settings.md#es-output-settings-yaml-config) that you’d like for the output. -9. Click **Save and apply settings**. +6. Choose whether integrations should automatically be synchronized on the remote {{es}} cluster. Refer to [Automatic integrations synchronization](#automatic-integrations-synchronization) below to configure this feature. +7. Choose whether or not the remote output should be the default for agent integrations or for agent monitoring data. When set, {{agents}} use this output to send data if no other output is set in the [agent policy](/reference/fleet/agent-policy.md). +8. Select which [performance tuning settings](/reference/fleet/es-output-settings.md#es-output-settings-performance-tuning-settings) you’d prefer in order to optimize {{agent}} for throughput, scale, or latency, or leave the default `balanced` setting. +9. Add any [advanced YAML configuration settings](/reference/fleet/es-output-settings.md#es-output-settings-yaml-config) that you’d like for the output. +10. Click **Save and apply settings**. -After the output is created, you can update an {{agent}} policy to use the new remote {{es}} cluster: +After the output is created, you can update an {{agent}} policy to use the new output and send data to the remote {{es}} cluster: 1. In {{fleet}}, open the **Agent policies** tab. 2. Click the agent policy to edit it, then click **Settings**. @@ -57,4 +59,38 @@ After the output is created, you can update an {{agent}} policy to use the new r The remote {{es}} cluster is now configured. -As a final step before using the remote {{es}} output, you need to make sure that for any integrations that have been [added to your {{agent}} policy](/reference/fleet/add-integration-to-policy.md), the integration assets have been installed on the remote {{es}} cluster. Refer to [Install and uninstall {{agent}} integration assets](/reference/fleet/install-uninstall-integration-assets.md) for the steps. +If you have chosen not to automatically synchronize integrations, you need to make sure that for any integrations that have been [added to your {{agent}} policy](/reference/fleet/add-integration-to-policy.md), the integration assets have been installed on the remote {{es}} cluster. Refer to [Install and uninstall {{agent}} integration assets](/reference/fleet/install-uninstall-integration-assets.md) for the steps. + +## Automatic integrations synchronization + +```{applies_to} +stack: ga 9.1 +``` + +When enabled, this feature keeps integrations synced between your main {{es}} cluster and remote {{es}} clusters. + +### Requirements + +This feature requires setting up [{{ccr}}](/deploy-manage/tools/cross-cluster-replication.md), which is available to Platinum and Enterprise [subscriptions](https://www.elastic.co/subscriptions). Remote clusters must be running the same version of {{es}} as the main cluster or a newer version that is compatible with {{ccr}}. + +Remote clusters require access to the [{{package-registry}}](/reference/fleet/index.md#package-registry-intro) to install integrations. + +### Configuration + +1. Configure {{ccr}} on the remote cluster. + + 1. In the remote cluster, open the {{kib}} menu and go to **Stack Management > Remote Clusters**. + 2. Refer to [Remote clusters](https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters.html) to add your main cluster (where the remote {{es}} output is configured) as a remote cluster. + 3. Go to **Stack Management > Cross-Cluster Replication**. + 4. Create a follower index named `fleet-synced-integrations-ccr-` that replicates the `fleet-synced-integrations` leader index on the main cluster. + 5. Resume replication once the follower index is created. + +2. In the main cluster, in the **Remote Kibana URL** field, add the Kibana URL of the remote cluster. + +3. Create an API key to access Kibana on the remote cluster. + + 1. Below the **Remote Kibana API Key** field, copy the API request. + 2. In the remote cluster, open the {{kib}} menu and go to **Management > Dev Tools**. + 3. Run the API request. + 4. Copy the encoded value of the generated API key. + 5. Back in the main cluster, paste the value you copied into the **Remote Kibana API Key** field.