diff --git a/reference/fleet/_snippets/agent_add_helm_repository.md b/reference/fleet/_snippets/agent_add_helm_repository.md new file mode 100644 index 0000000000..6c6e43f832 --- /dev/null +++ b/reference/fleet/_snippets/agent_add_helm_repository.md @@ -0,0 +1,22 @@ +Before installing, add the Elastic Helm repository to your local Helm configuration and verify the available versions of the `elastic-agent` chart. If the repository is already configured, run `helm repo update` to ensure you have the latest package information. + +1. Set up Helm repository: + + ```sh + helm repo add elastic https://helm.elastic.co/ + helm repo update + ``` + +2. Verify chart versions: + + ```sh + helm search repo elastic-agent --versions + ``` + + The previous command returns something similar to: + + ```sh + NAME CHART VERSION APP VERSION DESCRIPTION + elastic/elastic-agent 9.0.0 9.0.0 Elastic-Agent Helm Chart + elastic/elastic-agent 8.18.0 8.18.0 Elastic-Agent Helm Chart + ``` diff --git a/reference/fleet/example-kubernetes-fleet-managed-agent-helm.md b/reference/fleet/example-kubernetes-fleet-managed-agent-helm.md index 28ed264d96..7dd89915de 100644 --- a/reference/fleet/example-kubernetes-fleet-managed-agent-helm.md +++ b/reference/fleet/example-kubernetes-fleet-managed-agent-helm.md @@ -1,84 +1,118 @@ --- mapped_pages: - https://www.elastic.co/guide/en/fleet/current/example-kubernetes-fleet-managed-agent-helm.html +applies_to: + stack: ga 8.18 + serverless: all --- # Example: Install Fleet-managed Elastic Agent on Kubernetes using Helm [example-kubernetes-fleet-managed-agent-helm] -::::{warning} -This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. -:::: - +This example demonstrates how to install a {{fleet}}-managed {{agent}} on a {{k8s}} system using a Helm chart, collect {{k8s}} metrics and logs using the [Kubernetes Integration](integration-docs://reference/kubernetes/index.md), and send the data to an {{es}} cluster in {{ecloud}} for visualization in {{kib}}. -This example demonstrates how to install {{fleet}}-managed {{agent}} on a {{k8s}} system using a Helm chart, gather {{k8s}} metrics and send them to an {{es}} cluster in {{ecloud}}, and then view visualizations of those metrics in {{kib}}. +Although this tutorial uses an {{ech}} deployment, the same steps can be adapted for other deployment types. For self-managed, {{eck}}, or {{ece}} deployments, you might need to provide the {{fleet}} Server CA certificate during the {{agent}} installation, as outlined below. For an overview of the {{agent}} Helm chart and its benefits, refer to [Install {{agent}} on Kubernetes using Helm](/reference/fleet/install-on-kubernetes-using-helm.md). This guide takes you through these steps: -* [Install {{agent}}](#agent-fleet-managed-helm-example-install-agent) +* [Add the Elastic Helm repository](#preparations) +* [Create a {{fleet}} policy and install {{agent}}](#agent-fleet-managed-helm-example-install-agent) * [Install the Kubernetes integration](#agent-fleet-managed-helm-example-install-integration) * [Tidy up](#agent-fleet-managed-helm-example-tidy-up) - ## Prerequisites [agent-fleet-managed-helm-example-prereqs] To get started, you need: * A local install of the [Helm](https://helm.sh/) {{k8s}} package manager. -* An [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body) hosted {{es}} cluster on version 8.16 or higher. +* An [{{ech}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body) {{es}} cluster on version 8.18 or higher, with an [Integrations Server](/deploy-manage/deploy/elastic-cloud/ec-customize-deployment-components.md#ec_integrations_server) component. An {{serverless-full}} project also meets this requirement. * An active {{k8s}} cluster. -* A local clone of the [elastic/elastic-agent](https://github.com/elastic/elastic-agent/tree/8.16) GitHub repository. Make sure to use the `8.16` branch to ensure that {{agent}} has full compatibility with the Helm chart. +## Installation overview [overview] + +The installation and configuration steps shown in this example deploys the following components to monitor your Kubernetes cluster: + +* A default installation of [`kube-state-metrics` (KSM)](https://github.com/kubernetes/kube-state-metrics), configured as a dependency of the Helm chart. KSM is required by the Kubernetes integration to collect cluster-level metrics. + +* A group of {{agent}}s deployed as a [Kubernetes DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/), connected to {{fleet}}, and configured through a {{fleet}} policy to collect the following metrics and logs: + * Host-level metrics and logs through the [System integration](integration-docs://reference/system/index.md): This enables the monitoring of your Kubernetes nodes at OS level. {{agent}} Pods will collect system metrics and logs from their own hosts. + * Kubernetes metrics and logs through the [Kubernetes integration](integration-docs://reference/kubernetes/index.md): This enables Kubernetes monitoring at both cluster and node levels. All {{agent}} Pods will collect node-level Kubernetes metrics and logs from their own hosts, while one of the agents will also collect cluster-level metrics and events, acting as a [leader](./kubernetes_leaderelection-provider.md). + +By default, all resources are installed in the namespace defined by your current `kubectl` context. You can override this by specifying a different namespace using the `--namespace` option during installation. + +::::{note} +The proposed approach of a single {{agent}} DaemonSet to collect all metrics works well for small to medium-sized {{k8s}} clusters. + +For larger clusters, or when kube-state-metrics (KSM) metrics collection becomes a performance bottleneck, we recommend a more scalable architecture: move the KSM metric collection to a separate set of agents deployed as sidecars alongside KSM, with autosharding enabled. + +This can be easily implemented with the Helm chart. For details, refer to the [KSM autosharding example](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-ksm-sharding). +:::: + + +% we will uncomment the next line when the use cases are documented in the landing page :) +% For other architectures and use cases, refer to [Advanced use cases](./install-on-kubernetes-using-helm.md#advanced-use-cases). + +## Step 1: Add the Elastic Helm repository [preparations] -## Install {{agent}} [agent-fleet-managed-helm-example-install-agent] +:::{include} _snippets/agent_add_helm_repository.md +::: + +## Step 2: Create a {{fleet}} policy and install {{agent}} [agent-fleet-managed-helm-example-install-agent] 1. Open your {{ecloud}} deployment, and from the navigation menu select **Fleet**. 2. From the **Agents** tab, select **Add agent**. 3. In the **Add agent** UI, specify a policy name and select **Create policy**. Leave the **Collect system logs and metrics** option selected. 4. Scroll down in the **Add agent** flyout to the **Install Elastic Agent on your host** section. 5. Select the **Linux TAR** tab and copy the values for `url` and `enrollment-token`. You’ll use these when you run the `helm install` command. -6. Open a terminal shell and change into a directory in your local clone of the `elastic-agent` repo. -7. Copy this command. +6. Open a terminal shell on your local system where the Helm tool is installed and you have access to the {{k8s}} cluster. +7. Copy and prepare the command to install the chart: ```sh - helm install demo ./deploy/helm/elastic-agent \ + helm install demo elastic/elastic-agent \ --set agent.fleet.enabled=true \ - --set agent.fleet.url= \ - --set agent.fleet.token= \ + --set system.enabled=true \ + --set agent.fleet.url= \ # Substitute Fleet-URL with the URL that you copied earlier + --set agent.fleet.token= \ # Substitute Fleet-token with the enrollment token that you copied earlier. --set agent.fleet.preset=perNode ``` - Note that the command has these properties: - - * `helm install` runs the Helm CLI install tool. - * `demo` gives a name to the installed chart. You can choose any name. - * `./deploy/helm/elastic-agent` is a local path to the Helm chart to install (in time it’s planned to have a public URL for the chart). - * `--set agent.fleet.enabled=true` enables {{fleet}}-managed {{agent}}. The CLI parameter overrides the default `false` value for `agent.fleet.enabled` in the {{agent}} [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml) file. - * `--set agent.fleet.url=` sets the address where {{agent}} will connect to {{fleet}} in your {{ecloud}} deployment, over port 443 (again, overriding the value set by default in the {{agent}} [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml) file). - * `--set agent.fleet.token=` sets the enrollment token that {{agent}} uses to authenticate with {{fleet}}. - * `--set agent.fleet.preset=perNode` enables {{k8s}} metrics on `per node` basis. You can alternatively set cluster wide metrics (`clusterWide`) or kube-state-metrics (`ksmSharded`). + The command has these properties: - ::::{tip} - For a full list of all available YAML settings and descriptions, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent). - :::: + * `helm install`: Runs the Helm CLI install tool. You can use `helm upgrade` to modify or update an installed release. + * `demo`: The name for this specific installation of the chart, known as the **release name**. You can choose any name you like. + * `elastic/elastic-agent`: The name of the chart to install, using the format `/`. + * `--set agent.fleet.enabled=true`: Enables {{fleet}}-managed {{agent}}, which is disabled (`false`) by default. + * `--set system.enabled=true`: Adds the required volumes and mounts to enable host monitoring through the System integration. + * `--set agent.fleet.url=`: Specifies the address where {{agent}} connects to {{fleet}} Server in your {{ecloud}} deployment. + * `--set agent.fleet.token=`: Sets the enrollment token that {{agent}} uses to authenticate with {{fleet}} Server. + * `--set agent.fleet.preset=perNode`: Runs the agent as a `DaemonSet`, which is required for the purpose of this example. Refer to [](install-on-kubernetes-using-helm.md) for more details and use cases for this parameter. -8. Update the command to replace: + After your updates, the command should be similar to: - 1. `` with the URL that you copied earlier. - 2. `` with the enrollment token that you copied earlier. + ```sh + helm install demo elastic/elastic-agent \ + --set agent.fleet.enabled=true \ + --set system.enabled=true \ + --set agent.fleet.url=https://256575858845283fxxxxxxxd5265d2b4.fleet.us-central1.gcp.foundit.no:443 \ + --set agent.fleet.token=eSVvFDUvSUNPFldFdhhZNFwvS5xxxxxxxxxxxxFEWB1eFF1YedUQ1NWFXwr== \ + --set agent.fleet.preset=perNode + ``` - After your updates, the command should look something like this: + ::::{tip} + For a full list of all available values settings and descriptions, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent) and default [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml). - ```sh - helm install demo ./deploy/helm/elastic-agent \ - --set agent.fleet.enabled=true \ - --set agent.fleet.url=https://256575858845283fxxxxxxxd5265d2b4.fleet.us-central1.gcp.foundit.no:443 \ - --set agent.fleet.token=eSVvFDUvSUNPFldFdhhZNFwvS5xxxxxxxxxxxxFEWB1eFF1YedUQ1NWFXwr== \ - --set agent.fleet.preset=perNode - ``` + The following options could be useful for special use cases: + * `--namespace `: Allows to install all resources in a specific namespace. + * `--version `: Installs a specific version of the Helm chart and {{agent}}. Refer to [Preparations](#preparations) to check available versions. + * `--set agent.version=`: Installs a specific version of {{agent}}. By default, the chart installs the agent version that matches its own. + * `--set-file agent.fleet.ca.value=/local-path/to/fleet-ca.crt`: Provides the CA certificate used by the {{fleet}} Server. This is typically needed when the server uses a certificate signed by a private CA. Not required for {{fleet}} Servers running on {{ecloud}}. + * `--set agent.fleet.insecure=true`: Use this option to skip the {{fleet}} certificate verification if your {{fleet}} Server uses a self-signed certificate, such as when installed in quickstart mode. Not required for {{fleet}} Servers running on {{ecloud}}. Note that this option is not recommended for production environments. + * `--set kube-state-metrics.enabled=false`: In case you already have KSM installed in your cluster, and you don't want to install a second instance. + * `--set kube-state-metrics.fullnameOverride=ksm`: If you want to deploy KSM with a different release name (it defaults to `kube-state-metrics`). This can be useful if you have already a default installation of KSM and you want a second one. + :::: -9. Run the command. +8. Run the command. The command output should confirm that {{agent}} has been installed: @@ -89,32 +123,36 @@ To get started, you need: ... ``` -10. Run the `kubectl get pods -n default` command to confirm that the {{agent}} pod is running: +9. Run the `kubectl get pods -n default` command to confirm that the {{agent}} Pods are running. You should see one {{agent}} Pod running on each Kubernetes node: ```sh NAME READY STATUS RESTARTS AGE agent-pernode-demo-86mst 1/1 Running 0 12s ``` -11. In the **Add agent** flyout, wait a minute or so for confirmation that {{agent}} has successfully enrolled with {{fleet}} and that data is flowing: + ::::{note} + If your Kubernetes nodes have [`taints`](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) configured, you may need to add `tolerations` to the {{agent}} DaemonSet during installation to ensure the Pods can run on tainted nodes. + + You can do this by setting the `presets.perNode.tolerations[]` value, which accepts standard Kubernetes toleration definitions. + :::: + +10. In the **Add agent** flyout, wait a minute or so for confirmation that {{agent}} has successfully enrolled with {{fleet}} and that data is flowing: :::{image} images/helm-example-nodes-enrollment-confirmation.png :alt: Screen capture of Add Agent UI showing that the agent has enrolled in Fleet :screenshot: ::: -12. In {{fleet}}, open the **Agents** tab and see that an **Agent-pernode-demo-#** agent is running. -13. Select the agent to view its details. -14. On the **Agent details** tab, on the **Integrations** pane, expand `system-1` to confirm that logs and metrics are incoming. You can click either the `Logs` or `Metrics` link to view details. +11. In {{fleet}}, open the **Agents** tab and see that an **Agent-pernode-demo-#** agent is running. +12. Select the agent to view its details. +13. On the **Agent details** tab, on the **Integrations** pane, expand `system-1` to confirm that logs and metrics are incoming. You can click either the `Logs` or `Metrics` link to view details. :::{image} images/helm-example-nodes-logs-and-metrics.png :alt: Screen capture of the Logs and Metrics view on the Integrations pane :screenshot: ::: - - -## Install the Kubernetes integration [agent-fleet-managed-helm-example-install-integration] +## Step 3: Install the Kubernetes integration [agent-fleet-managed-helm-example-install-integration] Now that you’ve {{agent}} and data is flowing, you can set up the {{k8s}} integration. @@ -123,9 +161,15 @@ Now that you’ve {{agent}} and data is flowing, you can set up the {{k8s}} inte 3. On the {{k8s}} integration page, click **Add Kubernetes** to add the integration to your {{agent}} policy. 4. Scroll to the bottom of **Add Kubernetes integration** page. Under **Where to add this integration?** select the **Existing hosts** tab. On the **Agent policies** menu, select the agent policy that you created previously in the [Install {{agent}}](#agent-fleet-managed-helm-example-install-agent) steps. - You can leave all of the other integration settings at their default values. + You can leave all of the other integration settings at their default values. For details about the available inputs and data sets, refer to the [Kubernetes integration](integration-docs://reference/kubernetes/index.md) documentation. + + ::::{important} + All inputs under the **Collect Kubernetes metrics from kube-state-metrics** section default to `kube-state-metrics:8080` as the destination host. This works if you deployed KSM (kube-state-metrics) alongside {{agent}} during the chart installation, which is the default behavior, as both components are installed in the same namespace. -5. Click **Save and continue**. When prompted, select to **Add Elastic Agent later** since you’ve already added it using Helm. + If your KSM instance runs in a different namespace than {{agent}}, or if it uses a different service name, update the `host` setting in each data set of the integration to point to the KSM service. + :::: + +5. Click **Save and continue**. When prompted, select **Add Elastic Agent later**, because you’ve already added the agent using Helm. 6. On the {{k8s}} integration page, open the **Assets** tab and select the **[Metrics Kubernetes] Pods** dashboard. On the dashboard, you can view the status of your {{k8s}} pods, including metrics on memory usage, CPU usage, and network throughput. @@ -153,4 +197,8 @@ The uninstall should be confirmed as shown: release "demo" uninstalled ``` -As a reminder, for full details about using the {{agent}} Helm chart refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent). +## Next steps + +For full details about using the {{agent}} Helm chart, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent). + +Refer to the [examples](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples) section of the GitHub repository for advanced use cases, such as integrating {{agent}}s with [KSM autosharding](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-ksm-sharding), or configuring [mutual TLS authentication](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-certificates) between {{agent}}s and the {{fleet}} Server. \ No newline at end of file diff --git a/reference/fleet/example-kubernetes-standalone-agent-helm.md b/reference/fleet/example-kubernetes-standalone-agent-helm.md index b6d9f153ee..8aeb3c8f64 100644 --- a/reference/fleet/example-kubernetes-standalone-agent-helm.md +++ b/reference/fleet/example-kubernetes-standalone-agent-helm.md @@ -5,11 +5,6 @@ mapped_pages: # Example: Install standalone Elastic Agent on Kubernetes using Helm [example-kubernetes-standalone-agent-helm] -::::{warning} -This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. -:::: - - This example demonstrates how to install standalone {{agent}} on a Kubernetes system using a Helm chart, gather Kubernetes metrics and send them to an {{es}} cluster in {{ecloud}}, and then view visualizations of those metrics in {{kib}}. For an overview of the {{agent}} Helm chart and its benefits, refer to [Install {{agent}} on Kubernetes using Helm](/reference/fleet/install-on-kubernetes-using-helm.md). diff --git a/reference/fleet/install-on-kubernetes-using-helm.md b/reference/fleet/install-on-kubernetes-using-helm.md index 111558bb3d..23f23b96a5 100644 --- a/reference/fleet/install-on-kubernetes-using-helm.md +++ b/reference/fleet/install-on-kubernetes-using-helm.md @@ -5,11 +5,6 @@ mapped_pages: # Install Elastic Agent on Kubernetes using Helm [install-on-kubernetes-using-helm] -::::{warning} -This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. -:::: - - Starting with {{stack}} version 8.16, a Helm chart is available for installing {{agent}} in a Kubernetes environment. A Helm-based install offers several advantages, including simplified deployment, availability in marketplaces, streamlined ugrades, as well as quick rollbacks whenever they’re needed. Features of the Helm-based {{agent}} install include: diff --git a/reference/fleet/toc.yml b/reference/fleet/toc.yml index 60a0dbe243..bb32917cee 100644 --- a/reference/fleet/toc.yml +++ b/reference/fleet/toc.yml @@ -27,8 +27,9 @@ toc: - file: elastic-agent-container.md - file: running-on-kubernetes-managed-by-fleet.md - file: install-on-kubernetes-using-helm.md - - file: example-kubernetes-standalone-agent-helm.md - - file: example-kubernetes-fleet-managed-agent-helm.md + children: + - file: example-kubernetes-standalone-agent-helm.md + - file: example-kubernetes-fleet-managed-agent-helm.md - file: advanced-kubernetes-managed-by-fleet.md - file: configuring-kubernetes-metadata.md - file: running-on-gke-managed-by-fleet.md