From 045bab656ae9259b53e4567e4ee23bbad85911d9 Mon Sep 17 00:00:00 2001 From: Bibas Date: Fri, 27 Dec 2024 11:33:21 +0100 Subject: [PATCH 1/5] docs(cockpit): added documentation on how to add custom logs from k8s cluster to cockpit int-add-observability --- .../how-to/send-log-from-k8s-to-cockpit.mdx | 144 ++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx diff --git a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx new file mode 100644 index 0000000000..5f3e3635b7 --- /dev/null +++ b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx @@ -0,0 +1,144 @@ +--- +meta: + title: How to send logs from your Kubernetes cluster to your Cockpit + description: Learn how to send your pod logs to your Cockpit using Scaleway's comprehensive guide. This tutorial covers sending Kubernetes pods logs to Scaleway's Cockpit for centralized monitoring and analysis using Grafana, ensuring efficient monitoring and log analysis in your infrastructure. +content: + h1: How to send logs from your Kubernetes cluster to your Cockpit + paragraph: Learn how to send your pod logs to your Cockpit using Scaleway's comprehensive guide. This tutorial covers sending Kubernetes pods logs to Scaleway's Cockpit for centralized monitoring and analysis using Grafana, ensuring efficient monitoring and log analysis in your infrastructure. +tags: kubernetes cockpit logs observability monitoring cluster +categories: + - observability +dates: + validation: TBD + posted: TBD +--- + +This page shows you how to send your application logs created in a Kubernetes cluster to your Cockpit either by using a Helm Chart or by deploying this Helm chart with Terraform. + +In this example, we use [k8s-monitoring](https://artifacthub.io/packages/helm/grafana/k8s-monitoring/1.6.16) which install an Alloy Daemon set to your Kubernetes Cluster to export logs to your Cockpit. + + + + - A Scaleway account logged into the [console](https://console.scaleway.com) + - [Owner](/identity-and-access-management/iam/concepts/#owner) status or [IAM permissions](/identity-and-access-management/iam/concepts/#permission) allowing you to perform actions in the intended Organization + - [Created](/observability/cockpit/how-to/create-external-data-sources/) an custom external data source + - [Created](/observability/cockpit/how-to/create-token/) a Cockpit Token for the same region as the data source + - A running Kubernetes Cluster containing your application deployed + - For Terraform : [Created](/identity-and-access-management/iam/how-to/create-api-keys/) an API key and retrieved your API secret key + + + + + - Sending logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the [product pricing](https://www.scaleway.com/en/pricing/?tags=available,managedservices-observability-cockpit) for more information. + + +## Configure the Helm chart + +Create a `values.yml` file to configure your Helm chart, here is an example: + + + Do not forget to replace: + - `$SCW_CLUSTER_NAME` by the name of your Scaleway Cluster + - `$COCKPIT_CUSTOM_DATASOURCE_HOST` by the hostname of your custom endpoint (do not include the `/loki/api/v1/push` ) + - `$COCKPIT_TOKEN` by your Cockpit Token + + +```yaml +cluster: + name: "$SCW_CLUSTER_NAME" +externalServices: + loki: + host: "$COCKPIT_CUSTOM_DATASOURCE_HOST" + tenantId: "$COCKPIT_TOKEN" + authMode: none + externalLabels: { + cluster: "$SCW_CLUSTER_NAME" + } + +logs: + enabled: true + pod_logs: + enabled: true + discovery: "all" + annotation: "cockpit.scw.cloud/logs.autogather" + # -- Only capture logs from pods in these namespaces (`[]` means all namespaces). + namespaces: [] + # -- Do not capture logs from any pods in these namespaces. + excludeNamespaces: [] + +metrics: + enabled: false +kube-state-metrics: + enabled: false +prometheus-node-exporter: + enabled: false +prometheus-operator-crds: + enabled: false +opencost: + enabled: false + +alloy: + logging: + level: info + format: logfmt +alloy-events: + logging: + level: info + format: logfmt +alloy-logs: + logging: + # -- Level at which Alloy log lines should be written. + level: info + # -- Format to use for writing Alloy log lines. + format: logfmt +``` + + +- Note that this is only an example to send logs to your cockpit, you can also send metrics to cockpit using this Helm chart + + +## Send Kubernetes logs to your Cockpit using Helm chart + +1. Connect your kubectl to your Scaleway Cluster +2. Apply your Helm chart with the `values.yml` file by running + ``` + helm install -f /your-path/to/values.yml my-log-ingester k8s-monitoring --version 1.6.16 + ``` + Do not forget to change the -f flag with the correct path to your `values.yml` file + +## Send Kubernetes logs to your Cockpit using Helm chart with Terraform + +1. Set up the Helm Terraform provider + ```terraform + provider "helm" { + kubernetes { + host = your_k8s_cluster_host + token = your_k8s_cluster_token + cluster_ca_certificate = base64decode( + your_k8s_cluster_ca_certificate + ) + } + } + ``` +2. Create a Helm release ressource with the path to your `values.yml` + ``` + resource "helm_release" "log-ingester" { + name = "my-log-ingester" + repository = "https://grafana.github.io/helm-charts" + chart = "k8s-monitoring" + version = "1.6.16" + + namespace = "log-ingester" + create_namespace = true + values = [file("/your-path/to/values.yml")] + } + ``` +3. Apply the new terraform configuration with `terraform apply` + +## Explore your logs + +Now that your logs are exported to your Cockpit, you can access and query them. + +1. Open your Cockpit Dashboard +2. Click the **Home** icon > **Explore**. Select your custom data source in the upper left corner. +3. Filter the query by only searching for the label `cluster` with the value corresponding to your cluster name and your log should display From 8867ccd99b9bc12885429a05001ae3d78c85d488 Mon Sep 17 00:00:00 2001 From: Bibas Date: Fri, 27 Dec 2024 11:43:26 +0100 Subject: [PATCH 2/5] docs(observability): add comment to explain annotation --- observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx index 5f3e3635b7..c6a1ecce10 100644 --- a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx +++ b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx @@ -60,6 +60,7 @@ logs: pod_logs: enabled: true discovery: "all" + # This annotation is mandatory to gather your logs annotation: "cockpit.scw.cloud/logs.autogather" # -- Only capture logs from pods in these namespaces (`[]` means all namespaces). namespaces: [] From a25aa6a1030c92911d08622f5781851ff6376bc6 Mon Sep 17 00:00:00 2001 From: nerda-codes <87707325+nerda-codes@users.noreply.github.com> Date: Tue, 31 Dec 2024 10:26:31 +0100 Subject: [PATCH 3/5] docs(cockpit): neda review Co-authored-by: Jessica <113192637+jcirinosclwy@users.noreply.github.com> --- .../how-to/send-log-from-k8s-to-cockpit.mdx | 50 ++++++++----------- 1 file changed, 21 insertions(+), 29 deletions(-) diff --git a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx index c6a1ecce10..2d174131ce 100644 --- a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx +++ b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx @@ -9,24 +9,22 @@ tags: kubernetes cockpit logs observability monitoring cluster categories: - observability dates: - validation: TBD - posted: TBD + validation: 2025/01/03 + posted: 2025/01/03 --- -This page shows you how to send your application logs created in a Kubernetes cluster to your Cockpit either by using a Helm Chart or by deploying this Helm chart with Terraform. +This page shows you how to send application logs created in a Kubernetes cluster to your Cockpit either by using a Helm chart or by deploying a Helm chart with [Terraform](https://www.terraform.io/). -In this example, we use [k8s-monitoring](https://artifacthub.io/packages/helm/grafana/k8s-monitoring/1.6.16) which install an Alloy Daemon set to your Kubernetes Cluster to export logs to your Cockpit. +In this example, we use [k8s-monitoring](https://artifacthub.io/packages/helm/grafana/k8s-monitoring/1.6.16) which installs an Alloy Daemon set to your Kubernetes cluster to export logs to your Cockpit. - A Scaleway account logged into the [console](https://console.scaleway.com) - [Owner](/identity-and-access-management/iam/concepts/#owner) status or [IAM permissions](/identity-and-access-management/iam/concepts/#permission) allowing you to perform actions in the intended Organization - - [Created](/observability/cockpit/how-to/create-external-data-sources/) an custom external data source - - [Created](/observability/cockpit/how-to/create-token/) a Cockpit Token for the same region as the data source - - A running Kubernetes Cluster containing your application deployed - - For Terraform : [Created](/identity-and-access-management/iam/how-to/create-api-keys/) an API key and retrieved your API secret key - - + - [Created](/observability/cockpit/how-to/create-external-data-sources/) a custom external data source + - [Created](/observability/cockpit/how-to/create-token/) a Cockpit token for the same region as the data source + - A running Kubernetes cluster containing your application deployed + - [Created](/identity-and-access-management/iam/how-to/create-api-keys/) an API key and retrieved your API secret key - Sending logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the [product pricing](https://www.scaleway.com/en/pricing/?tags=available,managedservices-observability-cockpit) for more information. @@ -34,14 +32,7 @@ In this example, we use [k8s-monitoring](https://artifacthub.io/packages/helm/gr ## Configure the Helm chart -Create a `values.yml` file to configure your Helm chart, here is an example: - - - Do not forget to replace: - - `$SCW_CLUSTER_NAME` by the name of your Scaleway Cluster - - `$COCKPIT_CUSTOM_DATASOURCE_HOST` by the hostname of your custom endpoint (do not include the `/loki/api/v1/push` ) - - `$COCKPIT_TOKEN` by your Cockpit Token - +Create a `values.yml` file to configure your Helm chart, using the example below. Make sure that you replace `$SCW_CLUSTER_NAME` with the name of your Scaleway Kubernetes cluster, `$COCKPIT_CUSTOM_DATASOURCE_HOST` with the hostname of your custom endpoint (excluding the `/loki/api/v1/push` part), and `$COCKPIT_TOKEN` with your Cockpit token. ```yaml cluster: @@ -95,21 +86,21 @@ alloy-logs: ``` -- Note that this is only an example to send logs to your cockpit, you can also send metrics to cockpit using this Helm chart +The template above is only an example to send logs to your Cockpit. You can also send metrics to Cockpit using this Helm chart. ## Send Kubernetes logs to your Cockpit using Helm chart -1. Connect your kubectl to your Scaleway Cluster -2. Apply your Helm chart with the `values.yml` file by running +1. Connect your kubectl to your Kubernetes cluster +2. Run the following command to apply your Helm chart with the `values.yml` file: ``` helm install -f /your-path/to/values.yml my-log-ingester k8s-monitoring --version 1.6.16 ``` - Do not forget to change the -f flag with the correct path to your `values.yml` file + Make sure to replace `-f` flag with the correct path to your `values.yml` file. ## Send Kubernetes logs to your Cockpit using Helm chart with Terraform -1. Set up the Helm Terraform provider +1. Set up the Helm Terraform provider: ```terraform provider "helm" { kubernetes { @@ -121,7 +112,7 @@ alloy-logs: } } ``` -2. Create a Helm release ressource with the path to your `values.yml` +2. Create a Helm release resource with the path to your `values.yml`: ``` resource "helm_release" "log-ingester" { name = "my-log-ingester" @@ -134,12 +125,13 @@ alloy-logs: values = [file("/your-path/to/values.yml")] } ``` -3. Apply the new terraform configuration with `terraform apply` +3. Run `terraform apply` to apply the new Terraform configuration. -## Explore your logs +## Explore your logs in Grafana Now that your logs are exported to your Cockpit, you can access and query them. -1. Open your Cockpit Dashboard -2. Click the **Home** icon > **Explore**. Select your custom data source in the upper left corner. -3. Filter the query by only searching for the label `cluster` with the value corresponding to your cluster name and your log should display +1. Click **Cockpit** in the Observability section of the [console](https://console.scaleway.com/) side menu. The **Cockpit** overview page displays. +2. Click **Open dashboards** to open your managed dashboards in Grafana. You are redirected to the Grafana website. +3. Click the **Home** icon > **Explore**. Select your custom data source in the upper left corner. +4. Filter the query by only searching for the label `cluster` with the value corresponding to your cluster name and your logs should display. From 09a844e06a5dee36220c59cf7a088b31c16e4973 Mon Sep 17 00:00:00 2001 From: Bibas Date: Thu, 2 Jan 2025 11:12:24 +0100 Subject: [PATCH 4/5] fix(docs): typo in closing tag of a block --- observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx index 2d174131ce..2a2530cb63 100644 --- a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx +++ b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx @@ -28,7 +28,7 @@ In this example, we use [k8s-monitoring](https://artifacthub.io/packages/helm/gr - Sending logs for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the [product pricing](https://www.scaleway.com/en/pricing/?tags=available,managedservices-observability-cockpit) for more information. - + ## Configure the Helm chart From 79f24f631484dc2383823a683cad04824381192b Mon Sep 17 00:00:00 2001 From: Bibas Date: Mon, 6 Jan 2025 15:55:18 +0100 Subject: [PATCH 5/5] add(docs): link to metrics tutorial in note --- observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx index 2a2530cb63..488cea1a96 100644 --- a/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx +++ b/observability/cockpit/how-to/send-log-from-k8s-to-cockpit.mdx @@ -87,6 +87,7 @@ alloy-logs: The template above is only an example to send logs to your Cockpit. You can also send metrics to Cockpit using this Helm chart. +You can check our guide to [send metrics from your cluster to Cockpit](// ADD LINK TO METRICS TUTO) ## Send Kubernetes logs to your Cockpit using Helm chart