Skip to content

Commit 98244d3

Browse files
author
Bibas
committed
add(docs): docs to send metrics from k8s cluster to cockpit
1 parent 79f24f6 commit 98244d3

File tree

1 file changed

+164
-0
lines changed

1 file changed

+164
-0
lines changed

‎observability/cockpit/how-to/send-metrics-from-k8s-to-cockpit.mdx‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
---
2+
meta:
3+
title: How to send metrics from your Kubernetes cluster to your Cockpit
4+
description: Learn how to send your pod metrics to your Cockpit using Scaleway's comprehensive guide. This tutorial covers sending Kubernetes pods metrics to Scaleway's Cockpit for centralized monitoring and analysis using Grafana, ensuring efficient monitoring and metrics analysis in your infrastructure.
5+
content:
6+
h1: How to send metrics from your Kubernetes cluster to your Cockpit
7+
paragraph: Learn how to send your pod metrics to your Cockpit using Scaleway's comprehensive guide. This tutorial covers sending Kubernetes pods metrics to Scaleway's Cockpit for centralized monitoring and analysis using Grafana, ensuring efficient monitoring and metrics analysis in your infrastructure.
8+
tags: kubernetes cockpit metrics observability monitoring cluster
9+
categories:
10+
- observability
11+
dates:
12+
validation: 2025/01/07
13+
posted: 2025/01/07
14+
---
15+
16+
17+
This page shows you how to send application metrics created in a Kubernetes cluster to your Cockpit either by using a Helm chart or by deploying a Helm chart with [Terraform](https://www.terraform.io/).
18+
19+
In this example, we use [k8s-monitoring](https://artifacthub.io/packages/helm/grafana/k8s-monitoring/1.6.16) which installs an Alloy Daemon set to your Kubernetes cluster to export metrics to your Cockpit.
20+
21+
<Macro id="requirements" />
22+
23+
- A Scaleway account metricsged into the [console](https://console.scaleway.com)
24+
- [Owner](/identity-and-access-management/iam/concepts/#owner) status or [IAM permissions](/identity-and-access-management/iam/concepts/#permission) allowing you to perform actions in the intended Organization
25+
- [Created](/observability/cockpit/how-to/create-external-data-sources/) a custom external data source of type metrics
26+
- [Created](/observability/cockpit/how-to/create-token/) a Cockpit token for the same region as the data source
27+
- A running Kubernetes cluster containing a deployed application exposing metrics
28+
- [Created](/identity-and-access-management/iam/how-to/create-api-keys/) an API key and retrieved your API secret key
29+
30+
<Message type="important">
31+
- Sending metrics for Scaleway resources or personal data using an external path is a billable feature. In addition, any data that you push yourself is billed, even if you send data from Scaleway products. Refer to the [product pricing](https://www.scaleway.com/en/pricing/?tags=available,managedservices-observability-cockpit) for more information.
32+
</Message>
33+
34+
35+
## Configure the Helm chart
36+
37+
Create a `values.yml` file to configure your Helm chart, using the example below. Make sure that you replace `$SCW_CLUSTER_NAME` with the name of your Scaleway Kubernetes cluster, `$COCKPIT_CUSTOM_DATASOURCE_HOST` with the hostname of your custom endpoint (excluding the `/loki/api/v1/push` part), and `$COCKPIT_TOKEN` with your Cockpit token.
38+
39+
```yaml
40+
cluster:
41+
name: "$SCW_CLUSTER_NAME"
42+
externalServices:
43+
prometheus:
44+
host: "$COCKPIT_CUSTOM_DATASOURCE_HOST"
45+
tenantId: "$COCKPIT_TOKEN"
46+
writeEndpoint: "/api/v1/push"
47+
authMode: none
48+
externalLabels: {
49+
cluster: "$SCW_CLUSTER_NAME"
50+
}
51+
52+
metrics:
53+
enabled: true
54+
scrapeInterval: 60s
55+
autoDiscover:
56+
enabled: true
57+
# -- Only capture metrics from pods in these namespaces (`[]` means all namespaces).
58+
namespaces: []
59+
60+
logs:
61+
enabled: false
62+
kube-state-metrics:
63+
enabled: false
64+
prometheus-node-exporter:
65+
enabled: false
66+
prometheus-operator-crds:
67+
enabled: false
68+
opencost:
69+
enabled: false
70+
```
71+
72+
<Message type="info">
73+
The template above is only an example to send metrics to your Cockpit. You can also send logs to Cockpit using this Helm chart.
74+
You can check our guide to [send logs from your cluster to Cockpit](// ADD LINK TO LOGS TUTO)
75+
</Message>
76+
77+
## Add annotation to your deployed pod to enable auto-discovery
78+
79+
In order for k8s-monitoring to discover the pods it needs to scrape, you need to add specific annotation to the pods you want to scrape
80+
81+
Add annotation to indicate to k8s-monitoring to scrape the pods from your deployment. Make sure to replace $METRIC_PORT with your Prometheus port.
82+
83+
### Kubernetes
84+
85+
```yaml
86+
apiVersion: apps/v1
87+
kind: Deployment
88+
metadata:
89+
...
90+
annotations:
91+
"k8s.grafana.com/metrics.portNumber" = "$METRIC_PORT"
92+
"k8s.grafana.com/scrape" = "true"
93+
spec:
94+
...
95+
```
96+
97+
### Terraform
98+
99+
```terraform
100+
resource "kubernetes_deployment_v1" "your_application_deployment" {
101+
...
102+
spec {
103+
...
104+
template {
105+
metadata {
106+
...
107+
annotations = {
108+
"k8s.grafana.com/metrics.portNumber" = "$METRIC_PORT"
109+
"k8s.grafana.com/scrape" = "true"
110+
}
111+
}
112+
...
113+
}
114+
}
115+
}
116+
```
117+
118+
## Send Kubernetes metrics to your Cockpit using Helm chart with Terraform
119+
120+
1. Set up the Helm Terraform provider:
121+
```terraform
122+
provider "helm" {
123+
kubernetes {
124+
host = your_k8s_cluster_host
125+
token = your_k8s_cluster_token
126+
cluster_ca_certificate = base64decode(
127+
your_k8s_cluster_ca_certificate
128+
)
129+
}
130+
}
131+
```
132+
2. Create a Helm release resource with the path to your `values.yml`:
133+
```
134+
resource "helm_release" "metrics-ingester" {
135+
name = "my-metrics-ingester"
136+
repository = "https://grafana.github.io/helm-charts"
137+
chart = "k8s-monitoring"
138+
version = "1.6.16"
139+
140+
namespace = "metrics-ingester"
141+
create_namespace = true
142+
values = [file("/your-path/to/values.yml")]
143+
}
144+
```
145+
3. Run `terraform apply` to apply the new Terraform configuration.
146+
147+
## Send Kubernetes metrics to your Cockpit using Helm chart
148+
149+
1. Connect your kubectl to your Kubernetes cluster
150+
2. Run the following command to apply your Helm chart with the `values.yml` file:
151+
```
152+
helm install -f /your-path/to/values.yml my-metrics-ingester k8s-monitoring --version 1.6.16
153+
```
154+
Make sure to replace `-f` flag with the correct path to your `values.yml` file.
155+
156+
157+
## Explore your metrics in Grafana
158+
159+
Now that your metrics are exported to your Cockpit, you can access and query them.
160+
161+
1. Click **Cockpit** in the Observability section of the [console](https://console.scaleway.com/) side menu. The **Cockpit** overview page displays.
162+
2. Click **Open dashboards** to open your managed dashboards in Grafana. You are redirected to the Grafana website.
163+
3. Click the **Home** icon > **Explore**. Select your custom data source in the upper left corner.
164+
4. You can now query your metrics from your Kubernetes cluster and use the datasource to create graph.

0 commit comments

Comments
 (0)