You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You can now send **data plane** logs from your [Kapsule or Kosmos](https://www.scaleway.com/en/kubernetes) clusters to [Cockpit](https://www.scaleway.com/en/cockpit/), providing centralized, real-time access to application and system logs. Reduce complexity and manual work thanks to this integration, powered by a **Promtail** deployment via [Easy Deploy](https://www.scaleway.com/en/docs/containers/kubernetes/how-to/enable-easy-deploy/).
16
+
You can now send **data plane** logs from your [Kapsule or Kosmos](https://www.scaleway.com/en/kubernetes) clusters to [Cockpit](https://www.scaleway.com/en/cockpit/), providing centralized, real-time access to application and system logs. Reduce complexity and manual work thanks to this integration, powered by a **Promtail** deployment via [Easy Deploy](/containers/kubernetes/how-to/enable-easy-deploy/).
17
17
18
18
With this feature:
19
19
20
20
-**Enhance observability**: View logs from all your Kubernetes containers in one place.
21
21
-**Simplify troubleshooting**: Quickly drill down into specific pods or containers without needing to configure a separate logging stack.
22
22
23
23
<Messagetype="important">
24
-
This feature does incur costs based on the volume of logs ingested. Refer to [Cockpit Usage and Pricing](/observability/cockpit/reference-content/understanding-cockpit-usage/) for more details and best practices to avoid unexpected bills.
24
+
This feature does incur costs based on the volume of logs ingested. Refer to [Cockpit FAQ](/faq/cockpit/#how-am-i-billed-for-using-cockpit-with-custom-data) for more details and best practices to avoid unexpected bills. |
25
25
</Message>
26
26
27
27
<Macroid="requirements" />
28
28
29
29
- A running [Kapsule](/containers/kubernetes/how-to/create-cluster/) or [Kosmos](/containers/kubernetes/how-to/create-kosmos-cluster/) cluster.
30
-
- Sufficient permissions to deploy Easy Deploy applications on your cluster
30
+
- An API Key with [IAM permissions](/identity-and-access-management/iam/reference-content/permission-sets/) to edit your cluster `(KubernetesFullAccess` or `KubernetesSystemMastersGroupAccess`)
31
+
-[A token](/observability/cockpit/how-to/create-token/) with permissions to push to, and query logs from Cockpit
31
32
32
-
## Architecture and limitations
33
+
## Architecture and Limitations
33
34
34
35
### Control Plane vs. Data Plane
35
36
36
-
-**Control Plane**: Fully managed by Scaleway. Users can already [monitor control plane components](/containers/kubernetes/how-to/monitor-cluster/) (e.g., `kube-apiserver`, `CCM`, `CSI`) via Cockpit.
37
+
-**Control Plane**: Fully managed by Scaleway. Users can already [monitor control plane components](/containers/kubernetes/how-to/monitor-cluster/) (e.g., `kube-apiserver`, `CCM`, `CSI`) via Cockpit.
37
38
-**Data Plane**: Runs in your Scaleway Project (customer-managed instances, `kubelet`, `containerd`, customer Pods, etc.). You have **full access** to the data plane, including the ability to SSH into nodes.
38
39
39
40
Because the data plane is entirely under your control, **logs from any components running on these nodes are considered your own data**. Consequently, shipping these logs to Cockpit is billed based on data ingestion.
40
41
41
42
## How it works
42
43
43
-
The system leverages **Promtail** (a lightweight log collector) running on your Kapsule or Kosmos cluster. Promtail forwards logs to your Cockpit instance’s Loki endpoint:
44
+
The system leverages **Promtail** (a lightweight log collector) running on your Kapsule or Kosmos cluster. Promtail forwards logs to the Loki endpoint of your Cockpit instance:
44
45
45
46
1.**Promtail** can collect logs from:
46
47
-**Container stdout/stderr** (pods)
47
48
-**systemd journal** (e.g., `kubelet.service`)
48
49
2.**Log data** is transmitted to **Cockpit** (Loki).
49
50
3.**Cockpit** stores and indexes these logs.
50
51
51
-
## Step-by-Step: Enabling container logs in Cockpit
52
+
## Step-by-step: Enabling container logs in Cockpit
52
53
53
-
You can use Scaleway’s **Easy Deploy** to add a Promtail deployment to your cluster:
54
+
You can use Scaleway’s **[Easy Deploy](/containers/kubernetes/how-to/enable-easy-deploy/)** to add a Promtail deployment to your cluster:
54
55
55
56
1.**Open the Scaleway Console** and go to your **Kubernetes** cluster.
56
57
2. Navigate to the **Easy Deploy** tab.
@@ -59,10 +60,9 @@ You can use Scaleway’s **Easy Deploy** to add a Promtail deployment to your cl
59
60
- Collect container logs for **all namespaces** (by default).
You may edit the default configuration of the deployment to specify the sources of logs to ingest (under `config.snippets.scrapeConfigs` in the yaml file): `cockpit_promtail_scrape_config_pods: “… list of namespaces…” cockpit_promtail_scrape_config_journal: “… list of system components…”` |
65
-
</Message>
65
+
</Message>
66
66
67
67
### Example Promtail configuration
68
68
@@ -89,14 +89,14 @@ extraVolumes:
89
89
```
90
90
91
91
<Messagetype="note">
92
-
The placeholders like `{{{ cockpit_bearer_token }}}` and `{{{ cockpit_loki_push_url }}}` are automatically replaced by the Easy Deploy system with your actual values. |
92
+
Template values like `{{{ cockpit_bearer_token }}}` and `{{{ cockpit_loki_push_url }}}` are automatically replaced by the Easy Deploy system with your actual values. Please do not overwrite any of these two values. |
93
93
</Message>
94
94
95
95
## Observing logs in Cockpit
96
96
97
97
Once Promtail is running:
98
98
99
-
1. Go to **Cockpit** → **Kubernetes Cluster Pod Logs**in the Scaleway Console (or open your own Grafana connected to Cockpit).
99
+
1. Go to **Cockpit** → **Kubernetes Cluster Pod Logs** (or open your own Grafana connected to Cockpit).
100
100
2.**Filter** by:
101
101
-`Datasource` which is automatically created upon deployment, and visible in the Cockpit console
102
102
-`Cluster Name` ( e.g. `my-kapsule-cluster`)
@@ -108,13 +108,17 @@ Once Promtail is running:
108
108
109
109
Sending logs to Cockpit is billed based on the **total volume of logs ingested**. You are charged *€0.15 per million log lines*\* (samples) or partial million.
110
110
111
-
For more details on controlling your costs, see our [Cockpit Usage and Pricing guide](/observability/cockpit/reference-content/understanding-cockpit-usage/). Key points include:
111
+
Key points include:
112
112
113
113
-**Logging rate**: The more logs you produce (e.g. high-traffic workloads or verbose logging), the higher the bill.
114
114
-**Filtering**: Limit logs to critical namespaces or system components only.
115
115
116
+
<Messagetype="note">
117
+
You may edit the default configuration of the deployment to adjust the volume of logs to ingest: `extraLimitsConfig: | readline_rate_enabled: true #rate limiting readline_rate: 10000 #log lines / sec readline_burst: 10000 #cap for burst lines readline_rate_drop: true #drop excess lines`
118
+
</Message>
119
+
116
120
<Messagetype="tip">
117
-
Always monitor the logs ingestion rate in the dedicated dashboards provided in Cockpit, just below the log tables, to avoid surprises.
121
+
Always monitor the logs ingestion rate in the dedicated dashboards provided in Cockpit, just below the log tables, to avoid surprises.
118
122
</Message>
119
123
120
124
## Security considerations
@@ -126,18 +130,18 @@ For more details on controlling your costs, see our [Cockpit Usage and Pricing g
126
130
## Troubleshooting
127
131
128
132
-**No logs appearing** in Cockpit:
129
-
1. Verify that the Promtail pod is running (`kubectl get pods -n <promtail-namespace>`).
130
-
2. Inspect Promtail logs for errors.
133
+
- Verify that the Promtail pod is running (`kubectl get pods -n <promtail-namespace>`).
134
+
- Inspect Promtail logs for errors.
131
135
132
136
133
137
-**High log ingestion cost**:
134
-
1. Review your **deployment configuration** to filter out verbose logs or unneeded namespaces.
135
-
2. Check **log ingestion rate** in the dedicated dashboards for unusual spikes.
138
+
- Review your **deployment configuration** to filter out verbose logs or unneeded namespaces.
139
+
- Check **log ingestion rate** in the dedicated dashboards for unusual spikes.
0 commit comments