|
| 1 | +--- |
| 2 | +meta: |
| 3 | + title: How to monitor your Kubernetes Kapsule Data Plane with Cockpit |
| 4 | + description: This page explains how to integrate Kubernetes container logs with Scaleway Cockpit using Promtail |
| 5 | +content: |
| 6 | + h1: How to monitor your Kubernetes Kapsule Data Plane with Cockpit |
| 7 | + paragraph: This page explains how to integrate Kubernetes container logs with Scaleway Cockpit using Promtail |
| 8 | +categories: |
| 9 | + - iot-hub |
| 10 | +tags: kubernetes kapsule kosmos cockpit promtail logs |
| 11 | +dates: |
| 12 | + validation: 2025-01-27 |
| 13 | + posted: 2025-01-17 |
| 14 | +--- |
| 15 | + |
| 16 | +You can now send **data plane** logs from your [Kapsule or Kosmos](https://www.scaleway.com/en/kubernetes) clusters to [Cockpit](https://www.scaleway.com/en/cockpit/), providing centralized, real-time access to application and system logs. Reduce complexity and manual work thanks to this integration, powered by a **Promtail** deployment via [Easy Deploy](/kubernetes/how-to/enable-easy-deploy/). |
| 17 | + |
| 18 | +With this feature: |
| 19 | + |
| 20 | +- **Enhance observability**: View logs from all your Kubernetes containers in one place. |
| 21 | +- **Simplify troubleshooting**: Quickly drill down into specific pods or containers without needing to configure a separate logging stack. |
| 22 | + |
| 23 | +<Message type="important"> |
| 24 | + This feature does incur costs based on the volume of logs ingested. Refer to [Cockpit FAQ](/faq/cockpit/#how-am-i-billed-for-using-cockpit-with-custom-data) for more details and best practices to avoid unexpected bills. |
| 25 | +</Message> |
| 26 | + |
| 27 | +<Macro id="requirements" /> |
| 28 | + |
| 29 | +- A running [Kapsule](/kubernetes/how-to/create-cluster/) or [Kosmos](/kubernetes/how-to/create-kosmos-cluster/) cluster. |
| 30 | +- An API Key with [IAM permissions](/iam/reference-content/permission-sets/) to edit your cluster `(KubernetesFullAccess` or `KubernetesSystemMastersGroupAccess`) |
| 31 | +- [A token](/cockpit/how-to/create-token/) with permissions to push to, and query logs from Cockpit |
| 32 | + |
| 33 | +## Architecture and limitations |
| 34 | + |
| 35 | +### Control Plane vs. Data Plane |
| 36 | + |
| 37 | +- **Control Plane**: Fully managed by Scaleway. Users can already [monitor control plane components](/kubernetes/how-to/monitor-cluster/) (e.g., `kube-apiserver`, `CCM`, `CSI`) via Cockpit. |
| 38 | +- **Data Plane**: Runs in your Scaleway Project (customer-managed instances, `kubelet`, `containerd`, customer Pods, etc.). You have **full access** to the data plane, including the ability to SSH into nodes. |
| 39 | + |
| 40 | +| Feature | Control Plane | Data Plane | |
| 41 | +|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------| |
| 42 | +| Responsibility | Fully managed by Scaleway | Managed by the customer (runs in your Scaleway Project) | |
| 43 | +| Components | `kube-apiserver`, `CCM`, `CSI`, etc. | `kubelet`, `containerd`, customer Pods, and system components like `kubelet.service`. | |
| 44 | +| Access | Users can monitor components via Cockpit ([see how-to guide](/kubernetes/how-to/monitor-cluster/)) | Full access to data, including SSH into nodes, log management, and custom configurations. | |
| 45 | +| Billing | Included in cluster costs | Billed based on log ingestion volume (see pricing below). | |
| 46 | + |
| 47 | +Because the data plane is entirely under your control, **logs from any components running on these nodes are considered your own data**. Consequently, shipping these logs to Cockpit is billed based on data ingestion. |
| 48 | + |
| 49 | +## How it works |
| 50 | + |
| 51 | +The system leverages **Promtail** (a lightweight log collector) running on your Kapsule or Kosmos cluster. Promtail forwards logs to the Loki endpoint of your Cockpit instance: |
| 52 | + |
| 53 | +1. **Promtail** can collect logs from: |
| 54 | + - **Container stdout/stderr** (pods) |
| 55 | + - **systemd journal** (e.g., `kubelet.service`) |
| 56 | +2. **Log data** is transmitted to **Cockpit** (Loki). |
| 57 | +3. **Cockpit** stores and indexes these logs. |
| 58 | + |
| 59 | +## Step-by-step: Enabling container logs in Cockpit |
| 60 | + |
| 61 | +You can use Scaleway’s **[Easy Deploy](/kubernetes/how-to/enable-easy-deploy/)** to add a Promtail deployment to your cluster: |
| 62 | + |
| 63 | +1. **Open the Scaleway Console** and go to your **Kubernetes** cluster. |
| 64 | +2. Navigate to the **Easy Deploy** tab. |
| 65 | +3. Select **Promtail for Cockpit** from the library. |
| 66 | +4. **Deploy** the application. Promtail will install on your cluster with default settings that: |
| 67 | + - Collect container logs for **all namespaces** (by default). |
| 68 | + - Collect systemd journal logs (e.g., `kubelet.service`). |
| 69 | + - Forward logs securely to **Cockpit**. |
| 70 | + <Message type="note"> |
| 71 | + You can edit the default deployment configuration to filter logs by source (under `config.snippets.scrapeConfigs` in the YAML file). For example: |
| 72 | + ```yaml |
| 73 | + cockpit_promtail_scrape_config_pods: "namespace1,namespace2" |
| 74 | + cockpit_promtail_scrape_config_journal: "kubelet.service,kube-proxy.service" |
| 75 | + ``` |
| 76 | + </Message> |
| 77 | +### Example Promtail configuration |
| 78 | +Below is a simplified snippet of the configuration that Easy Deploy generates by default: |
| 79 | +```yaml |
| 80 | +config: |
| 81 | + clients: |
| 82 | + - bearer_token: "{{{ cockpit_bearer_token }}}" # Automatically set by Easy Deploy |
| 83 | + url: "{{{ cockpit_loki_push_url }}}" # Automatically set by Easy Deploy |
| 84 | + |
| 85 | + snippets: |
| 86 | + scrapeConfigs: | |
| 87 | + {{{- cockpit_promtail_scrape_config_pods }}} # Default: log all pods |
| 88 | + {{{- cockpit_promtail_scrape_config_journal }}} # Default: log all system components |
| 89 | +extraVolumeMounts: |
| 90 | + - mountPath: /var/log/journal |
| 91 | + name: journal |
| 92 | + readOnly: true |
| 93 | +extraVolumes: |
| 94 | + - hostPath: |
| 95 | + path: /var/log/journal |
| 96 | + name: journal |
| 97 | +``` |
| 98 | +<Message type="note"> |
| 99 | + Template values like `{{{ cockpit_bearer_token }}}` (Bearer Token) and `{{{ cockpit_loki_push_url }}}` (Loki URL) are automatically set. Avoid modifying these values. |
| 100 | +</Message> |
| 101 | + |
| 102 | +## Observing logs in Cockpit |
| 103 | + |
| 104 | +Once Promtail is running: |
| 105 | + |
| 106 | +1. Go to **Cockpit** → **Kubernetes Cluster Pod Logs** (or open your own Grafana connected to Cockpit). |
| 107 | +2. **Filter** by: |
| 108 | + - `Datasource` which is automatically created upon deployment, and visible in the Cockpit console |
| 109 | + - `Cluster Name` ( e.g. `my-kapsule-cluster`) |
| 110 | + - `namespace`, `pod`, or `container` labels to isolate specific workloads |
| 111 | + - **Time range** to limit how far back in history you want to query |
| 112 | +3. **Analyze** logs in real-time or historical mode to troubleshoot issues, watch for errors, or track performance. |
| 113 | + |
| 114 | +## Usage & pricing |
| 115 | + |
| 116 | +Sending logs to Cockpit is billed based on the **total volume of logs ingested**. You are charged *€0.15 per million log lines* (samples) or partial million. |
| 117 | + |
| 118 | +Key points include: |
| 119 | + |
| 120 | +- **Logging rate**: The more logs you produce (e.g. high-traffic workloads or verbose logging), the higher the bill. |
| 121 | +- **Filtering**: Limit logs to critical namespaces or system components only. |
| 122 | + |
| 123 | +<Message type="note"> |
| 124 | + You may edit the default configuration of the deployment to adjust the volume of logs to ingest: |
| 125 | + ```yaml |
| 126 | + extraLimitsConfig: | |
| 127 | + readline_rate_enabled: true # rate limiting |
| 128 | + readline_rate: 10000 # log lines / sec |
| 129 | + readline_burst: 10000 # cap for burst lines |
| 130 | + readline_rate_drop: true # drop excess lines |
| 131 | + ``` |
| 132 | + |
| 133 | +</Message> |
| 134 | + |
| 135 | +<Message type="tip"> |
| 136 | + Always monitor the logs ingestion rate in the dedicated dashboards provided in Cockpit, just below the log tables, to avoid surprises. |
| 137 | +</Message> |
| 138 | + |
| 139 | +## Security considerations |
| 140 | + |
| 141 | +- **Authentication**: The Promtail client uses a Cockpit Bearer Token to authenticate. Keep this token secret; do not store it in publicly accessible repos. |
| 142 | +- **Encryption**: Communication between Promtail and Cockpit (HTTPS) encrypts logs in transit. |
| 143 | +- **Access Control**: Ensure only trusted team members can deploy Easy Deploy applications or modify cluster-level configurations. |
| 144 | + |
| 145 | +## Troubleshooting |
| 146 | + |
| 147 | +- **No logs appearing** in Cockpit: |
| 148 | + - Verify that the Promtail pod is running. |
| 149 | + ```bash |
| 150 | + kubectl get pods -n <promtail-namespace> |
| 151 | + ``` |
| 152 | + - Inspect Promtail logs for errors. |
| 153 | + ```bash |
| 154 | + kubectl logs <promtail-pod-name> -n <promtail-namespace> |
| 155 | + ``` |
| 156 | + |
| 157 | +- **High log ingestion cost**: |
| 158 | + - Review your **deployment configuration** to filter out verbose logs or unneeded namespaces. |
| 159 | + - Check **log ingestion rate** in the dedicated dashboards for unusual spikes. |
| 160 | + |
| 161 | +## Further resources |
| 162 | + |
| 163 | +- [Observability Cockpit Overview](/cockpit/) |
| 164 | +- [Push logs to Cockpit (How-To)](/cockpit/how-to/send-metrics-logs-to-cockpit/) |
| 165 | +- [Send logs from your Kubernetes cluster to your Cockpit (How-To)](/cockpit/how-to/send-log-from-k8s-to-cockpit/) |
| 166 | +- [Send metrics from your Kubernetes cluster to your Cockpit (How-To)](/cockpit/how-to/send-metrics-from-k8s-to-cockpit/) |
| 167 | +- [Promtail Documentation](https://grafana.com/docs/loki/latest/clients/promtail/) |
| 168 | +- [Scaleway Kapsule Documentation](/kubernetes/kapsule/quickstart/) |
| 169 | +- [Scaleway Kosmos Documentation](/kubernetes/kosmos/quickstart/) |
0 commit comments