Skip to content

Commit 395baa7

Browse files
bene2k1bene2k1
committed
docs(k8s): add helm troubleshooting
1 parent 02e995f commit 395baa7

File tree

2 files changed

+138
-0
lines changed

2 files changed

+138
-0
lines changed

‎containers/kubernetes/troubleshooting/configuring-sql-storage-backend-helm.mdx‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
---
2+
meta:
3+
title: Configuring a SQL storage backend in Helm on Scaleway Kubernetes Kapsule
4+
description: Troubleshoot issues deploying large Helm releases on Scaleway Kubernetes Kapsule by configuring a SQL storage backend.
5+
content:
6+
h1: Configuring a SQL storage backend in Helm on Scaleway Kubernetes Kapsule
7+
paragraph: Troubleshoot issues deploying large Helm releases on Scaleway Kubernetes Kapsule by configuring a SQL storage backend.
8+
tags: kapsule helm sql backend
9+
dates:
10+
validation: 2024-11-22
11+
posted: 2024-11-22
12+
categories:
13+
- kubernetes
14+
---
15+
16+
When deploying large Helm releases on Scaleway Kapsule Kubernetes, you might encounter errors like the following:
17+
18+
```bash
19+
rpc error: code = ResourceExhausted desc = etcd quota exceeded: size 55000000/55000000, burst 424242/100000000
20+
```
21+
22+
This happens because Helm stores release information in Kubernetes **Secrets** or **ConfigMaps**, which are limited to 1MB due to **etcd** storage constraints.
23+
24+
To resolve this issue, you can configure Helm to use an SQL storage backend such as **PostgreSQL**, bypassing the etcd size limitations and enabling larger Helm releases.
25+
26+
<Macro id="requirements" />
27+
28+
- A Scaleway account logged into the [console](https://console.scaleway.com)
29+
- [Owner](/identity-and-access-management/iam/concepts/#owner) status or [IAM permissions](/identity-and-access-management/iam/concepts/#permission) allowing you to perform actions in the intended Organization
30+
- Created a [Kubernetes Kapsule cluster](/containers/kubernetes/how-to/create-cluster/)
31+
32+
## Setting up a PostgreSQL database
33+
34+
You can set up a PostgreSQL database using one of the two options below.
35+
36+
### Option A: Scaleway's managed PostgreSQL Service
37+
1. [Create a PostgreSQL instance](/managed-databases/postgresql-and-mysql/how-to/create-a-database/) using Scaleway's Managed Database service to create a PostgreSQL database.
38+
2. [Whitelist your cluster's IPs](/managed-databases/postgresql-and-mysql/how-to/manage-allowed-ip-addresses/) to allow your Kubernetes cluster to connect to the database.
39+
3. Note down the connection details. Save the host, port, database name, username, and password for later use.
40+
41+
### Option B: Deploy PostgreSQL in Kubernetes
42+
1. Deploy PostgreSQL using Helm in your cluster:
43+
```bash
44+
helm repo add bitnami https://charts.bitnami.com/bitnami
45+
helm install helm-postgres bitnami/postgresql \
46+
--set auth.username=helm,auth.password=yourpassword,auth.database=helm_db
47+
```
48+
2. [Expose](https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-intro/) the PostgreSQL service within the cluster.
49+
50+
<Message type="important">
51+
**Update security group rules**<br />
52+
When you create a Scaleway Kapsule cluster, a new security group blocks incoming traffic by default. To allow external traffic (if needed):
53+
54+
1. Navigate to **Compute > Security Groups** in the Scaleway console.
55+
2. Locate the security group associated with your cluster (e.g., `kubernetes-<cluster-id>`).
56+
3. Modify rules to allow:
57+
- TCP traffic on port `80` for HTTP (from `0.0.0.0/0`).
58+
- TCP traffic on port `443` for HTTPS (from `0.0.0.0/0`).
59+
</Message>
60+
61+
## Configuring Helm to use the SQL backend
62+
63+
To use PostgreSQL as Helm's storage backend, set the following environment variables in your terminal:
64+
```bash
65+
export HELM_DRIVER=sql
66+
export HELM_DRIVER_SQL_CONNECTION_STRING="postgresql://username:password@host:port/database"
67+
```
68+
<Message type="note">
69+
Replace `username`, `password`, `host`, `port`, and `database` with your PostgreSQL details.
70+
</Message>
71+
72+
Once set, Helm will store release metadata in the specified PostgreSQL database instead of Kubernetes Secrets or ConfigMaps.
73+
74+
## Verifying the configuration
75+
76+
To confirm Helm is configured correctly, run:
77+
```bash
78+
helm list --all-namespaces
79+
```
80+
81+
Expected Output:
82+
83+
- A list of Helm releases managed by the PostgreSQL backend.
84+
- If there are no releases, ensure the connection string is correct and PostgreSQL is reachable.
85+
86+
If configured correctly, Helm will use the SQL backend, and you will not encounter etcd size limitations.
87+
88+
## Migrating existing releases (if applicable)
89+
90+
Helm does not automatically migrate releases between storage backends. Follow these steps to manually migrate existing releases:
91+
92+
### Backup existing releases
93+
94+
- Run the following command to backup your existing releases:
95+
```bash
96+
kubectl get secrets --all-namespaces -l "owner=helm" -o yaml > helm-backup.yaml
97+
```
98+
99+
### Reinstall releases
100+
101+
- Redeploy the releases using the SQL backend:
102+
```bash
103+
helm upgrade --install <release-name> <chart-name> --namespace <namespace>
104+
```
105+
<Message type="note">
106+
Ensure you use the same release names and namespaces.
107+
</Message>
108+
109+
### Verify releases
110+
111+
- Run the following command to verify your releases:
112+
```bash
113+
helm list --all-namespaces
114+
```
115+
116+
### Clean up old secrets (after confirming everything works)
117+
118+
- Use the following command to delete your backup (after confirming everything works):
119+
```bash
120+
kubectl delete -f helm-backup.yaml
121+
```
122+
123+
## Troubleshooting common issues and solutions
124+
125+
| Issue | Solution |
126+
|-------------------------------------|--------------------------------------------------------------------------|
127+
| Connection timeout or authentication failures | Verify network connectivity, firewall rules, and database credentials. |
128+
| Errors persist after switching to SQL backend | Check `HELM_DRIVER` and `HELM_DRIVER_SQL_CONNECTION_STRING` variables. |
129+
| Missing previous releases | Releases in Secrets/`ConfigMaps` are not auto-migrated. Redeploy them. |
130+
131+
### Need further assistance?
132+
133+
Learn more about configuring Helm with an SQL storage backend by visiting the [official Helm documentation on SQL storage](https://helm.sh/docs/topics/plugins/helm-secrets/).<br />
134+
If you have further questions, feel free to engage with the [Scaleway Slack community](https://slack.scaleway.com) to exchange insights, share experiences, and discover practical solutions.

‎menu/navigation.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1744,6 +1744,10 @@
17441744
"label": "Exposing Kubernetes services to the internet",
17451745
"slug": "exposing-services"
17461746
},
1747+
{
1748+
"label": "Configuring a SQL storage backend in Helm",
1749+
"slug": "configuring-sql-stoage-backend-helm"
1750+
},
17471751
{
17481752
"label": "Modifying kernel parameters in a Kubernetes cluster using a DaemonSet",
17491753
"slug": "modifying-kernel-parameters-kubernetes-cluster"

0 commit comments

Comments
 (0)