Skip to content

Commit d385a7c

Browse files
rjouhannRomain Jouhannetggrosseveurgg
authored
Add helm-pg-redis: HA-ready PostgreSQL/Redis Helm presets for GitGuardian (#33)
* feat(helm-redis): Add examples to stand up helm and redis for GitGuardian * Remove scripts and add recommendations for cloud provider * Update helm-pg-redis/values/redis/standalone-large.yaml Co-authored-by: Gautier <[email protected]> * simplify readme * Apply suggestions from code review Co-authored-by: Gautier <[email protected]> * update common image for PG * update redis and formating * explain how to create the gim-replicated-registry secret * remove Customization * Address comments from Steve * (helm-pg-redis): common usage --------- Co-authored-by: Romain Jouhannet <[email protected]> Co-authored-by: Gautier <[email protected]> Co-authored-by: Gautier Grosseveur <[email protected]>
1 parent 6bad71d commit d385a7c

13 files changed

+449
-1
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,9 @@ Below is a brief overview of the tools available in this repository:
1010
| -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
1111
| [api-migration](./api-migration) | Facilitates the migration of incident remediation progress across different environments, including SaaS ↔ Self-Hosted, Self-Hosted ↔ Self-Hosted, and SaaS ↔ SaaS. |
1212
| [new-arch-migration](./new-arch-migration) | Assists in transitioning from the legacy GitGuardian architecture to the new architecture for Self-Hosted environments. |
13-
| [helm-preflights](./helm-preflights) | Ensures GitGuardian requirements are met prior installation or upgrade via [Helm on existing clusters](https://docs.gitguardian.com/self-hosting/installation/installation-existing-helm) by conducting tests from both the local user and the Kubernetes cluster. |
1413
| [honeytoken-tools](./honeytoken-tools) | Script to disseminate honeytokens in your repositories via Pull Requests |
1514
| [team-mapping-github-gitguardian](./team-mapping-github-gitguardian) | An example script using the GitHub and GitGuardian APIs to map GitHub Teams and the repositories they own to GitGuardian Teams and their perimeters. |
1615
| [team-mapping-gitlab-gitguardian](./team-mapping-gitlab-gitguardian) | An example script using the Gitlab and GitGuardian APIs to map Gitlab Groups and the repositories they own to GitGuardian Teams and their perimeters. |
1716
| [secret-key-rotation](./secret-key-rotation) | Rotate the Database Encryption Secret Key. |
17+
| [helm-preflights](./helm-preflights) | Ensures GitGuardian requirements are met prior installation or upgrade via [Helm on existing clusters](https://docs.gitguardian.com/self-hosting/installation/installation-existing-helm) by conducting tests from both the local user and the Kubernetes cluster. |
18+
| [helm-pg-redis](./helm-pg-redis) | Helm values and install commands to deploy PostgreSQL and Redis for GitGuardian (HA recommended). Includes small/medium/large presets aligned with [Scaling](https://docs.gitguardian.com/self-hosting/management/infrastructure-management/scaling). |

helm-pg-redis/README.md

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
### Helm: PostgreSQL and Redis for GitGuardian
2+
3+
This folder provides ready-to-use Helm configurations to deploy PostgreSQL and Redis in your Kubernetes cluster for use with the GitGuardian application.
4+
5+
- PostgreSQL topologies:
6+
- **HA (replication)**: primary with read replica(s); recommended for production for resilience and read scaling.
7+
- **Standalone**: single-primary setup; suitable for PoC or testing only.
8+
- Redis topology:
9+
- **Standalone** with persistence (aligned with the scaling guide). If you require Redis replication/sentinel, extend these values as needed.
10+
- All presets are available in three sizes aligned with the GitGuardian scaling guide: **small**, **medium**, **large**.
11+
12+
### Recommendation for cloud providers
13+
14+
If you deploy GitGuardian on a public cloud, prefer the provider's managed services for PostgreSQL and Redis instead of running them in-cluster:
15+
16+
- AWS: [Amazon RDS/Aurora (PostgreSQL)](https://docs.gitguardian.com/self-hosting/installation/databases/postgres-rds), [Amazon ElastiCache (Redis)](https://docs.gitguardian.com/self-hosting/installation/databases/redis-elasticache)
17+
- GCP: [Cloud SQL for PostgreSQL](https://docs.gitguardian.com/self-hosting/installation/databases/postgres-cloudsql), [Memorystore for Redis](https://docs.gitguardian.com/self-hosting/installation/databases/redis-memorystore)
18+
- Azure: [Azure Database for PostgreSQL](https://docs.gitguardian.com/self-hosting/installation/databases/postgres-azure), [Azure Cache for Redis](https://docs.gitguardian.com/self-hosting/installation/databases/redis-azure-cache)
19+
20+
These Helm values are intended for existing-cluster installations or environments where managed services are not available. Managed services typically offer higher availability, automated backups/maintenance, and operational SLAs.
21+
22+
⚠️ Deploying PostgreSQL/Redis using these Bitnami Helm examples on Red Hat OpenShift is not supported.
23+
24+
Presets are mapped as follows:
25+
26+
- PostgreSQL
27+
- Small: Primary 4 vCPU / 8 GiB / 200 Gi; Read replica 2 vCPU / 4 GiB / 200 Gi
28+
- Medium: Primary 8 vCPU / 32 GiB / 250 Gi; Read replica 4 vCPU / 16 GiB / 250 Gi
29+
- Large: Primary 16 vCPU / 64 GiB / 300 Gi; Read replica 8 vCPU / 32 GiB / 300 Gi
30+
- Redis (standalone)
31+
- Small: 2 vCPU / 2 GiB / 20 Gi
32+
- Medium: 4 vCPU / 8 GiB / 40 Gi
33+
- Large: 8 vCPU / 16 GiB / 100 Gi
34+
35+
For broader infrastructure guidance and context on these sizes, see the GitGuardian Scaling guide: [Scaling](https://docs.gitguardian.com/self-hosting/management/infrastructure-management/scaling).
36+
37+
You can customize storage classes, resource requests/limits, and replica counts by editing the values files or using `--set` overrides.
38+
39+
### Prerequisites
40+
41+
- Helm 3.x and `kubectl` configured against your target cluster
42+
- A default `StorageClass` or an explicit one you will set in values files
43+
- Cluster capacity matching the selected preset(s)
44+
- A Kubernetes namespace where PostgreSQL, Redis, and the GitGuardian application will be installed.
45+
- Example: `kubectl create ns gitguardian`
46+
- An image pull secret named `gim-replicated-registry` in `<namespace>` to download the PostgreSQL image:
47+
```bash
48+
LICENSE_ID="<your_licenseID>"
49+
NAMESPACE=<namespace>
50+
echo "{\"auths\": {\"proxy.replicated.com\": {\"auth\": \"$(echo -n \"${LICENSE_ID}:${LICENSE_ID}\" | base64)\"}, \"registry.replicated.com\": {\"auth\": \"$(echo -n \"${LICENSE_ID}:${LICENSE_ID}\" | base64)\"}}}" > ~/.docker/config.json
51+
kubectl -n $NAMESPACE create secret generic gim-replicated-registry \
52+
--from-file=.dockerconfigjson=$HOME/.docker/config.json \
53+
--type=kubernetes.io/dockerconfigjson
54+
```
55+
- If you need help obtaining your LICENSE_ID, contact support at [email protected].
56+
57+
### Quick start
58+
59+
Use the commands below to install Bitnami charts with the preset values files. Replace `<namespace>` accordingly.
60+
61+
1) Add Bitnami repo (if not already added) and create/use a namespace:
62+
63+
```bash
64+
helm repo add bitnami https://charts.bitnami.com/bitnami && helm repo update
65+
NAMESPACE=<namespace>
66+
kubectl get ns "$NAMESPACE" >/dev/null 2>&1 || kubectl create ns "$NAMESPACE"
67+
```
68+
69+
2) Optional: customize a values file
70+
71+
Edit the YAML presets under `values/postgres/` (and `values/redis/` later) to fit your environment. Typical edits include `auth.username`, `auth.database`, `persistence.storageClass`, `persistence.size`, and `resources`.
72+
73+
3) Install PostgreSQL (choose topology and size):
74+
75+
```bash
76+
# Standalone (PoC/testing) - small preset
77+
helm upgrade --install pg bitnami/postgresql \
78+
-n "$NAMESPACE" \
79+
-f helm-pg-redis/values/postgres/common.yaml \
80+
-f helm-pg-redis/values/postgres/standalone-small.yaml \
81+
--wait
82+
83+
# HA (recommended for production) - medium preset
84+
helm upgrade --install pg bitnami/postgresql \
85+
-n "$NAMESPACE" \
86+
-f helm-pg-redis/values/postgres/common.yaml \
87+
-f helm-pg-redis/values/postgres/ha-medium.yaml \
88+
--wait
89+
```
90+
91+
4) Install Redis (choose size):
92+
93+
```bash
94+
# Standalone - small preset
95+
helm upgrade --install redis bitnami/redis \
96+
-n "$NAMESPACE" \
97+
-f helm-pg-redis/values/redis/common.yaml \
98+
-f helm-pg-redis/values/redis/standalone-small.yaml \
99+
--wait
100+
101+
# Standalone - large preset
102+
helm upgrade --install redis bitnami/redis \
103+
-n "$NAMESPACE" \
104+
-f helm-pg-redis/values/redis/common.yaml \
105+
-f helm-pg-redis/values/redis/standalone-large.yaml \
106+
--wait
107+
```
108+
109+
5) Retrieve credentials and assemble connection strings:
110+
111+
```bash
112+
NAMESPACE="$NAMESPACE"
113+
114+
# PostgreSQL
115+
PG_RELEASE=pg
116+
PG_PRIMARY_SERVICE="$PG_RELEASE-postgresql"
117+
PG_PASSWORD=$(kubectl get secret -n "$NAMESPACE" "$PG_RELEASE-postgresql" -o jsonpath='{.data.postgres-password}' | base64 -d)
118+
# Optional app user password if set via auth.password
119+
PG_APP_PASSWORD=$(kubectl get secret -n "$NAMESPACE" "$PG_RELEASE-postgresql" -o jsonpath='{.data.password}' 2>/dev/null | base64 -d || true)
120+
121+
echo "PostgreSQL host: $PG_PRIMARY_SERVICE.$NAMESPACE.svc.cluster.local:5432"
122+
echo "PostgreSQL postgres user password: $PG_PASSWORD"
123+
echo "PostgreSQL app user password (if configured): ${PG_APP_PASSWORD:-<not-set>}"
124+
125+
# Redis
126+
REDIS_RELEASE=redis
127+
REDIS_SERVICE="$REDIS_RELEASE-redis-master" # standalone/replication master service
128+
REDIS_PASSWORD=$(kubectl get secret -n "$NAMESPACE" "$REDIS_RELEASE-redis" -o jsonpath='{.data.redis-password}' | base64 -d)
129+
130+
echo "Redis host: $REDIS_SERVICE.$NAMESPACE.svc.cluster.local:6379"
131+
echo "Redis password: $REDIS_PASSWORD"
132+
```
133+
134+
### Using with GitGuardian
135+
136+
Provide these connection details to the GitGuardian application (via your Helm values for the GitGuardian chart, KOTS config, or environment):
137+
138+
For deploying GitGuardian in an existing Kubernetes cluster using Helm, follow the official guide: [Helm-based installation](https://docs.gitguardian.com/self-hosting/installation/installation-existing-helm).
139+
140+
- PostgreSQL
141+
- Note: The values below are example defaults and can be customized via `auth.username`, `auth.database`, and optionally `auth.password` in the values files.
142+
- Host: `<pg-release>-postgresql.<namespace>.svc.cluster.local`
143+
- Port: `5432`
144+
- Database: `gitguardian-db` (if using the provided values)
145+
- Username: `gitguardian-user` (if using the provided values) or `postgres`
146+
- Password: from secret above
147+
- Example URL: `postgresql://gitguardian-user:<password>@<host>:5432/gitguardian-db`
148+
149+
- Redis
150+
- Host: `<redis-release>-redis-master.<namespace>.svc.cluster.local`
151+
- Port: `6379`
152+
- Password: from secret above
153+
- Example URL: `redis://:<password>@<host>:6379/0`
154+
155+
If you use the PostgreSQL HA presets, the read service is also available (Bitnami creates a `-read` service for replicas). GitGuardian generally needs a single primary endpoint for writes; configure read/write splitting only if supported and desired within your environment.
156+
157+
### Uninstall
158+
159+
Replace $NAMESPACE accordingly.
160+
161+
```bash
162+
helm uninstall pg -n $NAMESPACE || true
163+
helm uninstall redis -n $NAMESPACE || true
164+
```
165+
166+
This deletes only the Helm releases. PersistentVolumes may remain depending on your `reclaimPolicy` and release settings.
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Common PostgreSQL image and initContainers configuration shared by all presets
2+
global:
3+
security:
4+
allowInsecureImages: true
5+
6+
auth:
7+
username: gitguardian-user
8+
database: gitguardian-db
9+
## password will be autogenerated and stored in a Secret
10+
11+
image:
12+
registry: proxy.replicated.com/proxy/gitguardian/513715405986.dkr.ecr.us-west-2.amazonaws.com
13+
repository: services/postgres-pgvector
14+
tag: '16-v0.8.0'
15+
pullSecrets:
16+
- 'gim-replicated-registry'
17+
18+
initContainers:
19+
- name: extensions
20+
image: proxy.replicated.com/proxy/gitguardian/513715405986.dkr.ecr.us-west-2.amazonaws.com/services/postgres-pgvector:16-v0.8.0
21+
imagePullPolicy: '{{ $.Values.image.pullPolicy }}'
22+
imagePullSecrets: 'gim-replicated-registry'
23+
restartPolicy: Always
24+
securityContext:
25+
allowPrivilegeEscalation: false
26+
capabilities:
27+
drop:
28+
- ALL
29+
privileged: false
30+
readOnlyRootFilesystem: true
31+
runAsGroup: 1001
32+
runAsNonRoot: true
33+
runAsUser: 1001
34+
seLinuxOptions: {}
35+
seccompProfile:
36+
type: RuntimeDefault
37+
resources:
38+
requests:
39+
cpu: 10m
40+
memory: 64Mi
41+
command:
42+
- /bin/sh
43+
- -c
44+
- |
45+
until pg_isready; do
46+
echo "Waiting for PostgreSQL..."
47+
sleep 3
48+
done
49+
echo "Creating pgvector extension..."
50+
psql -c "CREATE EXTENSION IF NOT EXISTS vector;"
51+
echo "pgvector setup done."
52+
echo "Sleeping..."
53+
sleep infinity
54+
env:
55+
- name: PGHOST
56+
value: 'postgresql'
57+
- name: PGPORT
58+
value: 5432
59+
- name: PGUSER
60+
value: postgres
61+
- name: PGPASSWORD
62+
valueFrom:
63+
secretKeyRef:
64+
name: 'postgresql'
65+
key: 'postgres-password'
66+
- name: PGDATABASE
67+
value: gitguardian-db
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
architecture: replication
2+
3+
primary:
4+
persistence:
5+
enabled: true
6+
size: 300Gi
7+
resources:
8+
requests:
9+
cpu: "16000m"
10+
memory: "64Gi"
11+
limits:
12+
cpu: "16000m"
13+
memory: "64Gi"
14+
extendedConfiguration: |-
15+
max_connections = 500
16+
shared_buffers = 128MB
17+
timezone = 'UTC'
18+
idle_in_transaction_session_timeout = '1h'
19+
20+
readReplicas:
21+
replicaCount: 1
22+
persistence:
23+
enabled: true
24+
size: 300Gi
25+
resources:
26+
requests:
27+
cpu: "8000m"
28+
memory: "32Gi"
29+
limits:
30+
cpu: "8000m"
31+
memory: "32Gi"
32+
extendedConfiguration: |-
33+
max_connections = 500
34+
shared_buffers = 128MB
35+
timezone = 'UTC'
36+
idle_in_transaction_session_timeout = '1h'
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
architecture: replication
2+
3+
primary:
4+
persistence:
5+
enabled: true
6+
size: 250Gi
7+
resources:
8+
requests:
9+
cpu: "8000m"
10+
memory: "32Gi"
11+
limits:
12+
cpu: "8000m"
13+
memory: "32Gi"
14+
extendedConfiguration: |-
15+
max_connections = 500
16+
shared_buffers = 128MB
17+
timezone = 'UTC'
18+
idle_in_transaction_session_timeout = '1h'
19+
20+
readReplicas:
21+
replicaCount: 1
22+
persistence:
23+
enabled: true
24+
size: 250Gi
25+
resources:
26+
requests:
27+
cpu: "4000m"
28+
memory: "16Gi"
29+
limits:
30+
cpu: "4000m"
31+
memory: "16Gi"
32+
extendedConfiguration: |-
33+
max_connections = 500
34+
shared_buffers = 128MB
35+
timezone = 'UTC'
36+
idle_in_transaction_session_timeout = '1h'
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
architecture: replication
2+
3+
primary:
4+
persistence:
5+
enabled: true
6+
size: 200Gi
7+
resources:
8+
requests:
9+
cpu: "4000m"
10+
memory: "8Gi"
11+
limits:
12+
cpu: "4000m"
13+
memory: "8Gi"
14+
extendedConfiguration: |-
15+
max_connections = 500
16+
shared_buffers = 128MB
17+
timezone = 'UTC'
18+
idle_in_transaction_session_timeout = '1h'
19+
20+
readReplicas:
21+
replicaCount: 1
22+
persistence:
23+
enabled: true
24+
size: 200Gi
25+
resources:
26+
requests:
27+
cpu: "2000m"
28+
memory: "4Gi"
29+
limits:
30+
cpu: "2000m"
31+
memory: "4Gi"
32+
extendedConfiguration: |-
33+
max_connections = 500
34+
shared_buffers = 128MB
35+
timezone = 'UTC'
36+
idle_in_transaction_session_timeout = '1h'
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
architecture: standalone
2+
3+
primary:
4+
persistence:
5+
enabled: true
6+
size: 300Gi
7+
resources:
8+
requests:
9+
cpu: "16000m"
10+
memory: "64Gi"
11+
limits:
12+
cpu: "16000m"
13+
memory: "64Gi"
14+
extendedConfiguration: |-
15+
max_connections = 500
16+
shared_buffers = 128MB
17+
timezone = 'UTC'
18+
idle_in_transaction_session_timeout = '1h'
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
architecture: standalone
2+
3+
primary:
4+
persistence:
5+
enabled: true
6+
size: 250Gi
7+
resources:
8+
requests:
9+
cpu: "8000m"
10+
memory: "32Gi"
11+
limits:
12+
cpu: "8000m"
13+
memory: "32Gi"
14+
extendedConfiguration: |-
15+
max_connections = 500
16+
shared_buffers = 128MB
17+
timezone = 'UTC'
18+
idle_in_transaction_session_timeout = '1h'

0 commit comments

Comments
 (0)