Skip to content

Commit 9d7f3b7

Browse files
emmanuelmathotemmanuelmathot
authored
Refactor PostgreSQL configuration and remove deprecated database setup (#215)
* Refactor PostgreSQL configuration and remove deprecated database setup - Introduced a unified PostgreSQL configuration structure in values.yaml, replacing the old db configuration. - Added new helper functions for managing PostgreSQL environment variables and secrets based on the selected configuration type (postgrescluster, external-plaintext, external-secret). - Removed old database-related templates (ConfigMap, Deployment, PVC, Secrets, Service) that are no longer needed. - Updated the pgstacbootstrap job and configmap templates to align with the new PostgreSQL configuration. - Implemented validation for PostgreSQL settings to ensure required fields are set based on the selected type. * Add PostgreSQL host reader and writer environment variables, and include DATABASE_URL for connection string * Added a clarifying comment in values.yaml to explain that values in the external secret (host, port, database) will override the corresponding values defined in external.host, external.port, and external.database. Confirmed that the conditional blocks in deployment.yaml were already consolidated to eliminate redundancy. The file was already using a single include statement for PostgreSQL environment variables: env: {{- include "eoapi.postgresqlEnv" $ | nindent 12 }} Removed the unused eoapi.mapLegacyPostgresql helper function from _helpers.tpl as it wasn't being referenced anywhere in the codebase. * Refactor: Update comment for unified PostgreSQL configuration in values.yaml
1 parent 9e6e72b commit 9d7f3b7

File tree

10 files changed

+277
-267
lines changed

10 files changed

+277
-267
lines changed

‎helm-chart/eoapi/templates/_helpers.tpl‎

Lines changed: 228 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -62,18 +62,57 @@ Create the name of the service account to use
6262
{{- end }}
6363

6464
{{/*
65-
Secrets for postgres/postgis access have to be
66-
derived from what the crunchydata operator creates
65+
PostgreSQL environment variables based on the configured type
66+
*/}}
67+
{{- define "eoapi.postgresqlEnv" -}}
68+
{{- if eq .Values.postgresql.type "postgrescluster" }}
69+
{{- include "eoapi.postgresclusterSecrets" . }}
70+
{{- else if eq .Values.postgresql.type "external-plaintext" }}
71+
{{- include "eoapi.externalPlaintextPgSecrets" . }}
72+
{{- else if eq .Values.postgresql.type "external-secret" }}
73+
{{- include "eoapi.externalSecretPgSecrets" . }}
74+
{{- end }}
75+
{{- end }}
6776

68-
Also note that we want to use the pgbouncer-<port|host|uri>
69-
but currently it doesn't support `search_path` parameters
70-
(https://github.com/pgbouncer/pgbouncer/pull/73) which
71-
are required for much of *pgstac
77+
{{/*
78+
PostgreSQL cluster secrets
7279
*/}}
73-
{{- define "eoapi.pgstacSecrets" -}}
80+
{{- define "eoapi.postgresclusterSecrets" -}}
7481
{{- range $userName, $v := .Values.postgrescluster.users -}}
7582
{{/* do not render anything for the "postgres" user */}}
7683
{{- if not (eq (index $v "name") "postgres") }}
84+
# Standard PostgreSQL environment variables
85+
- name: PGUSER
86+
valueFrom:
87+
secretKeyRef:
88+
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
89+
key: user
90+
- name: PGPORT
91+
valueFrom:
92+
secretKeyRef:
93+
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
94+
key: port
95+
- name: PGHOST
96+
valueFrom:
97+
secretKeyRef:
98+
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
99+
key: host
100+
- name: PGPASSWORD
101+
valueFrom:
102+
secretKeyRef:
103+
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
104+
key: password
105+
- name: PGDATABASE
106+
valueFrom:
107+
secretKeyRef:
108+
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
109+
key: dbname
110+
- name: PGBOUNCER_URI
111+
valueFrom:
112+
secretKeyRef:
113+
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
114+
key: pgbouncer-uri
115+
# Legacy variables for backward compatibility
77116
- name: POSTGRES_USER
78117
valueFrom:
79118
secretKeyRef:
@@ -109,11 +148,6 @@ are required for much of *pgstac
109148
secretKeyRef:
110149
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
111150
key: dbname
112-
- name: PGBOUNCER_URI
113-
valueFrom:
114-
secretKeyRef:
115-
name: {{ $.Release.Name }}-pguser-{{ index $v "name" }}
116-
key: pgbouncer-uri
117151
- name: DATABASE_URL
118152
valueFrom:
119153
secretKeyRef:
@@ -128,6 +162,188 @@ are required for much of *pgstac
128162
key: uri
129163
{{- end }}
130164

165+
{{/*
166+
External PostgreSQL with plaintext credentials
167+
*/}}
168+
{{- define "eoapi.externalPlaintextPgSecrets" -}}
169+
# Standard PostgreSQL environment variables
170+
- name: PGUSER
171+
value: {{ .Values.postgresql.external.credentials.username | quote }}
172+
- name: PGPORT
173+
value: {{ .Values.postgresql.external.port | quote }}
174+
- name: PGHOST
175+
value: {{ .Values.postgresql.external.host | quote }}
176+
- name: PGPASSWORD
177+
value: {{ .Values.postgresql.external.credentials.password | quote }}
178+
- name: PGDATABASE
179+
value: {{ .Values.postgresql.external.database | quote }}
180+
# Legacy variables for backward compatibility
181+
- name: POSTGRES_USER
182+
value: {{ .Values.postgresql.external.credentials.username | quote }}
183+
- name: POSTGRES_PORT
184+
value: {{ .Values.postgresql.external.port | quote }}
185+
- name: POSTGRES_HOST
186+
value: {{ .Values.postgresql.external.host | quote }}
187+
- name: POSTGRES_HOST_READER
188+
value: {{ .Values.postgresql.external.host | quote }}
189+
- name: POSTGRES_HOST_WRITER
190+
value: {{ .Values.postgresql.external.host | quote }}
191+
- name: POSTGRES_PASS
192+
value: {{ .Values.postgresql.external.credentials.password | quote }}
193+
- name: POSTGRES_DBNAME
194+
value: {{ .Values.postgresql.external.database | quote }}
195+
- name: DATABASE_URL
196+
value: "postgresql://{{ .Values.postgresql.external.credentials.username }}:{{ .Values.postgresql.external.credentials.password }}@{{ .Values.postgresql.external.host }}:{{ .Values.postgresql.external.port }}/{{ .Values.postgresql.external.database }}"
197+
{{- end }}
198+
199+
{{/*
200+
External PostgreSQL with secret credentials
201+
*/}}
202+
{{- define "eoapi.externalSecretPgSecrets" -}}
203+
# Standard PostgreSQL environment variables
204+
- name: PGUSER
205+
valueFrom:
206+
secretKeyRef:
207+
name: {{ .Values.postgresql.external.existingSecret.name }}
208+
key: {{ .Values.postgresql.external.existingSecret.keys.username }}
209+
- name: PGPASSWORD
210+
valueFrom:
211+
secretKeyRef:
212+
name: {{ .Values.postgresql.external.existingSecret.name }}
213+
key: {{ .Values.postgresql.external.existingSecret.keys.password }}
214+
# Legacy variables for backward compatibility
215+
- name: POSTGRES_USER
216+
valueFrom:
217+
secretKeyRef:
218+
name: {{ .Values.postgresql.external.existingSecret.name }}
219+
key: {{ .Values.postgresql.external.existingSecret.keys.username }}
220+
- name: POSTGRES_PASS
221+
valueFrom:
222+
secretKeyRef:
223+
name: {{ .Values.postgresql.external.existingSecret.name }}
224+
key: {{ .Values.postgresql.external.existingSecret.keys.password }}
225+
226+
# Host, port, and database can be from the secret or from values
227+
{{- if .Values.postgresql.external.existingSecret.keys.host }}
228+
- name: PGHOST
229+
valueFrom:
230+
secretKeyRef:
231+
name: {{ .Values.postgresql.external.existingSecret.name }}
232+
key: {{ .Values.postgresql.external.existingSecret.keys.host }}
233+
- name: POSTGRES_HOST
234+
valueFrom:
235+
secretKeyRef:
236+
name: {{ .Values.postgresql.external.existingSecret.name }}
237+
key: {{ .Values.postgresql.external.existingSecret.keys.host }}
238+
- name: POSTGRES_HOST_READER
239+
valueFrom:
240+
secretKeyRef:
241+
name: {{ .Values.postgresql.external.existingSecret.name }}
242+
key: {{ .Values.postgresql.external.existingSecret.keys.host }}
243+
- name: POSTGRES_HOST_WRITER
244+
valueFrom:
245+
secretKeyRef:
246+
name: {{ .Values.postgresql.external.existingSecret.name }}
247+
key: {{ .Values.postgresql.external.existingSecret.keys.host }}
248+
{{- else }}
249+
- name: PGHOST
250+
value: {{ .Values.postgresql.external.host | quote }}
251+
- name: POSTGRES_HOST
252+
value: {{ .Values.postgresql.external.host | quote }}
253+
- name: POSTGRES_HOST_READER
254+
value: {{ .Values.postgresql.external.host | quote }}
255+
- name: POSTGRES_HOST_WRITER
256+
value: {{ .Values.postgresql.external.host | quote }}
257+
{{- end }}
258+
259+
{{- if .Values.postgresql.external.existingSecret.keys.port }}
260+
- name: PGPORT
261+
valueFrom:
262+
secretKeyRef:
263+
name: {{ .Values.postgresql.external.existingSecret.name }}
264+
key: {{ .Values.postgresql.external.existingSecret.keys.port }}
265+
- name: POSTGRES_PORT
266+
valueFrom:
267+
secretKeyRef:
268+
name: {{ .Values.postgresql.external.existingSecret.name }}
269+
key: {{ .Values.postgresql.external.existingSecret.keys.port }}
270+
{{- else }}
271+
- name: PGPORT
272+
value: {{ .Values.postgresql.external.port | quote }}
273+
- name: POSTGRES_PORT
274+
value: {{ .Values.postgresql.external.port | quote }}
275+
{{- end }}
276+
277+
{{- if .Values.postgresql.external.existingSecret.keys.database }}
278+
- name: PGDATABASE
279+
valueFrom:
280+
secretKeyRef:
281+
name: {{ .Values.postgresql.external.existingSecret.name }}
282+
key: {{ .Values.postgresql.external.existingSecret.keys.database }}
283+
- name: POSTGRES_DBNAME
284+
valueFrom:
285+
secretKeyRef:
286+
name: {{ .Values.postgresql.external.existingSecret.name }}
287+
key: {{ .Values.postgresql.external.existingSecret.keys.database }}
288+
{{- else }}
289+
- name: PGDATABASE
290+
value: {{ .Values.postgresql.external.database | quote }}
291+
- name: POSTGRES_DBNAME
292+
value: {{ .Values.postgresql.external.database | quote }}
293+
{{- end }}
294+
295+
# Add DATABASE_URL for connection string
296+
{{- if .Values.postgresql.external.existingSecret.keys.uri }}
297+
- name: DATABASE_URL
298+
valueFrom:
299+
secretKeyRef:
300+
name: {{ .Values.postgresql.external.existingSecret.name }}
301+
key: {{ .Values.postgresql.external.existingSecret.keys.uri }}
302+
{{- else }}
303+
- name: DATABASE_URL
304+
value: "postgresql://$(PGUSER):$(PGPASSWORD)@$(PGHOST):$(PGPORT)/$(PGDATABASE)"
305+
{{- end }}
306+
{{- end }}
307+
308+
{{/*
309+
Validate PostgreSQL configuration
310+
*/}}
311+
{{- define "eoapi.validatePostgresql" -}}
312+
{{- if eq .Values.postgresql.type "postgrescluster" }}
313+
{{- if not .Values.postgrescluster.enabled }}
314+
{{- fail "When postgresql.type is 'postgrescluster', postgrescluster.enabled must be true" }}
315+
{{- end }}
316+
{{- include "eoapi.validatePostgresCluster" . }}
317+
{{- else if eq .Values.postgresql.type "external-plaintext" }}
318+
{{- if not .Values.postgresql.external.host }}
319+
{{- fail "When postgresql.type is 'external-plaintext', postgresql.external.host must be set" }}
320+
{{- end }}
321+
{{- if not .Values.postgresql.external.credentials.username }}
322+
{{- fail "When postgresql.type is 'external-plaintext', postgresql.external.credentials.username must be set" }}
323+
{{- end }}
324+
{{- if not .Values.postgresql.external.credentials.password }}
325+
{{- fail "When postgresql.type is 'external-plaintext', postgresql.external.credentials.password must be set" }}
326+
{{- end }}
327+
{{- else if eq .Values.postgresql.type "external-secret" }}
328+
{{- if not .Values.postgresql.external.existingSecret.name }}
329+
{{- fail "When postgresql.type is 'external-secret', postgresql.external.existingSecret.name must be set" }}
330+
{{- end }}
331+
{{- if not .Values.postgresql.external.existingSecret.keys.username }}
332+
{{- fail "When postgresql.type is 'external-secret', postgresql.external.existingSecret.keys.username must be set" }}
333+
{{- end }}
334+
{{- if not .Values.postgresql.external.existingSecret.keys.password }}
335+
{{- fail "When postgresql.type is 'external-secret', postgresql.external.existingSecret.keys.password must be set" }}
336+
{{- end }}
337+
{{- if not .Values.postgresql.external.existingSecret.keys.host }}
338+
{{- if not .Values.postgresql.external.host }}
339+
{{- fail "When postgresql.type is 'external-secret' and existingSecret.keys.host is not set, postgresql.external.host must be set" }}
340+
{{- end }}
341+
{{- end }}
342+
{{- else }}
343+
{{- fail "postgresql.type must be one of: 'postgrescluster', 'external-plaintext', 'external-secret'" }}
344+
{{- end }}
345+
{{- end }}
346+
131347
{{/*
132348
values.schema.json doesn't play nice combined value checks
133349
so we use this helper function to check autoscaling rules
@@ -192,17 +408,3 @@ that you can only use traefik as ingress when `testing=true`
192408
{{- end -}}
193409
194410
{{- end -}}
195-
196-
{{/*
197-
validate:
198-
that you cannot have db.enabled and (postgrescluster.enabled or pgstacBootstrap.enabled)
199-
*/}}
200-
{{- define "eoapi.validateTempDB" -}}
201-
{{- if and (.Values.db.enabled) (.Values.postgrescluster.enabled) -}}
202-
{{- fail "you cannot use have both db.enabled and postgresclsuter.enabled" -}}
203-
{{- end -}}
204-
{{- if and (.Values.db.enabled) (.Values.pgstacBootstrap.enabled) -}}
205-
{{- fail "you cannot use have both db.enabled and pgstacBootstrap.enabled" -}}
206-
{{- end -}}
207-
208-
{{- end -}}

‎helm-chart/eoapi/templates/db/configmap.yaml‎

Lines changed: 0 additions & 38 deletions
This file was deleted.

0 commit comments

Comments
 (0)