Skip to content

Commit 416a234

Browse files
teoTomato-Lai
teo
and
Tomato-Lai
authored
feat: add PosixFS storage support
- Add PosixFS as alternative storage mode to S3 - Configure separate persistence for PosixFS volumes - Enable volume separation for system and user data - Tested successfully on Rackspace with 100GB user storage Co-authored-by: Teo Hoinaru <[email protected]>
1 parent f55a8b2 commit 416a234

File tree

11 files changed

+100
-9
lines changed

11 files changed

+100
-9
lines changed

‎charts/opencloud/README.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,14 @@ This will prepend `my-registry.com/` to all image references in the chart. For e
268268
| `opencloud.smtp.insecure` | SMTP insecure | `false` |
269269
| `opencloud.smtp.authentication` | SMTP authentication | `plain` |
270270
| `opencloud.smtp.encryption` | SMTP encryption | `starttls` |
271+
| `opencloud.storage.mode` | Choice between s3 and posixfs for user files | `s3` |
272+
273+
### OpenCloud S3 Storage Settings
274+
275+
The following options configure S3 for user file storage, either with the internal MinIO instance or with an external S3 provider.
276+
277+
| Parameter | Description | Default |
278+
| --------- | ----------- | ------- |
271279
| `opencloud.storage.s3.internal.enabled` | Enable internal MinIO instance | `true` |
272280
| `opencloud.storage.s3.internal.existingSecret` | Name of the existing secret | `` |
273281
| `opencloud.storage.s3.internal.rootUser` | MinIO root user | `opencloud` |
@@ -276,6 +284,7 @@ This will prepend `my-registry.com/` to all image references in the chart. For e
276284
| `opencloud.storage.s3.internal.region` | MinIO region | `default` |
277285
| `opencloud.storage.s3.internal.resources` | CPU/Memory resource requests/limits | See values.yaml |
278286
| `opencloud.storage.s3.internal.persistence.enabled` | Enable MinIO persistence | `true` |
287+
| `opencloud.storage.s3.internal.persistence.existingClaim` | Name of existing PVC instead of the settings below | `` |
279288
| `opencloud.storage.s3.internal.persistence.size` | Size of the MinIO persistent volume | `30Gi` |
280289
| `opencloud.storage.s3.internal.persistence.storageClass` | MinIO storage class | `""` |
281290
| `opencloud.storage.s3.internal.persistence.accessMode` | MinIO access mode | `ReadWriteOnce` |
@@ -288,6 +297,22 @@ This will prepend `my-registry.com/` to all image references in the chart. For e
288297
| `opencloud.storage.s3.external.bucket` | External S3 bucket | `""` |
289298
| `opencloud.storage.s3.external.createBucket` | Create bucket if it doesn't exist | `true` |
290299

300+
### OpenCloud PosixFS Storage Settings
301+
302+
The following options allow setting up a POSIX-compatible filesystem (such as NFS or CephFS) for user file storage instead of S3. This is useful for environments where object storage is not available or not desired.
303+
304+
| Parameter | Description | Default |
305+
| --------- | ----------- | ------- |
306+
| `opencloud.storage.posixfs.idCacheStore` | Cache store, between 'memory', 'redis-sentinel', 'nats-js-kv', 'noop' | `nats-js-kv` |
307+
| `opencloud.storage.posixfs.rootPath` | Path of storage root directory in openCloud pod | `/var/lib/opencloud/storage` |
308+
| `opencloud.storage.posixfs.persistence.enabled` | Enable persistence for PosixFS | `true` |
309+
| `opencloud.storage.posixfs.persistence.existingClaim` | Name of existing PVC instead of the settings below | `""` |
310+
| `opencloud.storage.posixfs.persistence.size` | Size of the PosixFS persistent volume | `30Gi` |
311+
| `opencloud.storage.posixfs.persistence.storageClass` | Storage class for PosixFS volume | `""` |
312+
| `opencloud.storage.posixfs.persistence.accessMode` | Access mode for PosixFS volume | `ReadWriteMany` |
313+
314+
**Note:** When using `posixfs` mode, ensure that the underlying storage supports the required access mode (e.g., `ReadWriteMany` for multiple replicas). The underlying filesystem must support `flock` and `xattrs` so for NFS the minimum version is 4.2.
315+
291316
### NATS Messaging Configuration
292317

293318
| Parameter | Description | Default |
@@ -435,7 +460,7 @@ The following HTTPRoutes are created when `httpRoute.enabled` is set to `true`:
435460
- Port: 8080
436461
- Headers: Adds Permissions-Policy header to prevent browser features like interest-based advertising
437462
438-
3. **MinIO HTTPRoute** (when `opencloud.storage.s3.internal.enabled` is `true`):
463+
3. **MinIO HTTPRoute** (when `opencloud.storage.mode` is `s3` and `opencloud.storage.s3.internal.enabled` is `true`):
439464
- Hostname: `global.domain.minio`
440465
- Service: `{{ release-name }}-minio`
441466
- Port: 9001

‎charts/opencloud/templates/gateway/gateway.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ spec:
6969
matchLabels:
7070
kubernetes.io/metadata.name: {{ .Values.httpRoute.gateway.namespace | default .Release.Namespace }}
7171
{{- end }}
72-
{{- if and .Values.opencloud.storage.s3.internal.enabled .Values.opencloud.storage.s3.internal.httpRoute.enabled }}
72+
{{- if and (eq .Values.opencloud.storage.mode "s3") .Values.opencloud.storage.s3.internal.enabled .Values.opencloud.storage.s3.internal.httpRoute.enabled }}
7373
{{- if .Values.global.tls.enabled }}
7474
- name: minio-https
7575
{{- else }}

‎charts/opencloud/templates/gateway/minio-httproute.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
{{- if and .Values.httpRoute.enabled .Values.opencloud.storage.s3.internal.enabled .Values.opencloud.storage.s3.internal.httpRoute.enabled }}
1+
{{- if and .Values.httpRoute.enabled (eq .Values.opencloud.storage.mode "s3") .Values.opencloud.storage.s3.internal.enabled .Values.opencloud.storage.s3.internal.httpRoute.enabled }}
22
apiVersion: gateway.networking.k8s.io/v1beta1
33
kind: HTTPRoute
44
metadata:

‎charts/opencloud/templates/minio/deployment.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
{{- if .Values.opencloud.storage.s3.internal.enabled }}
1+
{{- if and (eq .Values.opencloud.storage.mode "s3") .Values.opencloud.storage.s3.internal.enabled }}
22
apiVersion: apps/v1
33
kind: Deployment
44
metadata:

‎charts/opencloud/templates/minio/pvc.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
{{- if and .Values.opencloud.storage.s3.internal.enabled (and .Values.opencloud.storage.s3.internal.persistence.enabled (not .Values.opencloud.storage.s3.internal.persistence.existingClaim)) }}
1+
{{- if and (eq .Values.opencloud.storage.mode "s3") .Values.opencloud.storage.s3.internal.enabled (and .Values.opencloud.storage.s3.internal.persistence.enabled (not .Values.opencloud.storage.s3.internal.persistence.existingClaim)) }}
22
apiVersion: v1
33
kind: PersistentVolumeClaim
44
metadata:

‎charts/opencloud/templates/minio/secrets.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
{{- if and (not .Values.opencloud.storage.s3.internal.existingSecret) .Values.opencloud.storage.s3.internal.enabled }}
1+
{{- if and (eq .Values.opencloud.storage.mode "s3") (and (not .Values.opencloud.storage.s3.internal.existingSecret) .Values.opencloud.storage.s3.internal.enabled) }}
22
apiVersion: v1
33
kind: Secret
44
metadata:

‎charts/opencloud/templates/minio/service.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
{{- if .Values.opencloud.storage.s3.internal.enabled }}
1+
{{- if and (eq .Values.opencloud.storage.mode "s3") .Values.opencloud.storage.s3.internal.enabled }}
22
apiVersion: v1
33
kind: Service
44
metadata:

‎charts/opencloud/templates/opencloud/deployment.yaml‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -317,12 +317,21 @@ spec:
317317
# Storage configuration
318318
# Always use decomposeds3 for user storage and decomposed for system storage
319319
- name: STORAGE_USERS_DRIVER
320+
{{- if eq .Values.opencloud.storage.mode "posixfs" }}
321+
value: "posix"
322+
{{- else }}
320323
value: "decomposeds3"
324+
{{- end }}
321325
- name: STORAGE_SYSTEM_DRIVER
322326
value: "decomposed"
323327

328+
{{- if eq .Values.opencloud.storage.mode "posixfs" }}
329+
- name: STORAGE_USERS_POSIX_ROOT
330+
value: {{ .Values.opencloud.storage.posixfs.rootPath | quote }}
331+
- name: STORAGE_USERS_ID_CACHE_STORE
332+
value: {{ .Values.opencloud.storage.posixfs.idCacheStore | quote }}
324333
# S3 storage configuration
325-
{{- if .Values.opencloud.storage.s3.external.enabled }}
334+
{{- else if and (eq .Values.opencloud.storage.mode "s3") .Values.opencloud.storage.s3.external.enabled }}
326335
# External S3 storage
327336
- name: STORAGE_USERS_DECOMPOSEDS3_ENDPOINT
328337
value: {{ .Values.opencloud.storage.s3.external.endpoint | quote }}
@@ -397,6 +406,10 @@ spec:
397406
mountPath: /etc/opencloud
398407
- name: data
399408
mountPath: /var/lib/opencloud
409+
{{- if and (eq .Values.opencloud.storage.mode "posixfs") .Values.opencloud.storage.posixfs.persistence.enabled }}
410+
- name: posixfs
411+
mountPath: {{ .Values.opencloud.storage.posixfs.rootPath | default "/var/lib/opencloud/storage" | quote }}
412+
{{- end }}
400413
- name: config-json
401414
mountPath: /var/lib/opencloud/config.json
402415
subPath: config.json
@@ -439,6 +452,15 @@ spec:
439452
{{- else }}
440453
emptyDir: {}
441454
{{- end }}
455+
{{- if and (eq .Values.opencloud.storage.mode "posixfs") .Values.opencloud.storage.posixfs.persistence.enabled }}
456+
- name: posixfs
457+
persistentVolumeClaim:
458+
{{- if .Values.opencloud.storage.posixfs.persistence.existingClaim }}
459+
claimName: {{ .Values.opencloud.storage.posixfs.persistence.existingClaim | quote }}
460+
{{- else }}
461+
claimName: {{ include "opencloud.opencloud.fullname" . }}-posixfs
462+
{{- end }}
463+
{{- end }}
442464
- name: config-json
443465
configMap:
444466
name: {{ include "opencloud.opencloud.fullname" . }}-config-json

‎charts/opencloud/templates/opencloud/pvc.yaml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ spec:
2222
{{- end }}
2323
{{- end }}
2424
{{- end }}
25+
2526
{{- if and .Values.opencloud.enabled (and .Values.opencloud.persistence.data.enabled (not .Values.opencloud.persistence.data.existingClaim)) }}
2627
---
2728
apiVersion: v1
@@ -48,3 +49,28 @@ spec:
4849
{{- end }}
4950
{{- end }}
5051

52+
{{- if and (eq .Values.opencloud.storage.mode "posixfs") (and .Values.opencloud.storage.posixfs.persistence.enabled (not .Values.opencloud.storage.posixfs.persistence.existingClaim)) }}
53+
---
54+
apiVersion: v1
55+
kind: PersistentVolumeClaim
56+
metadata:
57+
name: {{ include "opencloud.opencloud.fullname" . }}-posixfs
58+
annotations:
59+
"helm.sh/resource-policy": "keep"
60+
labels:
61+
{{- include "opencloud.labels" . | nindent 4 }}
62+
app.kubernetes.io/component: opencloud
63+
spec:
64+
accessModes:
65+
- {{ .Values.opencloud.storage.posixfs.persistence.accessMode | quote }}
66+
resources:
67+
requests:
68+
storage: {{ .Values.opencloud.storage.posixfs.persistence.size | quote }}
69+
{{- if .Values.opencloud.storage.posixfs.persistence.storageClass }}
70+
{{- if (eq "-" .Values.opencloud.storage.posixfs.persistence.storageClass) }}
71+
storageClassName: ""
72+
{{- else }}
73+
storageClassName: {{ .Values.opencloud.storage.posixfs.persistence.storageClass | quote }}
74+
{{- end }}
75+
{{- end }}
76+
{{- end }}

‎charts/opencloud/templates/opencloud/secrets.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ stringData:
1919
adminPassword: {{ .Values.opencloud.adminPassword }}
2020
{{- end }}
2121
---
22-
{{- if and (not .Values.opencloud.storage.s3.external.existingSecret) .Values.opencloud.storage.s3.external.enabled }}
22+
{{- if and (eq .Values.opencloud.storage.mode "s3") (not .Values.opencloud.storage.s3.external.existingSecret) .Values.opencloud.storage.s3.external.enabled }}
2323
apiVersion: v1
2424
kind: Secret
2525
metadata:

0 commit comments

Comments
 (0)