Skip to content

Commit 83f55b2

Browse files
robkooperlongshuicy
robkooper
and
longshuicy
authored
helmchart to deploy clowder2 in kubernetes (#119)
* start of helmchart * theme + host - add tar file of theme for keycloak - use single hostname * should not have been comitted * fix local.yaml * add elasticsearch_url * remove dependencies on /scripts folder; rewrite build error logging * fix elasticsearch url Co-authored-by: Chen Wang <[email protected]>
1 parent 68c87b7 commit 83f55b2

22 files changed

+907
-0
lines changed

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,3 +55,6 @@ dist
5555

5656
# keycloak mounted volume
5757
scripts/keycloak/data/*
58+
59+
# ignore clowder chart deps
60+
charts/clowder2/charts

‎charts/README.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Clowder 2
2+
3+
This depends on some subcharts, make sure to have them installed if you plan on modifying the helm chart:
4+
5+
```bash
6+
helm repo add bitnami https://charts.bitnami.com/bitnami
7+
helm dep build
8+
```
9+
10+
The chart has some common values, if you want to run this on your local machine you can use the following values (*.clowder2.ncsa.cloud is an DNS record that points to 127.0.0.1):
11+
12+
```yaml
13+
minio:
14+
ingress:
15+
hostname: minio.clowder2.ncsa.cloud
16+
apiIngress:
17+
hostname: minio.clowder2.ncsa.cloud
18+
19+
rabbitmq:
20+
ingress:
21+
hostname: rabbitmq.clowder2.ncsa.cloud
22+
```
23+
24+
Now you can install (or upgrade) clowder using:
25+
```bash
26+
helm upgrade --install --namespace clowder2 --create-namespace --values local.yaml clowder2 .
27+
```
28+
29+
30+
31+
# Docker Desktop
32+
33+
You will need an ingress controller, I like Traefik as my ingress controller. You install this with:
34+
35+
```bash
36+
helm install --namespace traefik --create-namespace traefik traefik/traefik
37+
```
38+
39+
If you want to enable the dashboard, you need the following file (dashboard.yaml):
40+
41+
```yaml
42+
apiVersion: traefik.containo.us/v1alpha1
43+
kind: IngressRoute
44+
metadata:
45+
name: dashboard
46+
spec:
47+
entryPoints:
48+
- web
49+
routes:
50+
- match: Host(`traefik.clowder2.ncsa.cloud`)
51+
kind: Rule
52+
services:
53+
- name: api@internal
54+
kind: TraefikService
55+
```
56+
57+
and apply it using:
58+
59+
```bash
60+
kubectl -n traefik apply -f dashboard.yaml
61+
```
62+

‎charts/clowder2/.helmignore‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/

‎charts/clowder2/Chart.lock‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
dependencies:
2+
- name: mongodb
3+
repository: https://charts.bitnami.com/bitnami
4+
version: 12.1.31
5+
- name: minio
6+
repository: https://charts.bitnami.com/bitnami
7+
version: 11.10.3
8+
- name: rabbitmq
9+
repository: https://charts.bitnami.com/bitnami
10+
version: 10.3.6
11+
- name: elasticsearch
12+
repository: https://charts.bitnami.com/bitnami
13+
version: 19.4.2
14+
- name: keycloak
15+
repository: https://charts.bitnami.com/bitnami
16+
version: 10.1.2
17+
digest: sha256:0c0e022885173885dfd6097d9a39dc22082515c41b93cda8cf9756a5d019a84f
18+
generated: "2022-09-30T15:19:12.308156-05:00"

‎charts/clowder2/Chart.yaml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
apiVersion: v2
2+
type: application
3+
4+
# Information about clowder.
5+
name: clowder2
6+
icon: https://isda.ncsa.illinois.edu/~kooper/charts/images/clowder.png
7+
home: https://clowderframework.org/
8+
description: >
9+
Open Source Data Management for Long Tail Data. Clowder is a customizable and
10+
scalable data management framework to support any data format and multiple
11+
research domains.
12+
13+
# This is the chart version. This version number should be incremented each time you make changes
14+
# to the chart and its templates, including the app version.
15+
# Versions are expected to follow Semantic Versioning (https://semver.org/)
16+
version: 0.1.0
17+
18+
# This is the version number of the application being deployed. This version number should be
19+
# incremented each time you make changes to the application. Versions are not expected to
20+
# follow Semantic Versioning. They should reflect the version the application is using.
21+
# It is recommended to use it with quotes.
22+
appVersion: "main"
23+
24+
# List of people that maintain this helm chart.
25+
maintainers:
26+
- name: Rob Kooper
27+
email: [email protected]
28+
- name: Chen Wang
29+
email: [email protected]
30+
31+
# location of source code
32+
sources:
33+
- https://github.com/clowder-framework/clowder2
34+
35+
# Dependencies for clowder. Some of the dependencies are only installed if they
36+
# are enabled.
37+
dependencies:
38+
- name: mongodb
39+
version: ~12
40+
repository: https://charts.bitnami.com/bitnami
41+
condition: mongodb.enabled
42+
- name: minio
43+
version: ~11
44+
repository: https://charts.bitnami.com/bitnami
45+
condition: minio.enabled
46+
- name: rabbitmq
47+
version: ~10
48+
repository: https://charts.bitnami.com/bitnami
49+
condition: rabbitmq.enabled
50+
- name: elasticsearch
51+
version: ~19
52+
repository: https://charts.bitnami.com/bitnami
53+
condition: elasticsearch.enabled
54+
- name: keycloak
55+
version: ~10
56+
repository: https://charts.bitnami.com/bitnami
57+
condition: keycloak.enabled

‎charts/clowder2/dashboard.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# dashboard.yaml
2+
apiVersion: traefik.containo.us/v1alpha1
3+
kind: IngressRoute
4+
metadata:
5+
name: dashboard
6+
spec:
7+
entryPoints:
8+
- web
9+
routes:
10+
- match: Host(`traefik.clowder2.ncsa.cloud`) && (PathPrefix(`/dashboard`) || PathPrefix(`/api`))
11+
kind: Rule
12+
services:
13+
- name: api@internal
14+
kind: TraefikService

‎charts/clowder2/files/clowder-theme.tgz‎

28.6 KB
Binary file not shown.

‎charts/clowder2/templates/NOTES.txt‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
1. Get the application URL by running these commands:
2+
{{- if .Values.ingress.enabled }}
3+
{{- range $host := .Values.ingress.hosts }}
4+
{{- range .paths }}
5+
http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
6+
{{- end }}
7+
{{- end }}
8+
{{- else if contains "NodePort" .Values.frontend.service.type }}
9+
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "clowder2.fullname" . }})
10+
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
11+
echo http://$NODE_IP:$NODE_PORT
12+
{{- else if contains "LoadBalancer" .Values.frontend.service.type }}
13+
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
14+
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "clowder2.fullname" . }}'
15+
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "clowder2.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
16+
echo http://$SERVICE_IP:{{ .Values.frontend.service.port }}
17+
{{- else if contains "ClusterIP" .Values.frontend.service.type }}
18+
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "clowder2.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
19+
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
20+
echo "Visit http://127.0.0.1:8080 to use your application"
21+
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
22+
{{- end }}

‎charts/clowder2/templates/_helpers.tpl‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
{{/*
2+
Expand the name of the chart.
3+
*/}}
4+
{{- define "clowder2.name" -}}
5+
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
6+
{{- end }}
7+
8+
{{/*
9+
Create a default fully qualified app name.
10+
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
11+
If release name contains chart name it will be used as a full name.
12+
*/}}
13+
{{- define "clowder2.fullname" -}}
14+
{{- if .Values.fullnameOverride }}
15+
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
16+
{{- else }}
17+
{{- $name := default .Chart.Name .Values.nameOverride }}
18+
{{- if contains $name .Release.Name }}
19+
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
20+
{{- else }}
21+
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
22+
{{- end }}
23+
{{- end }}
24+
{{- end }}
25+
26+
{{/*
27+
Create chart name and version as used by the chart label.
28+
*/}}
29+
{{- define "clowder2.chart" -}}
30+
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
31+
{{- end }}
32+
33+
{{/*
34+
Common labels
35+
*/}}
36+
{{- define "clowder2.labels" -}}
37+
helm.sh/chart: {{ include "clowder2.chart" . }}
38+
{{ include "clowder2.selectorLabels" . }}
39+
{{- if .Chart.AppVersion }}
40+
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
41+
{{- end }}
42+
app.kubernetes.io/managed-by: {{ .Release.Service }}
43+
{{- end }}
44+
45+
{{/*
46+
Selector labels
47+
*/}}
48+
{{- define "clowder2.selectorLabels" -}}
49+
app.kubernetes.io/name: {{ include "clowder2.name" . }}
50+
app.kubernetes.io/instance: {{ .Release.Name }}
51+
{{- end }}
52+
53+
{{/*
54+
Create the name of the service account to use
55+
*/}}
56+
{{- define "clowder2.serviceAccountName" -}}
57+
{{- if .Values.serviceAccount.create }}
58+
{{- default (include "clowder2.fullname" .) .Values.serviceAccount.name }}
59+
{{- else }}
60+
{{- default "default" .Values.serviceAccount.name }}
61+
{{- end }}
62+
{{- end }}

‎charts/clowder2/templates/backend/deployment.yaml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: {{ include "clowder2.fullname" . }}-backend
5+
labels:
6+
{{- include "clowder2.labels" . | nindent 4 }}
7+
spec:
8+
{{- if not .Values.autoscaling.enabled }}
9+
replicas: {{ .Values.replicaCount }}
10+
{{- end }}
11+
selector:
12+
matchLabels:
13+
app.kubernetes.io/name: {{ include "clowder2.name" . }}-backend
14+
app.kubernetes.io/instance: {{ .Release.Name }}
15+
template:
16+
metadata:
17+
{{- with .Values.podAnnotations }}
18+
annotations:
19+
{{- toYaml . | nindent 8 }}
20+
{{- end }}
21+
labels:
22+
app.kubernetes.io/name: {{ include "clowder2.name" . }}-backend
23+
app.kubernetes.io/instance: {{ .Release.Name }}
24+
spec:
25+
{{- with .Values.imagePullSecrets }}
26+
imagePullSecrets:
27+
{{- toYaml . | nindent 8 }}
28+
{{- end }}
29+
serviceAccountName: {{ include "clowder2.serviceAccountName" . }}
30+
securityContext:
31+
{{- toYaml .Values.podSecurityContext | nindent 8 }}
32+
containers:
33+
- name: backend
34+
securityContext:
35+
{{- toYaml .Values.securityContext | nindent 12 }}
36+
image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag | default .Chart.AppVersion }}"
37+
imagePullPolicy: {{ .Values.backend.image.pullPolicy }}
38+
env:
39+
- name: WEB_CONCURRENCY
40+
value: "4"
41+
- name: MINIO_SERVER_URL
42+
value: {{ include "clowder2.name" . }}-minio-headless:9000
43+
- name: MONGODB_URL
44+
value: mongodb://{{ include "clowder2.name" . }}-mongodb:27017
45+
- name: CLOWDER2_URL
46+
value: http://{{ .Values.hostname }}
47+
- name: auth_base
48+
value: $(CLOWDER2_URL)
49+
- name: auth_url
50+
value: $(CLOWDER2_URL)/keycloak/realms/clowder/protocol/openid-connect/auth?client_id=clowder2-backend&response_type=code
51+
- name: oauth2_scheme_auth_url
52+
value: http://{{ include "clowder2.name" .}}-keycloak-headless:8080/keycloak/realms/clowder/protocol/openid-connect/auth?client_id=clowder2-backend&response_type=code
53+
- name: auth_register_url
54+
value: $(CLOWDER2_URL)/keycloak/realms/clowder/protocol/openid-connect/registrations?client_id=clowder2-backend&response_type=code
55+
- name: auth_token_url
56+
value: http://{{ include "clowder2.name" .}}-keycloak-headless:8080/keycloak/realms/clowder/protocol/openid-connect/token
57+
- name: auth_server_url
58+
value: http://{{ include "clowder2.name" .}}-keycloak-headless:8080/keycloak/
59+
- name: keycloak_base
60+
value: $(CLOWDER2_URL)/api
61+
- name: frontend_url
62+
value: $(CLOWDER2_URL)
63+
- name: elasticsearch_url
64+
value: http://{{ include "clowder2.name" . }}-elasticsearch:9200
65+
ports:
66+
- name: http
67+
containerPort: 80
68+
protocol: TCP
69+
livenessProbe:
70+
httpGet:
71+
path: /
72+
port: http
73+
readinessProbe:
74+
httpGet:
75+
path: /
76+
port: http
77+
resources:
78+
{{- toYaml .Values.resources | nindent 12 }}
79+
{{- with .Values.nodeSelector }}
80+
nodeSelector:
81+
{{- toYaml . | nindent 8 }}
82+
{{- end }}
83+
{{- with .Values.affinity }}
84+
affinity:
85+
{{- toYaml . | nindent 8 }}
86+
{{- end }}
87+
{{- with .Values.tolerations }}
88+
tolerations:
89+
{{- toYaml . | nindent 8 }}
90+
{{- end }}

0 commit comments

Comments
 (0)