Skip to content

Commit 34ff63d

Browse files
camdencheekcamdencheek
authored
Qdrant: add helm charts (#334)
1 parent c72d647 commit 34ff63d

File tree

9 files changed

+296
-5
lines changed

9 files changed

+296
-5
lines changed

‎TEST.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ helm template -f ./override.yaml sourcegraph charts/sourcegraph/.
6767
Perform a diff of the rendered helm manifests before and after your change. There're many ways to produce the diff:
6868

6969
- Run `helm template` before and after the change, then run `diff bundle.old.yaml bundle.new.yaml`.
70-
- Run `helm install` before the change, then run `helm diff` to inspecth the diff.
70+
- Run `helm install` before the change, then run `helm diff` to inspect the diff.
7171

7272
### Deploy the chart
7373

‎charts/sourcegraph/CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ Use `**BREAKING**:` to denote a breaking change
88

99
## Unreleased
1010

11+
- Added a service for the Qdrant vector database
12+
1113
## 5.1.6
1214

1315
- Sourcegraph 5.1.6 is now available!

‎charts/sourcegraph/README.md‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ In addition to the documented values, all services also support the following va
9090
| codeIntelDB.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `codeintel-db` |
9191
| codeIntelDB.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
9292
| codeIntelDB.storageSize | string | `"200Gi"` | PVC Storage Request for `codeintel-db` data volume |
93-
| embeddings.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":101,"runAsUser":100}` | Security context for the `worker` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
93+
| embeddings.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":101,"runAsUser":100}` | Security context for the `embeddings` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
9494
| embeddings.enabled | bool | `false` | Enable `embeddings` |
9595
| embeddings.env | object | `{}` | Environment variables for the `embeddings` container |
9696
| embeddings.extraVolumeMounts | object | `{}` | |
@@ -99,7 +99,7 @@ In addition to the documented values, all services also support the following va
9999
| embeddings.image.name | string | `"embeddings"` | Docker image name for the `embeddings` image |
100100
| embeddings.name | string | `"embeddings"` | Name of the `embeddings` service |
101101
| embeddings.podSecurityContext | object | `{}` | Security context for the `embeddings` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
102-
| embeddings.resources | object | `{"limits":{"cpu":"8","memory":"64G"},"requests":{"cpu":"4","memory":"32G"}}` | Resource requests & limits for the `worker` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
102+
| embeddings.resources | object | `{"limits":{"cpu":"8","memory":"64G"},"requests":{"cpu":"4","memory":"32G"}}` | Resource requests & limits for the `embeddings` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
103103
| embeddings.serviceAccount.annotations | object | `{}` | |
104104
| embeddings.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `embeddings` |
105105
| embeddings.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
@@ -264,6 +264,24 @@ In addition to the documented values, all services also support the following va
264264
| prometheus.serviceAccount.create | bool | `true` | Enable creation of ServiceAccount |
265265
| prometheus.serviceAccount.name | string | `"prometheus"` | Name of the ServiceAccount to be created or an existing ServiceAccount |
266266
| prometheus.storageSize | string | `"200Gi"` | PVC Storage Request for `prometheus` data volume |
267+
| qdrant.config | object | `{"debug":true,"log_level":"INFO"}` | Resource requests & limits for the `qdrant` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
268+
| qdrant.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"fsGroup":101,"runAsGroup":101,"runAsUser":100}` | Security context for the `qdrant` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
269+
| qdrant.enabled | bool | `false` | Enable `qdrant` |
270+
| qdrant.env | object | `{}` | Environment variables for the `qdrant` container |
271+
| qdrant.extraVolumeMounts | object | `{}` | |
272+
| qdrant.extraVolumes | object | `{}` | |
273+
| qdrant.image.defaultTag | string | `"239247_2023-08-18_5.1-433e1b1c997f@sha256:eafcd7af2aca699fa9c9ce8e6aa674cc0470441f794baf031296d5d1cdadd0bc"` | Docker image tag for the `embeddings` image |
274+
| qdrant.image.name | string | `"qdrant"` | Docker image name for the `embeddings` image |
275+
| qdrant.name | string | `"qdrant"` | Name of the `qdrant` service |
276+
| qdrant.podSecurityContext | object | `{"fsGroup":101,"fsGroupChangePolicy":"OnRootMismatch","runAsGroup":101,"runAsUser":100}` | Security context for the `qdrant` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
277+
| qdrant.resources.limits.cpu | string | `"2"` | |
278+
| qdrant.resources.limits.memory | string | `"8G"` | |
279+
| qdrant.resources.requests.cpu | string | `"500m"` | |
280+
| qdrant.resources.requests.memory | string | `"2G"` | |
281+
| qdrant.serviceAccount.annotations | object | `{}` | |
282+
| qdrant.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `embeddings` |
283+
| qdrant.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
284+
| qdrant.storageSize | string | `"100Gi"` | PVC Storage Request for `qdrant` data volume |
267285
| redisCache.connection.endpoint | string | `"redis-cache:6379"` | Endpoint to use for redis-cache. Supports either host:port or IANA specification |
268286
| redisCache.connection.existingSecret | string | `""` | Name of existing secret to use for Redis endpoint The secret must contain the key `endpoint` and should follow IANA specification learn more from the [Helm docs](https://docs.sourcegraph.com/admin/install/kubernetes/helm#using-external-redis-instances) |
269287
| redisCache.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":1000,"runAsUser":999}` | Security context for the `redis-cache` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |

‎charts/sourcegraph/templates/qdrant/qdrant.ConfigMap.yaml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
{{- if .Values.qdrant.enabled -}}
2+
apiVersion: v1
3+
kind: ConfigMap
4+
metadata:
5+
name: {{ .Values.qdrant.name }}
6+
labels:
7+
deploy: sourcegraph
8+
app.kubernetes.io/component: qdrant
9+
data:
10+
config.yaml: |
11+
debug: {{ .Values.qdrant.config.debug }}
12+
log_level: {{ .Values.qdrant.config.log_level }}
13+
storage:
14+
storage_path: /data
15+
snapshots_path: /data/storage
16+
on_disk_payload: true
17+
service:
18+
http_port: 6333
19+
grpc_port: 6334
20+
telemetry_disabled: true
21+
# The following parameters can be configured
22+
# on a per-collection basis, so these are just defaults.
23+
performance:
24+
max_optimization_threads: 4
25+
optimizers:
26+
max_optimization_threads: 4
27+
mmap_threshold_kb: 1
28+
indexing_threshold_kb: 0 # disable indexing
29+
hnsw_index:
30+
m: 8
31+
ef_construct: 100
32+
full_scan_threshold: 10
33+
max_indexing_threads: 4
34+
on_disk: true
35+
payload_m: 8
36+
{{- end }}

‎charts/sourcegraph/templates/qdrant/qdrant.PersistentVolumeClaim.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{{- if .Values.qdrant.enabled -}}
2+
apiVersion: v1
3+
kind: PersistentVolumeClaim
4+
metadata:
5+
labels:
6+
deploy: sourcegraph
7+
app.kubernetes.io/component: qdrant
8+
name: qdrant
9+
spec:
10+
accessModes:
11+
- ReadWriteOnce
12+
resources:
13+
requests:
14+
storage: {{ .Values.qdrant.storageSize }}
15+
storageClassName: {{ .Values.storageClass.name }}
16+
{{- if .Values.qdrant.volumeName }}
17+
volumeName: {{ .Values.qdrant.volumeName }}
18+
{{- end }}
19+
{{- end }}

‎charts/sourcegraph/templates/qdrant/qdrant.Service.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
{{- if .Values.qdrant.enabled -}}
2+
apiVersion: v1
3+
kind: Service
4+
metadata:
5+
annotations:
6+
sourcegraph.prometheus/scrape: "true"
7+
prometheus.io/port: "6333"
8+
{{- if .Values.qdrant.serviceAnnotations }}
9+
{{- toYaml .Values.qdrant.serviceAnnotations | nindent 4 }}
10+
{{- end }}
11+
labels:
12+
app: qdrant
13+
deploy: sourcegraph
14+
app.kubernetes.io/component: qdrant
15+
{{- if .Values.qdrant.serviceLabels }}
16+
{{- toYaml .Values.qdrant.serviceLabels | nindent 4 }}
17+
{{- end }}
18+
name: qdrant
19+
spec:
20+
ports:
21+
- name: http
22+
port: 6333
23+
targetPort: http
24+
- name: grpc
25+
port: 6334
26+
targetPort: grpc
27+
selector:
28+
{{- include "sourcegraph.selectorLabels" . | nindent 4 }}
29+
app: qdrant
30+
type: {{ .Values.qdrant.serviceType | default "ClusterIP" }}
31+
{{- end }}

‎charts/sourcegraph/templates/qdrant/qdrant.ServiceAccount.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{{- if and .Values.qdrant.enabled .Values.qdrant.serviceAccount.create -}}
2+
apiVersion: v1
3+
kind: ServiceAccount
4+
metadata:
5+
labels:
6+
category: rbac
7+
deploy: sourcegraph
8+
app.kubernetes.io/component: {{ .Values.qdrant.name }}
9+
{{- include "sourcegraph.serviceAccountAnnotations" (list . "qdrant") | trim | nindent 2 }}
10+
name: {{ include "sourcegraph.serviceAccountName" (list . "qdrant") }}
11+
{{- end }}

‎charts/sourcegraph/templates/qdrant/qdrant.StatefulSet.yaml‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
{{- if .Values.qdrant.enabled -}}
2+
apiVersion: apps/v1
3+
kind: StatefulSet
4+
metadata:
5+
name: {{ .Values.qdrant.name }}
6+
annotations:
7+
description: Backend for vector search operations.
8+
labels:
9+
{{- include "sourcegraph.labels" . | nindent 4 }}
10+
{{- if .Values.qdrant.labels }}
11+
{{- toYaml .Values.qdrant.labels | nindent 4 }}
12+
{{- end }}
13+
deploy: sourcegraph
14+
app.kubernetes.io/component: qdrant
15+
spec:
16+
minReadySeconds: 10
17+
replicas: 1
18+
revisionHistoryLimit: {{ .Values.sourcegraph.revisionHistoryLimit }}
19+
selector:
20+
matchLabels:
21+
{{- include "sourcegraph.selectorLabels" . | nindent 6 }}
22+
app: {{ .Values.qdrant.name }}
23+
strategy:
24+
type: Recreate
25+
template:
26+
metadata:
27+
annotations:
28+
kubectl.kubernetes.io/default-container: qdrant
29+
{{- if .Values.sourcegraph.podAnnotations }}
30+
{{- toYaml .Values.sourcegraph.podAnnotations | nindent 8 }}
31+
{{- end }}
32+
{{- if .Values.qdrant.podAnnotations }}
33+
{{- toYaml .Values.qdrant.podAnnotations | nindent 8 }}
34+
{{- end }}
35+
labels:
36+
{{- include "sourcegraph.selectorLabels" . | nindent 8 }}
37+
{{- if .Values.sourcegraph.podLabels }}
38+
{{- toYaml .Values.sourcegraph.podLabels | nindent 8 }}
39+
{{- end }}
40+
{{- if .Values.qdrant.podLabels }}
41+
{{- toYaml .Values.qdrant.podLabels | nindent 8 }}
42+
{{- end }}
43+
app: {{ .Values.qdrant.name }}
44+
app.kubernetes.io/component: qdrant
45+
deploy: sourcegraph
46+
spec:
47+
containers:
48+
- name: {{ .Values.qdrant.name }}
49+
image: {{ include "sourcegraph.image" (list . "qdrant") }}
50+
imagePullPolicy: {{ .Values.sourcegraph.image.pullPolicy }}
51+
terminationMessagePolicy: FallbackToLogsOnError
52+
env:
53+
{{- range $name, $item := .Values.qdrant.env }}
54+
- name: {{ $name }}
55+
{{- $item | toYaml | nindent 10 }}
56+
{{- end }}
57+
ports:
58+
- containerPort: 6333
59+
name: http
60+
protocol: TCP
61+
- containerPort: 6334
62+
name: grpc
63+
protocol: TCP
64+
# TODO: use gRPC liveness/readiness probe once this PR lands: https://github.com/qdrant/qdrant/pull/2409
65+
readinessProbe:
66+
failureThreshold: 3
67+
httpGet:
68+
scheme: HTTP
69+
port: http
70+
initialDelaySeconds: 0
71+
periodSeconds: 10
72+
successThreshold: 1
73+
timeoutSeconds: 1
74+
livenessProbe:
75+
failureThreshold: 3
76+
httpGet:
77+
scheme: HTTP
78+
port: http
79+
initialDelaySeconds: 0
80+
periodSeconds: 10
81+
successThreshold: 1
82+
timeoutSeconds: 1
83+
volumeMounts:
84+
- name: qdrant-data
85+
mountPath: /data
86+
- name: config
87+
mountPath: /etc/qdrant
88+
{{- if .Values.qdrant.extraVolumeMounts }}
89+
{{- toYaml .Values.qdrant.extraVolumeMounts | nindent 8 }}
90+
{{- end }}
91+
{{- if not .Values.sourcegraph.localDevMode}}
92+
resources:
93+
{{- toYaml .Values.qdrant.resources | nindent 10 }}
94+
{{- end }}
95+
securityContext:
96+
{{- toYaml .Values.qdrant.containerSecurityContext | nindent 10 }}
97+
{{- if .Values.blobstore.extraContainers }}
98+
{{- toYaml .Values.blobstore.extraContainers | nindent 6 }}
99+
{{- end }}
100+
securityContext:
101+
{{- toYaml .Values.qdrant.podSecurityContext | nindent 8 }}
102+
{{- include "sourcegraph.nodeSelector" (list . "qdrant" ) | trim | nindent 6 }}
103+
{{- include "sourcegraph.affinity" (list . "qdrant" ) | trim | nindent 6 }}
104+
{{- include "sourcegraph.tolerations" (list . "qdrant" ) | trim | nindent 6 }}
105+
{{- if .Values.qdrant.serviceAccount.create }}
106+
serviceAccountName: {{ .Values.qdrant.serviceAccount.name }}
107+
{{- end}}
108+
{{- with .Values.sourcegraph.imagePullSecrets }}
109+
imagePullSecrets:
110+
{{- toYaml . | nindent 8 }}
111+
{{- end }}
112+
volumes:
113+
{{- if .Values.qdrant.extraVolumes }}
114+
{{- toYaml .Values.qdrant.extraVolumes | nindent 6 }}
115+
{{- end }}
116+
- name: qdrant-data
117+
persistentVolumeClaim:
118+
claimName: qdrant
119+
- name: config
120+
configMap:
121+
name: qdrant
122+
items:
123+
- key: config.yaml
124+
path: config.yaml
125+
{{- end }}

‎charts/sourcegraph/values.yaml‎

Lines changed: 51 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -292,7 +292,7 @@ embeddings:
292292
name: "embeddings"
293293
# -- Docker image tag for the `embeddings` image
294294
defaultTag: "5.1.6@sha256:e849f52e38637882e5d2ba3d7d27a656d897c4b4e2905e1fdb843536d9c948ab"
295-
# -- Resource requests & limits for the `worker` container,
295+
# -- Resource requests & limits for the `embeddings` container,
296296
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
297297
resources:
298298
limits:
@@ -303,7 +303,7 @@ embeddings:
303303
memory: 32G
304304
# -- Environment variables for the `embeddings` container
305305
env: {}
306-
# -- Security context for the `worker` container,
306+
# -- Security context for the `embeddings` container,
307307
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container)
308308
containerSecurityContext:
309309
allowPrivilegeEscalation: false
@@ -322,6 +322,55 @@ embeddings:
322322
extraVolumeMounts: {}
323323
extraVolumes: {}
324324

325+
qdrant:
326+
# -- Enable `qdrant`
327+
enabled: false
328+
# -- Name of the `qdrant` service
329+
name: qdrant
330+
image:
331+
# -- Docker image name for the `embeddings` image
332+
name: "qdrant"
333+
# -- Docker image tag for the `embeddings` image
334+
defaultTag: "239247_2023-08-18_5.1-433e1b1c997f@sha256:eafcd7af2aca699fa9c9ce8e6aa674cc0470441f794baf031296d5d1cdadd0bc"
335+
# -- Resource requests & limits for the `qdrant` container,
336+
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
337+
config:
338+
debug: true
339+
log_level: INFO
340+
resources:
341+
limits:
342+
cpu: "2"
343+
memory: 8G
344+
requests:
345+
cpu: "500m"
346+
memory: 2G
347+
# -- Environment variables for the `qdrant` container
348+
env: {}
349+
# -- Security context for the `qdrant` container,
350+
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container)
351+
containerSecurityContext:
352+
allowPrivilegeEscalation: false
353+
runAsUser: 100
354+
runAsGroup: 101
355+
fsGroup: 101
356+
# -- Security context for the `qdrant` container,
357+
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container)
358+
podSecurityContext:
359+
runAsUser: 100
360+
runAsGroup: 101
361+
fsGroup: 101
362+
fsGroupChangePolicy: "OnRootMismatch"
363+
serviceAccount:
364+
# -- Enable creation of ServiceAccount for `embeddings`
365+
create: false
366+
# -- Name of the ServiceAccount to be created or an existing ServiceAccount
367+
name: ""
368+
annotations: {}
369+
extraVolumeMounts: {}
370+
extraVolumes: {}
371+
# -- PVC Storage Request for `qdrant` data volume
372+
storageSize: 100Gi
373+
325374
frontend:
326375
# -- Environment variables for the `frontend` container
327376
# @default -- the chart will add some default environment values

0 commit comments

Comments
 (0)