Skip to content

Commit f901d95

Browse files
Sam Johnsabeatrix
Sam Johns
and
abeatrix
authored
Update Searcher and Symbols to use StatefulSets (#242)
* update searcher and symbols to statefulSets with PVCs * remove deployment files for searcher and symbols * add spec.serviceName for searcher and symbols sts * update sts * Update charts/sourcegraph/templates/searcher/searcher.StatefulSet.yaml Remove gitserver comments Co-authored-by: Beatrix <[email protected]> * Update charts/sourcegraph/templates/symbols/symbols.StatefulSet.yaml remove gitserver comment Co-authored-by: Beatrix <[email protected]> * Update charts/sourcegraph/templates/symbols/symbols.StatefulSet.yaml remove gitserver comment Co-authored-by: Beatrix <[email protected]> * Update charts/sourcegraph/templates/searcher/searcher.StatefulSet.yaml remove gitserver comment Co-authored-by: Beatrix <[email protected]> * update env to set CACHE_SIZE_MB * update CACHE to be maxed at 90% of PVC size and change PVCs to use Gi instead of G * Add default values and handling for storageSize keys * Update README.md Remove ephemeral storage from searcher resources * Update README * Update README --------- Co-authored-by: Beatrix <[email protected]>
1 parent fab1b6d commit f901d95

File tree

5 files changed

+57
-34
lines changed

5 files changed

+57
-34
lines changed

‎charts/sourcegraph/CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Use `**BREAKING**:` to denote a breaking change
1414
- Sourcegraph 4.5.0 is now available
1515

1616
- Deploy Sourcegraph Executors now deployable via helm [#236](https://github.com/sourcegraph/deploy-sourcegraph-helm/pull/236).
17+
- Searcher and Symbols now use StatefulSets and PVCs to avoid large `ephermeralStorage` requests [#242](https://github.com/sourcegraph/deploy-sourcegraph-helm/pull/242)
1718

1819
## 4.4.2
1920

‎charts/sourcegraph/README.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -284,11 +284,12 @@ In addition to the documented values, all services also support the following va
284284
| searcher.image.defaultTag | string | `"4.4.2@sha256:62554970dfe2a82c82fb82fa6b8fb77f0acef2ce0d9760c8745433f48d8113be"` | Docker image tag for the `searcher` image |
285285
| searcher.image.name | string | `"searcher"` | Docker image name for the `searcher` image |
286286
| searcher.name | string | `"searcher"` | Name used by resources. Does not affect service names or PVCs. |
287-
| searcher.podSecurityContext | object | `{}` | Security context for the `searcher` pod, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) |
287+
| searcher.podSecurityContext | object | `{"fsGroup":101,"fsGroupChangePolicy":"OnRootMismatch","runAsUser":100}` | Security context for the `searcher` pod, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) |
288288
| searcher.replicaCount | int | `2` | Number of `searcher` pod |
289-
| searcher.resources | object | `{"limits":{"cpu":"2","ephemeral-storage":"26G","memory":"2G"},"requests":{"cpu":"500m","ephemeral-storage":"25G","memory":"500M"}}` | Resource requests & limits for the `searcher` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
289+
| searcher.resources | object | `{"limits":{"cpu":"2","memory":"2G"},"requests":{"cpu":"500m","memory":"500M"}}` | Resource requests & limits for the `searcher` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
290290
| searcher.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `searcher` |
291291
| searcher.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
292+
| searcher.storageSize | string | `"26Gi"` | |
292293
| sourcegraph.affinity | object | `{}` | Global Affinity, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) |
293294
| sourcegraph.image.defaultTag | string | `"{{ .Chart.AppVersion }}"` | Global docker image tag |
294295
| sourcegraph.image.pullPolicy | string | `"IfNotPresent"` | Global docker image pull policy |
@@ -314,11 +315,12 @@ In addition to the documented values, all services also support the following va
314315
| symbols.image.defaultTag | string | `"4.4.2@sha256:8c00918153e3e4e1155c19ba59f93cc889ffaaac6f8f7dc91ff2e5d610e1b740"` | Docker image tag for the `symbols` image |
315316
| symbols.image.name | string | `"symbols"` | Docker image name for the `symbols` image |
316317
| symbols.name | string | `"symbols"` | Name used by resources. Does not affect service names or PVCs. |
317-
| symbols.podSecurityContext | object | `{}` | Security context for the `symbols` pod, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) |
318+
| symbols.podSecurityContext | object | `{"fsGroup":101,"fsGroupChangePolicy":"OnRootMismatch","runAsUser":100}` | Security context for the `symbols` pod, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) |
318319
| symbols.replicaCount | int | `1` | Number of `symbols` pod |
319-
| symbols.resources | object | `{"limits":{"cpu":"2","ephemeral-storage":"12G","memory":"2G"},"requests":{"cpu":"500m","ephemeral-storage":"10G","memory":"500M"}}` | Resource requests & limits for the `symbols` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
320+
| symbols.resources | object | `{"limits":{"cpu":"2","memory":"2G"},"requests":{"cpu":"500m","memory":"500M"}}` | Resource requests & limits for the `symbols` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
320321
| symbols.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `symbols` |
321322
| symbols.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
323+
| symbols.storageSize | string | `"12Gi"` | |
322324
| syntectServer.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":101,"runAsUser":100}` | Security context for the `syntect-server` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
323325
| syntectServer.image.defaultTag | string | `"4.4.2@sha256:e02c9d64e431815bf51b05036463d60a2472f4a6ba8c3c475083c3b116ed0fe6"` | Docker image tag for the `syntect-server` image |
324326
| syntectServer.image.name | string | `"syntax-highlighter"` | Docker image name for the `syntect-server` image |

charts/sourcegraph/templates/searcher/searcher.Deployment.yaml renamed to charts/sourcegraph/templates/searcher/searcher.StatefulSet.yaml

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: apps/v1
2-
kind: Deployment
2+
kind: StatefulSet
33
metadata:
44
annotations:
55
description: Backend for text search operations.
@@ -19,10 +19,8 @@ spec:
1919
matchLabels:
2020
{{- include "sourcegraph.selectorLabels" . | nindent 6 }}
2121
app: searcher
22-
strategy:
23-
rollingUpdate:
24-
maxSurge: 1
25-
maxUnavailable: 1
22+
serviceName: searcher
23+
updateStrategy:
2624
type: RollingUpdate
2725
template:
2826
metadata:
@@ -59,11 +57,11 @@ spec:
5957
{{- $item | toYaml | nindent 10 }}
6058
{{- end }}
6159
- name: SEARCHER_CACHE_SIZE_MB
62-
valueFrom:
63-
resourceFieldRef:
64-
containerName: searcher
65-
resource: requests.ephemeral-storage
66-
divisor: 1M
60+
{{- if .Values.searcher.storageSize }}
61+
value: {{ trimSuffix "Gi" .Values.searcher.storageSize | mul 900 | quote }}
62+
{{- else }}
63+
value: "23400"
64+
{{- end }}
6765
- name: POD_NAME
6866
valueFrom:
6967
fieldRef:
@@ -89,7 +87,7 @@ spec:
8987
timeoutSeconds: 5
9088
volumeMounts:
9189
- mountPath: /mnt/cache
92-
name: cache-ssd
90+
name: cache
9391
- mountPath: /tmp
9492
name: tmpdir
9593
{{- if .Values.searcher.extraVolumeMounts }}
@@ -116,9 +114,19 @@ spec:
116114
{{- include "sourcegraph.renderServiceAccountName" (list . "searcher") | trim | nindent 6 }}
117115
volumes:
118116
- emptyDir: {}
119-
name: cache-ssd
117+
name: cache
120118
- emptyDir: {}
121119
name: tmpdir
122120
{{- if .Values.searcher.extraVolumes }}
123121
{{- toYaml .Values.searcher.extraVolumes | nindent 6 }}
124122
{{- end }}
123+
volumeClaimTemplates:
124+
- metadata:
125+
name: cache
126+
spec:
127+
accessModes:
128+
- ReadWriteOnce
129+
resources:
130+
requests:
131+
storage: {{ .Values.searcher.storageSize | default "26Gi" }}
132+
storageClassName: {{ .Values.storageClass.name }}

charts/sourcegraph/templates/symbols/symbols.Deployment.yaml renamed to charts/sourcegraph/templates/symbols/symbols.StatefulSet.yaml

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: apps/v1
2-
kind: Deployment
2+
kind: StatefulSet
33
metadata:
44
annotations:
55
description: Backend for symbols operations.
@@ -19,11 +19,9 @@ spec:
1919
matchLabels:
2020
{{- include "sourcegraph.selectorLabels" . | nindent 6 }}
2121
app: symbols
22-
strategy:
23-
rollingUpdate:
24-
maxSurge: 1
25-
maxUnavailable: 1
22+
updateStrategy:
2623
type: RollingUpdate
24+
serviceName: symbols
2725
template:
2826
metadata:
2927
annotations:
@@ -55,11 +53,11 @@ spec:
5553
{{- $item | toYaml | nindent 10 }}
5654
{{- end }}
5755
- name: SYMBOLS_CACHE_SIZE_MB
58-
valueFrom:
59-
resourceFieldRef:
60-
containerName: symbols
61-
resource: requests.ephemeral-storage
62-
divisor: 1M
56+
{{- if .Values.symbols.storageSize }}
57+
value: {{ trimSuffix "Gi" .Values.symbols.storageSize | mul 900 | quote }}
58+
{{- else }}
59+
value: "10800"
60+
{{- end }}
6361
- name: POD_NAME
6462
valueFrom:
6563
fieldRef:
@@ -97,7 +95,7 @@ spec:
9795
name: debug
9896
volumeMounts:
9997
- mountPath: /mnt/cache
100-
name: cache-ssd
98+
name: cache
10199
- mountPath: /mnt/tmp
102100
name: tmp
103101
{{- if .Values.symbols.extraVolumeMounts }}
@@ -124,9 +122,19 @@ spec:
124122
{{- include "sourcegraph.renderServiceAccountName" (list . "symbols") | trim | nindent 6 }}
125123
volumes:
126124
- emptyDir: {}
127-
name: cache-ssd
125+
name: cache
128126
- emptyDir: {}
129127
name: tmp
130128
{{- if .Values.symbols.extraVolumes }}
131129
{{- toYaml .Values.symbols.extraVolumes | nindent 6 }}
132130
{{- end }}
131+
volumeClaimTemplates:
132+
- metadata:
133+
name: cache
134+
spec:
135+
accessModes:
136+
- ReadWriteOnce
137+
resources:
138+
requests:
139+
storage: {{ .Values.symbols.storageSize | default "12Gi" }}
140+
storageClassName: {{ .Values.storageClass.name }}

‎charts/sourcegraph/values.yaml‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1035,7 +1035,10 @@ searcher:
10351035
readOnlyRootFilesystem: true
10361036
# -- Security context for the `searcher` pod,
10371037
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod)
1038-
podSecurityContext: {}
1038+
podSecurityContext:
1039+
runAsUser: 100
1040+
fsGroup: 101
1041+
fsGroupChangePolicy: "OnRootMismatch"
10391042
# -- Name used by resources. Does not affect service names or PVCs.
10401043
name: "searcher"
10411044
# -- Number of `searcher` pod
@@ -1045,17 +1048,16 @@ searcher:
10451048
resources:
10461049
limits:
10471050
cpu: "2"
1048-
ephemeral-storage: 26G
10491051
memory: 2G
10501052
requests:
10511053
cpu: 500m
1052-
ephemeral-storage: 25G
10531054
memory: 500M
10541055
serviceAccount:
10551056
# -- Enable creation of ServiceAccount for `searcher`
10561057
create: false
10571058
# -- Name of the ServiceAccount to be created or an existing ServiceAccount
10581059
name: ""
1060+
storageSize: 26Gi
10591061

10601062
storageClass:
10611063
# -- Enable creation of storageClass.
@@ -1093,7 +1095,10 @@ symbols:
10931095
readOnlyRootFilesystem: true
10941096
# -- Security context for the `symbols` pod,
10951097
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod)
1096-
podSecurityContext: {}
1098+
podSecurityContext:
1099+
runAsUser: 100
1100+
fsGroup: 101
1101+
fsGroupChangePolicy: "OnRootMismatch"
10971102
# -- Name used by resources. Does not affect service names or PVCs.
10981103
name: "symbols"
10991104
# -- Number of `symbols` pod
@@ -1103,17 +1108,16 @@ symbols:
11031108
resources:
11041109
limits:
11051110
cpu: "2"
1106-
ephemeral-storage: 12G
11071111
memory: 2G
11081112
requests:
11091113
cpu: 500m
1110-
ephemeral-storage: 10G
11111114
memory: 500M
11121115
serviceAccount:
11131116
# -- Enable creation of ServiceAccount for `symbols`
11141117
create: false
11151118
# -- Name of the ServiceAccount to be created or an existing ServiceAccount
11161119
name: ""
1120+
storageSize: 12Gi
11171121

11181122
syntectServer:
11191123
image:

0 commit comments

Comments
 (0)