Skip to content

Commit a49fce2

Browse files
marcleblanc2marcleblanc2
authored
Add priorityClassName to remaining pods (#778)
Linear issue [FEIE-297: Add `priorityClassName` to remaining pods](https://linear.app/sourcegraph/issue/FEIE-297/add-priorityclassname-to-remaining-pods) - Customer's Kubernetes cluster policy blocks pods from starting if they do not have a priorityClassName in their config. - We already had support for priorityClassName, but only for ~5 pods, need to add this for all remaining pods - Added logic so that priorityClassName could be defined once, under the `sourcegraph` top level key, and / or under each pod's top-level key, which would override the config on the `sourcegraph` top level key, so the customer could configure: ```yaml sourcegraph: priorityClassName: p2 pgsql: priorityClassName: p1 ``` ### Checklist - [x] Follow the [manual testing process](https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/TEST.md) - [ ] Update [changelog](https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/charts/sourcegraph/CHANGELOG.md) - [ ] Update [Kubernetes update doc](https://docs.sourcegraph.com/admin/updates/kubernetes) ### Test plan - Tested with Helm template - Followed the manual testing process - Deployed it on my test cluster, with the following override file: ```yaml priorityClasses: - name: test value: 100 preemptionPolicy: Never description: "test" - name: test2 value: 102 preemptionPolicy: Never description: "test2" sourcegraph: priorityClassName: test ``` - Then re-applied, adding: ```yaml gitserver: priorityClassName: test2 ``` - Both worked, output: ``` [2025-12-15 03:58:52] config % kubectl get pods -o custom-columns=NAME:.metadata.name,PRIORITY_CLASS:.spec.priorityClassName,PRIORITY_VALUE:.spec.priority NAME PRIORITY_CLASS PRIORITY_VALUE blobstore-579cbc4cb9-2gn69 test 100 codeinsights-db-0 test 100 codeintel-db-0 test 100 gitserver-0 test2 102 gitserver-1 test2 102 grafana-0 test 100 indexed-search-0 test 100 pgsql-0 test 100 precise-code-intel-worker-5b6bd8d898-9zrbg test 100 prometheus-65468d765d-j4rgw test 100 redis-cache-595c746f84-2wxtf test 100 redis-store-5f4b87dbf4-8n24m test 100 searcher-0 test 100 sourcegraph-frontend-677d647479-77zrl test 100 syntect-server-657b89b6f4-p59x6 test 100 worker-6d68db5b5c-twxkk test 100 ```
1 parent 604350a commit a49fce2

38 files changed

+142
-40
lines changed

‎TEST.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ helm plugin install https://github.com/helm-unittest/helm-unittest
2323
Once the plugin is installed, you can run the unit tests using the following:
2424

2525
```bash
26-
helm unittest --helm3 ./charts/sourcegraph/.
26+
helm unittest ./charts/sourcegraph
2727
```
2828

2929
We currently do not have testing best practices or require unit tests for new changes, so add test cases at your best judgement if possible.
@@ -59,7 +59,7 @@ Make sure you test both enabled and disabled toggles. For example, if you added
5959
You have two options to target specificy Sourcegraph version. Add the below to your `override.yaml`:
6060

6161
```yaml
62-
sourcegraph:
62+
sourcegraph:
6363
image:
6464
defaultTag: "6.10.0"
6565
useGlobalTagAsDefault: true

‎charts/sourcegraph-executor/dind/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ In addition to the documented values, the `executor` and `private-docker-registr
7979
| sourcegraph.nodeSelector | object | `{}` | NodeSelector, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) |
8080
| sourcegraph.podAnnotations | object | `{}` | Add extra annotations to attach to all pods |
8181
| sourcegraph.podLabels | object | `{}` | Add extra labels to attach to all pods |
82+
| sourcegraph.priorityClassName | string | `""` | Assign a priorityClass to all pods (daemonSets, deployments, and statefulSets) |
8283
| sourcegraph.tolerations | list | `[]` | Tolerations, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
8384
| storageClass.allowedTopologies | object | `{}` | Persistent volumes topology configuration, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/#allowed-topologies) |
8485
| storageClass.create | bool | `false` | Enable creation of storageClass. Defaults to Google Cloud Platform. Disable if you have your own existing storage class |

‎charts/sourcegraph-executor/dind/templates/_helpers/_priorityClassName.tpl‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{{/*
2+
3+
Allow customers to assign a priorityClassName to all resources which create pods (ex. DaemonSets, Deployments, StatefulSets)
4+
5+
Customers can configure an instance-wide default priorty class name at .Values.sourcegraph.priorityClassName,
6+
and can override it for individual services, if needed, at .Values.<service>.priorityClassName
7+
8+
*/}}
9+
10+
{{- define "sourcegraph.priorityClassName" -}}
11+
{{- $top := index . 0 }}
12+
{{- $service := index . 1 }}
13+
{{- $globalPriorityClassName := (index $top.Values "sourcegraph" "priorityClassName") }}
14+
{{- $servicePriorityClassName := (index $top.Values $service "priorityClassName") }}
15+
{{- if $servicePriorityClassName }}
16+
priorityClassName: {{ $servicePriorityClassName | toYaml | trim }}
17+
{{- else if $globalPriorityClassName }}
18+
priorityClassName: {{ $globalPriorityClassName | toYaml | trim }}
19+
{{- end }}
20+
{{- end }}

‎charts/sourcegraph-executor/dind/templates/executor/executor.Deployment.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,7 @@ spec:
130130
affinity:
131131
{{- toYaml . | nindent 8 }}
132132
{{- end }}
133+
{{- with include "sourcegraph.priorityClassName" (list . "executor") | trim }}{{ . | nindent 6 }}{{- end }}
133134
{{- with .Values.sourcegraph.tolerations }}
134135
tolerations:
135136
{{- toYaml . | nindent 8 }}

‎charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Deployment.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ spec:
7474
affinity:
7575
{{- toYaml . | nindent 8 }}
7676
{{- end }}
77+
{{- with include "sourcegraph.priorityClassName" (list . "privateDockerRegistry") | trim }}{{ . | nindent 6 }}{{- end }}
7778
{{- with .Values.sourcegraph.tolerations }}
7879
tolerations:
7980
{{- toYaml . | nindent 8 }}

‎charts/sourcegraph-executor/dind/values.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ sourcegraph:
3030
podAnnotations: {}
3131
# -- Add extra labels to attach to all pods
3232
podLabels: {}
33+
# -- Assign a priorityClass to all pods (daemonSets, deployments, and statefulSets)
34+
priorityClassName: ""
3335

3436

3537
storageClass:

‎charts/sourcegraph-executor/k8s/README.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -60,16 +60,16 @@ In addition to the documented values, the `executor` and `private-docker-registr
6060
| executor.extraEnv | string | `nil` | Sets extra environment variables on the executor deployment. See `values.yaml` for the format. |
6161
| executor.frontendExistingSecret | string | `""` | Name of existing k8s Secret to use for frontend password The name of the secret must match `executor.name`, i.e., the name of the helm release used to deploy the helm chart. The k8s Secret must contain the key `EXECUTOR_FRONTEND_PASSWORD` matching the site config `executors.accessToken` value. `executor.frontendPassword` is ignored if this is enabled. |
6262
| executor.frontendPassword | string | `""` | The shared secret configured in the Sourcegraph instance site config under executors.accessToken. Required if `executor.frontendExistingSecret`` is not configured. |
63-
| executor.frontendUrl | string | `""` | The external URL of the Sourcegraph instance. Required. **Recommended:** set to the internal service endpoint (e.g. `http://sourcegraph-frontend.sourcegraph.svc.cluster.local:30080` if Sourcegraph is deployed in the `sourcegraph` namespace). This will avoid unnecessary network charges as traffic will stay within the local network. |
63+
| executor.frontendUrl | string | `""` | The external URL of the Sourcegraph instance. Required. **Recommended:** set to the internal service endpoint (e.g. `http://sourcegraph-frontend.sourcegraph.svc.cluster.local:30080` if Sourcegraph is deployed in the `sourcegraph` namespace). This will avoid unnecessary network charges as traffic will stay within the local network. |
6464
| executor.image.defaultTag | string | `"6.0.0@sha256:6dc771a0c281a41ef676213f2f84a63d99045cf2e58d43022554a8022070ed65"` | |
6565
| executor.image.name | string | `"executor-kubernetes"` | |
6666
| executor.kubeconfigPath | string | `""` | The path to the kubeconfig file. If not specified, the in-cluster config is used. |
6767
| executor.kubernetesJob.deadline | string | `"1200"` | The number of seconds after which a Kubernetes job will be terminated. |
68-
| executor.kubernetesJob.fsGroup | string | `"1000"` | The group ID which is set on the job PVC file system. |
69-
| executor.kubernetesJob.node.name | string | `""` | The name of the Kubernetes Node to create job pods on. If not specified, the pods are created on the first available node. |
68+
| executor.kubernetesJob.fsGroup | string | `"1000"` | The group ID which is set on the job PVC file system. |
69+
| executor.kubernetesJob.node.name | string | `""` | The name of the Kubernetes Node to create job pods on. If not specified, the pods are created on the first available node. |
7070
| executor.kubernetesJob.node.requiredAffinityMatchExpressions | string | `""` | The JSON encoded required affinity match expressions for Kubernetes Jobs. e.g. '[{\"key\":\"foo\",\"operator\":\"In\",\"values\":[\"bar\"]}]' |
7171
| executor.kubernetesJob.node.requiredAffinityMatchFields | string | `""` | The JSON encoded required affinity match fields for Kubernetes Jobs. e.g. '[{\"key\":\"foo\",\"operator\":\"In\",\"values\":[\"bar\"]}]' |
72-
| executor.kubernetesJob.node.selector | string | `""` | A comma separated list of values to use as a node selector for Kubernetes Jobs. e.g. `foo=bar,app=my-app` |
72+
| executor.kubernetesJob.node.selector | string | `""` | A comma separated list of values to use as a node selector for Kubernetes Jobs. e.g. `foo=bar,app=my-app` |
7373
| executor.kubernetesJob.node.tolerations | string | `""` | The JSON encoded tolerations for Kubernetes Jobs. e.g. '[{\"key\":\"foo\",\"operator\":\"Equal\",\"value\":\"bar\",\"effect\":\"NoSchedule\"}]' |
7474
| executor.kubernetesJob.pod.affinity | string | `""` | The JSON encoded pod affinity for Kubernetes Jobs. e.g. '[{\"labelSelector\": {\"matchExpressions\": [{\"key\": \"foo\",\"operator\": \"In\",\"values\": [\"bar\"]}]},\"topologyKey\": \"kubernetes.io/hostname\"}]' |
7575
| executor.kubernetesJob.pod.antiAffinity | string | `""` | The JSON encoded pod anti-affinity for Kubernetes Jobs. e.g. '[{\"labelSelector\": {\"matchExpressions\": [{\"key\": \"foo\",\"operator\": \"In\",\"values\": [\"bar\"]}]},\"topologyKey\": \"kubernetes.io/hostname\"}]' |
@@ -108,6 +108,7 @@ In addition to the documented values, the `executor` and `private-docker-registr
108108
| sourcegraph.nodeSelector | object | `{}` | NodeSelector, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) |
109109
| sourcegraph.podAnnotations | object | `{}` | Add extra annotations to attach to all pods |
110110
| sourcegraph.podLabels | object | `{}` | Add extra labels to attach to all pods |
111+
| sourcegraph.priorityClassName | string | `""` | Assign a priorityClass to all pods (daemonSets, deployments, and statefulSets) |
111112
| sourcegraph.tolerations | list | `[]` | Tolerations, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
112113
| storageClass.allowedTopologies | object | `{}` | Persistent volumes topology configuration, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/#allowed-topologies) |
113114
| storageClass.create | bool | `false` | Enable creation of storageClass. Defaults to Google Cloud Platform. Disable if you have your own existing storage class |

‎charts/sourcegraph-executor/k8s/templates/_helpers/_priorityClassName.tpl‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{{/*
2+
3+
Allow customers to assign a priorityClassName to all resources which create pods (ex. DaemonSets, Deployments, StatefulSets)
4+
5+
Customers can configure an instance-wide default priorty class name at .Values.sourcegraph.priorityClassName,
6+
and can override it for individual services, if needed, at .Values.<service>.priorityClassName
7+
8+
*/}}
9+
10+
{{- define "sourcegraph.priorityClassName" -}}
11+
{{- $top := index . 0 }}
12+
{{- $service := index . 1 }}
13+
{{- $globalPriorityClassName := (index $top.Values "sourcegraph" "priorityClassName") }}
14+
{{- $servicePriorityClassName := (index $top.Values $service "priorityClassName") }}
15+
{{- if $servicePriorityClassName }}
16+
priorityClassName: {{ $servicePriorityClassName | toYaml | trim }}
17+
{{- else if $globalPriorityClassName }}
18+
priorityClassName: {{ $globalPriorityClassName | toYaml | trim }}
19+
{{- end }}
20+
{{- end }}

‎charts/sourcegraph-executor/k8s/templates/executor.Deployment.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ spec:
9999
affinity:
100100
{{- toYaml . | nindent 8 }}
101101
{{- end }}
102+
{{- with include "sourcegraph.priorityClassName" (list . "executor") | trim }}{{ . | nindent 6 }}{{- end }}
102103
{{- with .Values.executor.tolerations }}
103104
tolerations:
104105
{{- toYaml . | nindent 8 }}

‎charts/sourcegraph-executor/k8s/values.yaml‎

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ sourcegraph:
3232
podAnnotations: { }
3333
# -- Add extra labels to attach to all pods
3434
podLabels: { }
35+
# -- Assign a priorityClass to all pods (daemonSets, deployments, and statefulSets)
36+
priorityClassName: ""
3537

3638

3739
storageClass:
@@ -68,7 +70,7 @@ executor:
6870
requests:
6971
cpu: 500m
7072
memory: 200Mi
71-
# -- The external URL of the Sourcegraph instance. Required. **Recommended:** set to the internal service endpoint (e.g. `http://sourcegraph-frontend.sourcegraph.svc.cluster.local:30080` if Sourcegraph is deployed in the `sourcegraph` namespace).
73+
# -- The external URL of the Sourcegraph instance. Required. **Recommended:** set to the internal service endpoint (e.g. `http://sourcegraph-frontend.sourcegraph.svc.cluster.local:30080` if Sourcegraph is deployed in the `sourcegraph` namespace).
7274
# This will avoid unnecessary network charges as traffic will stay within the local network.
7375
frontendUrl: ""
7476
# -- Name of existing k8s Secret to use for frontend password
@@ -86,13 +88,13 @@ executor:
8688
maximumNumJobs: 10
8789
# - The maximum wall time that can be spent on a single job.
8890
maximumRuntimePerJob: "30m"
89-
91+
9092
log:
9193
# -- Possible values are `dbug`, `info`, `warn`, `eror`, `crit`.
9294
level: "warn"
9395
format: "condensed"
9496
trace: "false"
95-
97+
9698
# -- The storage size of the PVC attached to the executor deployment.
9799
storageSize: 10Gi
98100
# -- The namespace in which jobs are generated by the executor.
@@ -102,24 +104,24 @@ executor:
102104
# -- The containerSecurityContext for the executor image
103105
securityContext:
104106
# @default -- nil; accepts [0, 2147483647]
105-
runAsUser:
107+
runAsUser:
106108
# @default -- nil; accepts [0, 2147483647]
107-
runAsGroup:
109+
runAsGroup:
108110
# @default -- nil; accepts [0, 2147483647]
109111
fsGroup:
110112
# @default -- false; accepts [true, false]
111113
privileged: false
112-
114+
113115
kubernetesJob:
114116
# -- The number of seconds after which a Kubernetes job will be terminated.
115117
deadline: "1200"
116118
# -- (int) The user ID to run Kubernetes jobs as.
117119
# @default -- `nil`; accepts [0, 2147483647]
118-
runAsUser:
120+
runAsUser:
119121
# -- (int) The group ID to run Kubernetes jobs as.
120122
# @default -- `nil`; accepts [0, 2147483647]
121-
runAsGroup:
122-
# -- The group ID which is set on the job PVC file system.
123+
runAsGroup:
124+
# -- The group ID which is set on the job PVC file system.
123125
fsGroup: "1000"
124126
resources:
125127
requests:
@@ -132,11 +134,11 @@ executor:
132134
cpu: ""
133135
# -- The maximum memory for a job.
134136
memory: "12Gi"
135-
137+
136138
node:
137-
# -- The name of the Kubernetes Node to create job pods on. If not specified, the pods are created on the first available node.
139+
# -- The name of the Kubernetes Node to create job pods on. If not specified, the pods are created on the first available node.
138140
name: ""
139-
# -- A comma separated list of values to use as a node selector for Kubernetes Jobs. e.g. `foo=bar,app=my-app`
141+
# -- A comma separated list of values to use as a node selector for Kubernetes Jobs. e.g. `foo=bar,app=my-app`
140142
selector: ""
141143
# -- The JSON encoded tolerations for Kubernetes Jobs. e.g. '[{\"key\":\"foo\",\"operator\":\"Equal\",\"value\":\"bar\",\"effect\":\"NoSchedule\"}]'
142144
tolerations: ""
@@ -150,28 +152,28 @@ executor:
150152
affinity: ""
151153
# -- The JSON encoded pod anti-affinity for Kubernetes Jobs. e.g. '[{\"labelSelector\": {\"matchExpressions\": [{\"key\": \"foo\",\"operator\": \"In\",\"values\": [\"bar\"]}]},\"topologyKey\": \"kubernetes.io/hostname\"}]'
152154
antiAffinity: ""
153-
155+
154156
debug:
155157
# -- If true, Kubernetes jobs will not be deleted after they complete. Not recommended for production use as it can hit cluster limits.
156158
keepJobs: "false"
157159
keepWorkspaces: "false"
158-
160+
159161
# -- Affinity,
160162
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
161163
affinity: { }
162-
164+
163165
# -- NodeSelector,
164166
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector)
165167
nodeSelector: { }
166-
168+
167169
# -- Tolerations,
168170
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)
169171
tolerations: [ ]
170-
172+
171173
# -- Sets extra environment variables on the executor deployment. See `values.yaml` for the format.
172174
extraEnv:
173175
# - name: MY_ENV
174176
# value: my_value
175-
177+
176178
# -- For local deployments the host is 'host.docker.internal' and this needs to be true
177179
dockerAddHostGateway: "false"

0 commit comments

Comments
 (0)