Skip to content

Commit 96bd4ae

Browse files
committed
Try to set --tensor-parallel-size based on GPU count
1 parent c29f8ed commit 96bd4ae

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

charts/azimuth-llm/templates/api/deployment.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,10 @@ spec:
4747
{{- if .Values.api.extraArgs -}}
4848
{{- .Values.api.extraArgs | toYaml | nindent 10 }}
4949
{{- end -}}
50+
{{- if gt .Values.api.gpus 1 }}
51+
- --tensor-parallel-size
52+
- {{ .Values.api.gpus }}
53+
{{- end }}
5054
{{- if .Values.huggingface.secretName -}}
5155
envFrom:
5256
- secretRef:

0 commit comments

Comments
 (0)