Skip to content

Commit e082b9f

Browse files
committed
add docs
1 parent 90e631f commit e082b9f

File tree

1 file changed

+11
-1
lines changed

1 file changed

+11
-1
lines changed

cmd/cli/docs/reference/docker_model_configure.yaml

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
command: docker model configure
22
short: Configure runtime options for a model
33
long: Configure runtime options for a model
4-
usage: docker model configure [--context-size=<n>] [--speculative-draft-model=<model>] [--hf_overrides=<json>] [--mode=<mode>] [--think] MODEL
4+
usage: docker model configure [--context-size=<n>] [--speculative-draft-model=<model>] [--hf_overrides=<json>] [--gpu-memory-utilization=<float>] [--mode=<mode>] [--think] MODEL
55
pname: docker model
66
plink: docker_model.yaml
77
options:
@@ -14,6 +14,16 @@ options:
1414
experimentalcli: false
1515
kubernetes: false
1616
swarm: false
17+
- option: gpu-memory-utilization
18+
value_type: float64
19+
description: |
20+
fraction of GPU memory to use for the model executor (0.0-1.0) - vLLM only
21+
deprecated: false
22+
hidden: false
23+
experimental: false
24+
experimentalcli: false
25+
kubernetes: false
26+
swarm: false
1727
- option: hf_overrides
1828
value_type: string
1929
description: HuggingFace model config overrides (JSON) - vLLM only

0 commit comments

Comments
 (0)