Skip to content

Commit 73b89b6

Browse files
authored
Merge pull request docker#110 from ArthurFlag/ENGDOCS-2787-update-model-pull-docs
docs: note about quantized models
2 parents cf6c379 + 0ab9371 commit 73b89b6

File tree

2 files changed

+10
-0
lines changed

2 files changed

+10
-0
lines changed

cmd/cli/docs/reference/docker_model_pull.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@ examples: |-
1616
1717
You can pull GGUF models directly from [Hugging Face](https://huggingface.co/models?library=gguf).
1818
19+
**Note about quantization:** If no tag is specified, the command tries to pull the `Q4_K_M` version of the model.
20+
If `Q4_K_M` doesn't exist, the command pulls the first GGUF found in the **Files** view of the model on HuggingFace.
21+
To specify the quantization, provide it as a tag, for example:
22+
`docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q4_K_S`
23+
1924
```console
2025
docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
2126
```

cmd/cli/docs/reference/model_pull.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,11 @@ docker model pull ai/smollm2
2222

2323
You can pull GGUF models directly from [Hugging Face](https://huggingface.co/models?library=gguf).
2424

25+
**Note about quantization:** If no tag is specified, the command tries to pull the `Q4_K_M` version of the model.
26+
If `Q4_K_M` doesn't exist, the command pulls the first GGUF found in the **Files** view of the model on HuggingFace.
27+
To specify the quantization, provide it as a tag, for example:
28+
`docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q4_K_S`
29+
2530
```console
2631
docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
2732
```

0 commit comments

Comments
 (0)