File tree Expand file tree Collapse file tree 2 files changed +10
-0
lines changed
Expand file tree Collapse file tree 2 files changed +10
-0
lines changed Original file line number Diff line number Diff line change @@ -16,6 +16,11 @@ examples: |-
1616
1717 You can pull GGUF models directly from [Hugging Face](https://huggingface.co/models?library=gguf).
1818
19+ **Note about quantization:** If no tag is specified, the command tries to pull the `Q4_K_M` version of the model.
20+ If `Q4_K_M` doesn't exist, the command pulls the first GGUF found in the **Files** view of the model on HuggingFace.
21+ To specify the quantization, provide it as a tag, for example:
22+ `docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q4_K_S`
23+
1924 ```console
2025 docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
2126 ```
Original file line number Diff line number Diff line change @@ -22,6 +22,11 @@ docker model pull ai/smollm2
2222
2323You can pull GGUF models directly from [ Hugging Face] ( https://huggingface.co/models?library=gguf ) .
2424
25+ ** Note about quantization:** If no tag is specified, the command tries to pull the ` Q4_K_M ` version of the model.
26+ If ` Q4_K_M ` doesn't exist, the command pulls the first GGUF found in the ** Files** view of the model on HuggingFace.
27+ To specify the quantization, provide it as a tag, for example:
28+ ` docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q4_K_S `
29+
2530``` console
2631docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF
2732```
You can’t perform that action at this time.
0 commit comments