Skip to content

Commit a7f304e

Browse files
committed
ollama tweaks
1 parent 481c63a commit a7f304e

File tree

1 file changed

+5
-3
lines changed

1 file changed

+5
-3
lines changed

docs/hub/ollama.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,9 @@ ollama run hf.co/bartowski/Humanish-LLama3-8B-Instruct-GGUF
2323

2424
## Custom Quantization
2525

26-
By default, the `Q4_K_M` quantization scheme is used. To select a different scheme, simply add a tag:
26+
By default, the `Q4_K_M` quantization scheme is used, when it's present inside the model repo. If not, we default to picking one reasonable quant type present inside the repo.
27+
28+
To select a different scheme, simply add a tag:
2729

2830
```sh
2931
ollama run hf.co/{username}/{repository}:{quantization}
@@ -40,15 +42,15 @@ ollama run hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0
4042
# the quantization name is case-insensitive, this will also work
4143
ollama run hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:iq3_m
4244

43-
# you can also select a specific file
45+
# you can also directly use the full filename as a tag
4446
ollama run hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Llama-3.2-3B-Instruct-IQ3_M.gguf
4547
```
4648

4749
## Custom Chat Template and Parameters
4850

4951
By default, a template will be selected automatically from a list of commonly used templates. It will be selected based on the built-in `tokenizer.chat_template` metadata stored inside the GGUF file.
5052

51-
If your GGUF file doesn't have a built-in template or uses a custom chat template, you can create a new file called `template` in the repository. The template must be a Go template, not a Jinja template. Here's an example:
53+
If your GGUF file doesn't have a built-in template or if you want to customize your chat template, you can create a new file called `template` in the repository. The template must be a Go template, not a Jinja template. Here's an example:
5254

5355
```
5456
{{ if .System }}<|system|>

0 commit comments

Comments
 (0)