Skip to content

Commit 3126abf

Browse files
authored
readme : update llama-server commands with fim presets (#34)
1 parent 30acfc0 commit 3126abf

File tree

1 file changed

+3
-12
lines changed

1 file changed

+3
-12
lines changed

README.md

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -54,28 +54,19 @@ Here are recommended settings, depending on the amount of VRAM that you have:
5454
- More than 16GB VRAM:
5555

5656
```bash
57-
llama-server \
58-
-hf ggml-org/Qwen2.5-Coder-7B-Q8_0-GGUF \
59-
--port 8012 -ngl 99 -fa -ub 1024 -b 1024 \
60-
--ctx-size 0 --cache-reuse 256
57+
llama-server --fim-qwen-7b-default
6158
```
6259

6360
- Less than 16GB VRAM:
6461

6562
```bash
66-
llama-server \
67-
-hf ggml-org/Qwen2.5-Coder-3B-Q8_0-GGUF \
68-
--port 8012 -ngl 99 -fa -ub 1024 -b 1024 \
69-
--ctx-size 0 --cache-reuse 256
63+
llama-server --fim-qwen-3b-default
7064
```
7165

7266
- Less than 8GB VRAM:
7367

7468
```bash
75-
llama-server \
76-
-hf ggml-org/Qwen2.5-Coder-1.5B-Q8_0-GGUF \
77-
--port 8012 -ngl 99 -fa -ub 1024 -b 1024 \
78-
--ctx-size 0 --cache-reuse 256
69+
llama-server --fim-qwen-1.5b-default
7970
```
8071

8172
<details>

0 commit comments

Comments
 (0)