File tree Expand file tree Collapse file tree 1 file changed +3
-12
lines changed Expand file tree Collapse file tree 1 file changed +3
-12
lines changed Original file line number Diff line number Diff line change @@ -54,28 +54,19 @@ Here are recommended settings, depending on the amount of VRAM that you have:
5454- More than 16GB VRAM:
5555
5656 ``` bash
57- llama-server \
58- -hf ggml-org/Qwen2.5-Coder-7B-Q8_0-GGUF \
59- --port 8012 -ngl 99 -fa -ub 1024 -b 1024 \
60- --ctx-size 0 --cache-reuse 256
57+ llama-server --fim-qwen-7b-default
6158 ```
6259
6360- Less than 16GB VRAM:
6461
6562 ``` bash
66- llama-server \
67- -hf ggml-org/Qwen2.5-Coder-3B-Q8_0-GGUF \
68- --port 8012 -ngl 99 -fa -ub 1024 -b 1024 \
69- --ctx-size 0 --cache-reuse 256
63+ llama-server --fim-qwen-3b-default
7064 ```
7165
7266- Less than 8GB VRAM:
7367
7468 ``` bash
75- llama-server \
76- -hf ggml-org/Qwen2.5-Coder-1.5B-Q8_0-GGUF \
77- --port 8012 -ngl 99 -fa -ub 1024 -b 1024 \
78- --ctx-size 0 --cache-reuse 256
69+ llama-server --fim-qwen-1.5b-default
7970 ```
8071
8172<details >
You can’t perform that action at this time.
0 commit comments