We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 1583908 commit b94412bCopy full SHA for b94412b
README.md
@@ -111,13 +111,13 @@ cmake --build build -j"$(nproc)"
111
./build/bin/llama-bench \
112
--amx \
113
-m /path-to-your-model.gguf \
114
- -t 32 -ngl 10 -b 256 -ub 256 -pg 1024 --no-warmup
+ -t 32 -ngl 10 -nopo 1 -b 512 -ub 512 -pg 512,512 --repetitions 3
115
116
# CLI (hybrid) quick generation
117
./build/bin/llama-cli \
118
119
120
- -t 32 -ngl 10 -c 4096 -n 64 -p "10 facts about birds" --no-warmup
+ -ngl 10 -t 32 -b 4096 -c 4096 -n 512 -p "10 facts about birds" -no-cnv
121
122
# Server (hybrid) – default port 8080
123
./build/bin/llama-server --amx \
0 commit comments