Skip to content

Commit 2565dfa

Browse files
committed
docs: update llama-bench
- added the recently added n-cpu-moe option to the docs while in there
1 parent 667663a commit 2565dfa

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

tools/llama-bench/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ options:
3030
--delay <0...N> (seconds) delay between each test (default: 0)
3131
-o, --output <csv|json|jsonl|md|sql> output format printed to stdout (default: md)
3232
-oe, --output-err <csv|json|jsonl|md|sql> output format printed to stderr (default: none)
33+
--list-devices list available devices and exit
3334
-v, --verbose verbose output
3435
--progress print test progress indicators
3536
@@ -48,11 +49,13 @@ test parameters:
4849
--cpu-strict <0|1> (default: 0)
4950
--poll <0...100> (default: 50)
5051
-ngl, --n-gpu-layers <n> (default: 99)
52+
-ncmoe, --n-cpu-moe <n> (default: 0)
5153
-rpc, --rpc <rpc_servers> (default: none)
5254
-sm, --split-mode <none|layer|row> (default: layer)
5355
-mg, --main-gpu <i> (default: 0)
5456
-nkvo, --no-kv-offload <0|1> (default: 0)
5557
-fa, --flash-attn <0|1> (default: 0)
58+
-dev, --device <dev0/dev1/...> (default: auto)
5659
-mmp, --mmap <0|1> (default: 1)
5760
-embd, --embeddings <0|1> (default: 0)
5861
-ts, --tensor-split <ts0/ts1/..> (default: 0)

0 commit comments

Comments
 (0)