File tree Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Original file line number Diff line number Diff line change @@ -87,7 +87,7 @@ The project is under active development, and we are [looking for feedback and co
8787| ` -ctk, --cache-type-k TYPE ` | KV cache data type for K (default: f16) |
8888| ` -ctv, --cache-type-v TYPE ` | KV cache data type for V (default: f16) |
8989| ` -dt, --defrag-thold N ` | KV cache defragmentation threshold (default: -1.0, < 0 - disabled)<br />(env: LLAMA_ARG_DEFRAG_THOLD) |
90- | ` -np, --parallel N ` | number of parallel sequences to decode (default: 1) |
90+ | ` -np, --parallel N ` | number of parallel sequences to decode (default: 1)< br />(env: LLAMA_ARG_N_PARALLEL) |
9191| ` -cb, --cont-batching ` | enable continuous batching (a.k.a dynamic batching) (default: enabled)<br />(env: LLAMA_ARG_CONT_BATCHING) |
9292| ` -nocb, --no-cont-batching ` | disable continuous batching<br />(env: LLAMA_ARG_NO_CONT_BATCHING) |
9393| ` --mlock ` | force system to keep model in RAM rather than swapping or compressing |
You can’t perform that action at this time.
0 commit comments