Skip to content

Commit cbef812

Browse files
authored
Update README.md with env: LLAMA_ARG_N_PARALLEL
1 parent a25f838 commit cbef812

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/server/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ The project is under active development, and we are [looking for feedback and co
8787
| `-ctk, --cache-type-k TYPE` | KV cache data type for K (default: f16) |
8888
| `-ctv, --cache-type-v TYPE` | KV cache data type for V (default: f16) |
8989
| `-dt, --defrag-thold N` | KV cache defragmentation threshold (default: -1.0, < 0 - disabled)<br/>(env: LLAMA_ARG_DEFRAG_THOLD) |
90-
| `-np, --parallel N` | number of parallel sequences to decode (default: 1) |
90+
| `-np, --parallel N` | number of parallel sequences to decode (default: 1)<br/>(env: LLAMA_ARG_N_PARALLEL) |
9191
| `-cb, --cont-batching` | enable continuous batching (a.k.a dynamic batching) (default: enabled)<br/>(env: LLAMA_ARG_CONT_BATCHING) |
9292
| `-nocb, --no-cont-batching` | disable continuous batching<br/>(env: LLAMA_ARG_NO_CONT_BATCHING) |
9393
| `--mlock` | force system to keep model in RAM rather than swapping or compressing |

0 commit comments

Comments
 (0)