Skip to content

Commit 630ddcc

Browse files
committed
update docs
1 parent 262950d commit 630ddcc

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

examples/server/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,7 @@ The project is under active development, and we are [looking for feedback and co
140140
| `-sp, --special` | special tokens output enabled (default: false) |
141141
| `--no-warmup` | skip warming up the model with an empty run |
142142
| `--spm-infill` | use Suffix/Prefix/Middle pattern for infill (instead of Prefix/Suffix/Middle) as some models prefer this. (default: disabled) |
143+
| `-mtp, --multi-token-probs` | allow getting probabilities for multiple tokens. note: this will slow down the generation speed (default: disabled)<br/>(env: LLAMA_ARG_MULTI_TOKEN_PROBS) |
143144
| `--pooling {none,mean,cls,last,rank}` | pooling type for embeddings, use model default if unspecified<br/>(env: LLAMA_ARG_POOLING) |
144145
| `-cb, --cont-batching` | enable continuous batching (a.k.a dynamic batching) (default: enabled)<br/>(env: LLAMA_ARG_CONT_BATCHING) |
145146
| `-nocb, --no-cont-batching` | disable continuous batching<br/>(env: LLAMA_ARG_NO_CONT_BATCHING) |

0 commit comments

Comments
 (0)