From 006167dd6504b2fa107dfc6067c54d7160c553b0 Mon Sep 17 00:00:00 2001 From: Burhanuddin Mustafa Lakdawala Date: Fri, 25 Oct 2024 12:52:46 -0700 Subject: [PATCH] Update server README.md --- examples/server/README.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/examples/server/README.md b/examples/server/README.md index bc737237eb018..9e298b6280308 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -761,8 +761,16 @@ Example: ``` Possible values for `slot[i].state` are: -- `0`: SLOT_STATE_IDLE -- `1`: SLOT_STATE_PROCESSING +- `0`: SLOT_STATE_IDLE + The slot is idle and ready to use. +- `1`: SLOT_STATE_PROCESSING_PROMPT + The slot is processing the input prompt tokens. +- `2`: SLOT_STATE_DONE_PROMPT + The slot has finished processing the input prompt. For embedding and rerank tasks the slot will be released soon, otherwise the slot will be used for generation. +- `3`: SLOT_STATE_GENERATING + The slot is generating output tokens. + +[State diagram](https://github.com/ggerganov/llama.cpp/pull/9283) ### GET `/metrics`: Prometheus compatible metrics exporter