Skip to content

Commit 17ae25c

Browse files
committed
[docs] Add logprobs and num of input tokens to docs
1 parent 1e2675e commit 17ae25c

File tree

1 file changed

+20
-5
lines changed

1 file changed

+20
-5
lines changed

docs/additional_outputs.md

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,6 @@ for more details.
4646
To enable, set `return_finish_reason` input tensor to `True`. The reason will be
4747
sent as a string on the `finish_reason` output tensor.
4848

49-
Supported since r24.12.
50-
5149
### Cumulative Log Probabilities
5250

5351
The cumulative log probability of the generated output text. See
@@ -57,7 +55,26 @@ for more details.
5755
To enable, set `return_cumulative_logprob` input tensor to `True`. The floating
5856
point value will be sent on the `cumulative_logprob` output tensor.
5957

60-
Supported since r24.12.
58+
### Log Probabilities
59+
60+
The log probabilities of the top probability words at each position if the
61+
[logprobs](https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/sampling_params.py#L146-L152)
62+
are requested. Only the log probabilities of the new tokens generated since the
63+
last response are returned on each new response. See
64+
[here](https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/outputs.py#L24-L25)
65+
for more details on the log probabilities.
66+
67+
To enable, set `return_logprobs` input tensor to `True`. The log probabilities
68+
will be sent on the `logprobs` output tensor as JSON.
69+
70+
### Number of Input Tokens
71+
72+
The number of token IDs of the prompt. See
73+
[here](https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/outputs.py#L79-L81)
74+
for more details.
75+
76+
To enable, set `return_num_input_tokens` input tensor to `True`. The unsigned
77+
integer value will be sent on the `num_input_tokens` output tensor.
6178

6279
### Number of Output Tokens
6380

@@ -71,8 +88,6 @@ for more details on the token IDs of the generated output text.
7188
To enable, set `return_num_output_tokens` input tensor to `True`. The unsigned
7289
integer value will be sent on the `num_output_tokens` output tensor.
7390

74-
Supported since r24.12.
75-
7691
## Examples
7792

7893
### Add Finish Reason to Outputs

0 commit comments

Comments
 (0)