@@ -46,8 +46,6 @@ for more details.
4646To enable, set ` return_finish_reason ` input tensor to ` True ` . The reason will be
4747sent as a string on the ` finish_reason ` output tensor.
4848
49- Supported since r24.12.
50-
5149### Cumulative Log Probabilities
5250
5351The cumulative log probability of the generated output text. See
@@ -57,7 +55,26 @@ for more details.
5755To enable, set ` return_cumulative_logprob ` input tensor to ` True ` . The floating
5856point value will be sent on the ` cumulative_logprob ` output tensor.
5957
60- Supported since r24.12.
58+ ### Log Probabilities
59+
60+ The log probabilities of the top probability words at each position if the
61+ [ logprobs] ( https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/sampling_params.py#L146-L152 )
62+ are requested. Only the log probabilities of the new tokens generated since the
63+ last response are returned on each new response. See
64+ [ here] ( https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/outputs.py#L24-L25 )
65+ for more details on the log probabilities.
66+
67+ To enable, set ` return_logprobs ` input tensor to ` True ` . The log probabilities
68+ will be sent on the ` logprobs ` output tensor as JSON.
69+
70+ ### Number of Input Tokens
71+
72+ The number of token IDs of the prompt. See
73+ [ here] ( https://github.com/vllm-project/vllm/blob/v0.6.3.post1/vllm/outputs.py#L79-L81 )
74+ for more details.
75+
76+ To enable, set ` return_num_input_tokens ` input tensor to ` True ` . The unsigned
77+ integer value will be sent on the ` num_input_tokens ` output tensor.
6178
6279### Number of Output Tokens
6380
@@ -71,8 +88,6 @@ for more details on the token IDs of the generated output text.
7188To enable, set ` return_num_output_tokens ` input tensor to ` True ` . The unsigned
7289integer value will be sent on the ` num_output_tokens ` output tensor.
7390
74- Supported since r24.12.
75-
7691## Examples
7792
7893### Add Finish Reason to Outputs
0 commit comments