Skip to content

Commit 453d204

Browse files
committed
add /apply-template documentation
1 parent 10448bf commit 453d204

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

examples/server/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -572,6 +572,14 @@ With input 'á' (utf8 hex: C3 A1) on tinyllama/stories260k
572572

573573
`tokens`: Set the tokens to detokenize.
574574

575+
### POST `/apply-template`: Apply chat template to a conversation
576+
577+
Uses the server's prompt template formatting functionality to convert chat messages to a single string expected by a chat model as input, but does not perform inference. Instead, the prompt string is returned in the `prompt` field of the JSON response. The prompt can then be modified as desired (for example, to insert "Sure!" at the beginning of the model's response) before sending to `/completion` to generate the chat response.
578+
579+
*Options:*
580+
581+
`messages`: (Required) Chat turns in the same format as `/v1/chat/completions`.
582+
575583
### POST `/embedding`: Generate embedding of a given text
576584

577585
> [!IMPORTANT]

0 commit comments

Comments
 (0)