Skip to content

Commit e64719a

Browse files
committed
Update Blog “using-structured-outputs-in-vllm”
1 parent 33355de commit e64719a

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

content/blog/using-structured-outputs-in-vllm.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,9 @@ LLMs are incredibly powerful, but their outputs can be inconsistent when a speci
2626
Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
2727

2828
How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
29-
![Structured outputs using vLLM](/img/structured_outputs_thumbnail.png "Structured outputs using vLLM")
29+
30+
31+
![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
3032

3133
### What is vLLM?
3234

0 commit comments

Comments
 (0)