Skip to content

Commit 6246b17

Browse files
committed
Update Blog “using-structured-outputs-in-vllm”
1 parent 57a9985 commit 6246b17

File tree

1 file changed

+8
-8
lines changed

1 file changed

+8
-8
lines changed

content/blog/using-structured-outputs-in-vllm.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ tags:
1313
---
1414
Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
1515

16-
### Why Structured Outputs?
16+
## Why Structured Outputs?
1717

1818
LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
1919

@@ -28,7 +28,7 @@ How these tools work? The idea is that we´ll be able to filter the list of poss
2828

2929
![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
3030

31-
### What is vLLM?
31+
## What is vLLM?
3232

3333
vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
3434

@@ -38,7 +38,7 @@ vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. I
3838

3939
These optimizations make vLLM one of the fastest and most versatile engines for production environments.
4040

41-
### Structured outputs on vLLM
41+
## Structured outputs on vLLM
4242

4343
vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
4444

@@ -49,7 +49,7 @@ vLLM extends the OpenAI API with additional parameters to enable structured outp
4949

5050
Here’s how each works, along with example outputs:
5151

52-
#### **1. Guided Choice**
52+
### **1. Guided Choice**
5353

5454
Simplest form of structured output, ensuring the response is one of a set of predefined options.
5555

@@ -74,7 +74,7 @@ print(completion.choices[0].message.content)
7474
positive
7575
```
7676

77-
#### **2. Guided Regex**
77+
### **2. Guided Regex**
7878

7979
Constrains output to match a regex pattern, useful for formats like email addresses.
8080

@@ -98,7 +98,7 @@ print(completion.choices[0].message.content)
9898
9999
```
100100

101-
#### **3. Guided JSON**
101+
### **3. Guided JSON**
102102

103103
Enforces a valid JSON format based on a schema, simplifying integration with other systems.
104104

@@ -139,7 +139,7 @@ print(completion.choices[0].message.content)
139139
}
140140
```
141141

142-
#### **4. Guided Grammar**
142+
### **4. Guided Grammar**
143143

144144
Uses an EBNF grammar to define complex output structures, such as SQL queries.
145145

@@ -167,7 +167,7 @@ print(completion.choices[0].message.content)
167167
SELECT * FROM users WHERE age > 30;
168168
```
169169

170-
### **Next Steps**
170+
## **Next Steps**
171171

172172
To start integrating structured outputs into your projects:
173173

0 commit comments

Comments
 (0)