You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When working with software, it's more challenging to parse free-form text outputs coming from language models. Structured outputs, like JSON, provide a clear format that software routines can read and process. This article explains how to use structured outputs to generate specific JSON schemas with the chat completions API with models deployed to Azure AI model inference in Azure AI services.
13
+
14
+
Typical scenarios of structured outputs include:
15
+
16
+
> [!div class="checklist"]
17
+
> * You need to extract specific information from a prompt and such information can be described as an schema with specific keys and types.
18
+
> * You need to parse information contained in the prompts.
19
+
> * You are using the model to control a workflow in your application where you can benefit from more rigid structures.
20
+
> * You are using the model as a zero-shot or few-shot learner.
21
+
22
+
## Prerequisites
23
+
24
+
To use structured outputs with chat completions models in your application, you need:
* A chat completions model deployment with JSON and structured outputs support. If you don't have one read [Add and configure models to Azure AI services](../../how-to/create-model-deployments.md).
29
+
30
+
* You can check which models support structured outputs by checking the column **Response format** in the [Models](../../concepts/models.md) article.
31
+
32
+
* This article uses `Cohere-command-r-plus-08-2024`.
When working with software, it's more challenging to parse free-form text outputs coming from language models. Structured outputs, like JSON, provide a clear format that software routines can read and process. This article explains how to use structured outputs to generate specific JSON schemas with the chat completions API with models deployed to Azure AI model inference in Azure AI services.
20
-
21
-
## Prerequisites
22
-
23
-
To use structured outputs with chat completions models in your application, you need:
* A chat completions model deployment with JSON and structured outputs support. If you don't have one read [Add and configure models to Azure AI services](../../how-to/create-model-deployments.md).
30
-
31
-
* You can check which models support structured outputs by checking the column **Response format** in the [Models](../../concepts/models.md) article.
32
-
33
-
* This article uses `Cohere-command-r-plus-08-2024`.
34
-
35
20
* Initialize a client to consume the model:
36
21
37
22
```python
@@ -104,20 +89,21 @@ __github_issue_schema.json__
104
89
}
105
90
```
106
91
107
-
Let's load this schema:
108
-
109
-
```python
110
-
withopen("github_issue_schema.json", "r") as f:
111
-
github_issue_schema = json.load(f)
112
-
```
113
-
114
92
When defining schemas, follow these recommendations:
115
93
116
94
> [!div class="checklist"]
117
95
> * Use clear and expressive keys.
118
96
> * Use `_` if you need to separate words to convey meaning.
119
97
> * Create clear titles and descriptions for important keys in your structure.
120
98
> * Evaluate multiple structures until finding the one that works best for your use case.
99
+
> * Take into account [limitations when indicating schemas](#supported-schemas). Limitations may vary per model.
Maintaining JSON schemas by hand is difficult and prone to errors. AI developers usually use [Pydantic](https://docs.pydantic.dev/) objects to describe the shapes of a given object. Pydantic is an open-source data validation library where you can flexibly define data structures for your applications.
@@ -175,7 +160,6 @@ The following example shows how you can use Pydantic to define an schema for a G
Structured Outputs can still contain mistakes. If you see mistakes, try adjusting your instructions, providing examples in the system instructions, or splitting tasks into simpler subtasks.
246
230
247
-
The following example uses validators in Pydantic to verify the schema:
231
+
It's a best practice to uses validators to ensure you get valid structures. In Pydantic, you can verify the schema of a given object as follows:
There are some limitations that models may place in how schemas are defined. When using structure outputs, consider the following limitations. Notice that limitations may vary per model.
245
+
246
+
#### Optional fields
247
+
248
+
Some models may require all the fields to be in the `required` section of the schema. If you need to use optional fields, use unions with null types to express that a given field can be optional.
When working with software, it's more challenging to parse free-form text outputs coming from language models. Structured outputs, like JSON, provide a clear format that software routines can read and process. This article explains how to use structured outputs to generate specific JSON schemas with the chat completions API with models deployed to Azure AI model inference in Azure AI services.
20
-
21
-
To use structured outputs with chat completions models in your application, you need:
* A chat completions model deployment with JSON and structured outputs support. If you don't have one read [Add and configure models to Azure AI services](../../how-to/create-model-deployments.md).
26
-
27
-
* You can check which models support structured outputs by checking the column **Response format** in the [Models](../../concepts/models.md) article.
28
-
29
-
* This article uses `Cohere-command-r-plus-08-2024`.
16
+
[!INCLUDE [intro](intro.md)]
30
17
31
18
## How to use structured outputs
32
19
@@ -78,12 +65,16 @@ __github_issue_schema.json__
78
65
79
66
We can use structure outputs with the defined schema as follows:
80
67
68
+
__Request__
69
+
81
70
```http
82
71
POST https://<resource>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview
83
72
Content-Type: application/json
84
73
api-key: <key>
85
74
```
86
75
76
+
__Body__
77
+
87
78
```json
88
79
{
89
80
"messages": [
@@ -92,13 +83,11 @@ api-key: <key>
92
83
"content": "Extract structured information from GitHub issues opened in our project.
93
84
94
85
Provide
95
-
- The title of the issue
96
-
- A 1-2 sentence description of the project
97
-
- The type of issue (Bug, Feature, Documentation, Regression)
98
-
- The operating system the issue was reported on
99
-
- Whether the issue is a duplicate of another issue
100
-
101
-
Do not guess."
86
+
- The title of the issue.
87
+
- A 1-2 sentence description of the project.
88
+
- The type of issue (Bug, Feature, Documentation, Regression).
89
+
- The operating system the issue was reported on.
90
+
- Whether the issue is a duplicate of another issue."
102
91
},
103
92
{
104
93
"role": "user",
@@ -142,11 +131,37 @@ api-key: <key>
142
131
143
132
Let's see how this works:
144
133
145
-
```output
134
+
__Response__
135
+
136
+
```json
146
137
{
147
-
"title": "Metadata tags issue on access control lists - ADLSgen2 setup",
148
-
"description": "Our project seems to have an issue with the metadata tag for groups when deploying the application with access control lists and necessary settings.",
149
-
"type": "Bug",
150
-
"operating_system": "Windows 10"
138
+
"id": "0a1234b5de6789f01gh2i345j6789klm",
139
+
"object": "chat.completion",
140
+
"created": 1718726686,
141
+
"model": "mistral-large-2407",
142
+
"choices": [
143
+
{
144
+
"index": 0,
145
+
"message": {
146
+
"role": "assistant",
147
+
"content": "{
148
+
\"title\": "Metadata tags issue on access control lists - ADLSgen2 setup\",
149
+
\"description\": "Our project seems to have an issue with the metadata tag for groups when deploying the application with access control lists and necessary settings.\",
0 commit comments