|
| 1 | +--- |
| 2 | +title: How to use structured outputs for chat models with Azure AI model inference |
| 3 | +titleSuffix: Azure AI Foundry |
| 4 | +description: Learn how to use structured outputs with chat completions with Azure AI model inference |
| 5 | +manager: scottpolly |
| 6 | +author: mopeakande |
| 7 | +reviewer: santiagxf |
| 8 | +ms.service: azure-ai-model-inference |
| 9 | +ms.topic: how-to |
| 10 | +ms.date: 1/21/2025 |
| 11 | +ms.author: mopeakande |
| 12 | +ms.reviewer: fasantia |
| 13 | +ms.custom: references_regions |
| 14 | +zone_pivot_groups: azure-ai-inference-samples |
| 15 | +--- |
| 16 | + |
| 17 | +[!INCLUDE [Feature preview](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/feature-preview.md)] |
| 18 | + |
| 19 | +When working with software, it's more challenging to parse free-form text outputs coming from language models. Structured outputs, like JSON, provide a clear format that software routines can read and process. This article explains how to use structured outputs to generate specific JSON schemas with the chat completions API with models deployed to Azure AI model inference in Azure AI services. |
| 20 | + |
| 21 | +## Prerequisites |
| 22 | + |
| 23 | +To use chat completion models in your application, you need: |
| 24 | + |
| 25 | +[!INCLUDE [how-to-prerequisites](../how-to-prerequisites.md)] |
| 26 | + |
| 27 | +[!INCLUDE [how-to-prerequisites-python](../how-to-prerequisites-python.md)] |
| 28 | + |
| 29 | +* A chat completions model deployment with JSON and structured outputs support. If you don't have one read [Add and configure models to Azure AI services](../../how-to/create-model-deployments.md). |
| 30 | + |
| 31 | + * This article uses `Cohere-command-r-plus-08-2024`. |
| 32 | + |
| 33 | +* Initialize a client to consume the model: |
| 34 | + |
| 35 | + ```python |
| 36 | + import os |
| 37 | + import json |
| 38 | + from azure.ai.inference import ChatCompletionsClient |
| 39 | + from azure.ai.inference.models import SystemMessage, UserMessage, JsonSchemaFormat |
| 40 | + from azure.core.credentials import AzureKeyCredential |
| 41 | + |
| 42 | + client = ChatCompletionsClient( |
| 43 | + endpoint="https://aiservices-demo-wus2.services.ai.azure.com/models", |
| 44 | + credential=AzureKeyCredential(os.environ["AZURE_INFERENCE_CREDENTIAL"]), |
| 45 | + model="Cohere-command-r-plus-08-2024", |
| 46 | + ) |
| 47 | + ``` |
| 48 | + |
| 49 | +## How to use structured outputs |
| 50 | + |
| 51 | +Structured outputs uses JSON schemas to enforce output structure. JSON schemas describe the shape of the JSON object including expected values, types, and which ones are required. Those JSON objects are encoded as an string within the response of the model. |
| 52 | + |
| 53 | +### Example |
| 54 | + |
| 55 | +To exemplify the scenario, let's try to parse the attributes of a GitHub Issue from it's description. |
| 56 | + |
| 57 | +```python |
| 58 | +import requests |
| 59 | + |
| 60 | +url = "https://api.github.com/repos/Azure-Samples/azure-search-openai-demo/issues/2231" |
| 61 | +response = requests.get(url) |
| 62 | +issue_body = response.json()["body"] |
| 63 | +``` |
| 64 | + |
| 65 | +The output of `issue_body` looks as follows: |
| 66 | + |
| 67 | +```output |
| 68 | +'<!--\r\nIF SUFFICIENT INFORMATION IS NOT PROVIDED VIA THE FOLLOWING TEMPLATE THE ISSUE MIGHT BE CLOSED WITHOUT FURTHER CONSIDERATION OR INVESTIGATION\r\n-->\r\n> Please provide us with the following information:\r\n> ---------------------------------------------------------------\r\n\r\n### This issue is for a: (mark with an `x`)\r\n```\r\n- [x] bug report -> please search issues before submitting\r\n- [ ] feature request\r\n- [ ] documentation issue or request\r\n- [ ] regression (a behavior that used to work and stopped in a new release)\r\n```\r\n\r\n### Minimal steps to reproduce\r\n> Deploy the app with auth and acl´s turned on, configure the acls file, run all the scripts needed.\r\n\r\n### Any log messages given by the failure\r\n> None\r\n\r\n### Expected/desired behavior\r\n> groups field to be filled the the groups id\'s that have permissions to "view the file"\r\n\r\n### OS and Version?\r\n> win 10\r\n### azd version?\r\n> azd version 1.11.0\r\n\r\n### Versions\r\n>\r\n\r\n### Mention any other details that might be useful\r\n\r\nAfter configuring the json with the perms all the scripts (`adlsgen2setup.py` and `prepdocs.ps1`) everything goes well but the groups metadata tag never gets to have any groups.\r\n\r\n\r\n\r\n\r\n> ---------------------------------------------------------------\r\n> Thanks! We\'ll be in touch soon.\r\n' |
| 69 | +``` |
| 70 | + |
| 71 | +### Define the schema |
| 72 | + |
| 73 | +The following JSON schema defines the schema of a GitHub issue: |
| 74 | + |
| 75 | +__github_issue_schema.json__ |
| 76 | + |
| 77 | +```json |
| 78 | +{ |
| 79 | + "title": "github_issue", |
| 80 | + "type": "object", |
| 81 | + "properties": { |
| 82 | + "title": { |
| 83 | + "title": "Title", |
| 84 | + "type": "string" |
| 85 | + }, |
| 86 | + "description": { |
| 87 | + "title": "Description", |
| 88 | + "type": "string" |
| 89 | + }, |
| 90 | + "type": { |
| 91 | + "enum": ["Bug", "Feature", "Documentation", "Regression"], |
| 92 | + "title": "Type", |
| 93 | + "type": "string" |
| 94 | + }, |
| 95 | + "operating_system": { |
| 96 | + "title": "Operating System", |
| 97 | + "type": "string" |
| 98 | + } |
| 99 | + }, |
| 100 | + "required": ["title", "description", "type", "operating_system"], |
| 101 | + "additionalProperties": false |
| 102 | +} |
| 103 | +``` |
| 104 | + |
| 105 | +Let's load this schema: |
| 106 | + |
| 107 | +```python |
| 108 | +with open("github_issue_schema.json", "r") as f: |
| 109 | + github_issue_schema = json.load(f) |
| 110 | +``` |
| 111 | + |
| 112 | +### Use structure outputs |
| 113 | + |
| 114 | +We can use structure outputs with the defined schema as follows: |
| 115 | + |
| 116 | +```python |
| 117 | +response = client.complete( |
| 118 | + response_format=JsonSchemaFormat( |
| 119 | + name="github_issue", |
| 120 | + schema=github_issue_schema, |
| 121 | + description="Describes a GitHub issue", |
| 122 | + strict=True, |
| 123 | + ), |
| 124 | + messages=[ |
| 125 | + SystemMessage(""" |
| 126 | + Extract structured information from GitHub issues opened in our project. |
| 127 | +
|
| 128 | + Provide |
| 129 | + - The title of the issue |
| 130 | + - A 1-2 sentence description of the project |
| 131 | + - The type of issue (Bug, Feature, Documentation, Regression) |
| 132 | + - The operating system the issue was reported on |
| 133 | + - Whether the issue is a duplicate of another issue |
| 134 | + """), |
| 135 | + UserMessage(issue_body), |
| 136 | + ], |
| 137 | +) |
| 138 | +``` |
| 139 | + |
| 140 | +Let's see how this works: |
| 141 | + |
| 142 | +```python |
| 143 | +json_response_message = json.loads(response.choices[0].message.content) |
| 144 | +print(json.dumps(json_response_message, indent=4)) |
| 145 | +``` |
| 146 | + |
| 147 | +```output |
| 148 | +{ |
| 149 | + "title": "Metadata tags issue on access control lists - ADLSgen2 setup", |
| 150 | + "description": "Our project seems to have an issue with the metadata tag for groups when deploying the application with access control lists and necessary settings.", |
| 151 | + "type": "Bug", |
| 152 | + "operating_system": "Windows 10" |
| 153 | +} |
| 154 | +``` |
| 155 | + |
| 156 | + |
| 157 | +## Use Pydantic objects |
| 158 | + |
| 159 | +Maintaining JSON schemas by hand is difficult and prone to errors. AI developers usually use [Pydantic](https://docs.pydantic.dev/) objects to describe the shapes of a given object. Pydantic is an open-source data validation library where you can flexibly define data structures for your applications. |
| 160 | + |
| 161 | +The following example shows how you can use Pydantic to define an schema for a GitHub issue. |
| 162 | + |
| 163 | +```python |
| 164 | +from pydantic import BaseModel |
| 165 | +from typing import Literal |
| 166 | +from enum import Enum |
| 167 | + |
| 168 | +class Issue(BaseModel, extra="forbid"): |
| 169 | + title: str |
| 170 | + description: str |
| 171 | + type: Literal["Bug", "Feature", "Documentation", "Regression"] |
| 172 | + operating_system: str |
| 173 | +``` |
| 174 | + |
| 175 | +Some things to notice: |
| 176 | + |
| 177 | +> [!div class="checklist"] |
| 178 | +> We represent schemas using a class that inherits from `BaseModel`. |
| 179 | +> We set `extra="forbid"` to instruct Pyndantic to do not accept additional properties from what we have specified. |
| 180 | +> We use type annotations to indicate the expected types. |
| 181 | +> `Literal` indicates we expect specific fixed values. |
| 182 | +
|
| 183 | +```python |
| 184 | +github_issue_schema = Issue.model_json_schema() |
| 185 | +``` |
| 186 | + |
| 187 | +Let's see how we can use the schema in the same way: |
| 188 | + |
| 189 | +```python |
| 190 | +response = client.complete( |
| 191 | + response_format=JsonSchemaFormat( |
| 192 | + name="github_issue", |
| 193 | + schema=github_issue_schema, |
| 194 | + description="Describes a GitHub issue", |
| 195 | + strict=True, |
| 196 | + ), |
| 197 | + messages=[ |
| 198 | + SystemMessage(""" |
| 199 | + Extract structured information from GitHub issues opened in our project. |
| 200 | +
|
| 201 | + Provide |
| 202 | + - The title of the issue |
| 203 | + - A 1-2 sentence description of the project |
| 204 | + - The type of issue (Bug, Feature, Documentation, Regression) |
| 205 | + - The operating system the issue was reported on |
| 206 | + - Whether the issue is a duplicate of another issue |
| 207 | + """), |
| 208 | + UserMessage(issue_body), |
| 209 | + ], |
| 210 | +) |
| 211 | +``` |
| 212 | + |
| 213 | +Let's see how this works: |
| 214 | + |
| 215 | +```python |
| 216 | +json_response_message = json.loads(response.choices[0].message.content) |
| 217 | +print(json.dumps(json_response_message, indent=4)) |
| 218 | +``` |
| 219 | + |
| 220 | +```output |
| 221 | +{ |
| 222 | + "title": "Metadata tags issue on access control lists - ADLSgen2 setup", |
| 223 | + "description": "Our project seems to have an issue with the metadata tag for groups when deploying the application with access control lists and necessary settings.", |
| 224 | + "type": "Bug", |
| 225 | + "operating_system": "Windows 10" |
| 226 | +} |
| 227 | +``` |
0 commit comments