Skip to content

Commit bb2eece

Browse files
committed
fix: nit
1 parent af3f19d commit bb2eece

File tree

2 files changed

+73
-78
lines changed

2 files changed

+73
-78
lines changed

articles/ai-foundry/model-inference/includes/use-structured-outputs/python.md

Lines changed: 73 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -239,24 +239,72 @@ except ValidationError as e:
239239
print(f"Validation error: {e}")
240240
```
241241

242-
## Structured outputs in images
242+
### Specifying an schema
243243

244-
You can use structured outputs with multi-modal models to extract information from data like images. Let's consider the following chart:
244+
There are some limitations that models may place in schemas definitions. Such limitations may vary per-model. **We recommend reviewing the documentation from the model provider** to verify that your schemas are valid.
245245

246-
:::image type="content" source="../../media/use-structured-outputs/example_graph_treecover.png" alt-text="An example image showing a chart with the annual loss in thousand square kilometers of global tree cover across different climate zones." lightbox="../../media/use-structured-outputs/example_graph_treecover.png":::
246+
The following guidelines applies to the majority of models:
247+
248+
#### Optional fields
247249

248-
Let's load this image:
250+
Some models may require all the fields to be in the `required` section of the schema. If you need to use optional fields, use unions with null types to express that a given field can be optional.
249251

250252
```python
251-
from azure.ai.inference.models import ImageContentItem, ImageUrl
253+
from pydantic import BaseModel
254+
from typing import Literal, Union
252255

253-
image_graph = ImageUrl.load(
254-
image_file="example_graph_treecover.png",
255-
image_format="png"
256-
)
256+
class Issue(BaseModel, extra="forbid"):
257+
title: str
258+
description: str
259+
type: Literal["Bug", "Feature", "Documentation", "Regression"]
260+
operating_system: Union[str, None]
257261
```
258262

259-
Let's define a generic schema that can be use to encode the information contained in a chart. We use [Pyndatic objects](#use-pydantic-objects) as described before.
263+
#### Nested types
264+
265+
Models may support indicating nesting types. You can compose complex structures as needed:
266+
267+
```python
268+
from pydantic import BaseModel
269+
from typing import Literal
270+
271+
class Project(BaseModel, extra="forbid"):
272+
name: str
273+
owner: str
274+
275+
class Issue(BaseModel, extra="forbid"):
276+
title: str
277+
description: str
278+
type: Literal["Bug", "Feature", "Documentation", "Regression"]
279+
operating_system: str
280+
project: Project
281+
```
282+
283+
Nested types also include recursive definition of types:
284+
285+
```python
286+
from pydantic import BaseModel
287+
from typing import Literal, List
288+
289+
class Issue(BaseModel, extra="forbid"):
290+
title: str
291+
description: str
292+
type: Literal["Bug", "Feature", "Documentation", "Regression"]
293+
operating_system: str
294+
related_issues: List[Issue]
295+
```
296+
297+
Verify the level of nesting supported by the model you are working with.
298+
299+
## Structured outputs in images
300+
301+
You can use structured outputs with multi-modal models to extract information from data like images.
302+
303+
Let's consider the following chart:
304+
305+
:::image type="content" source="../../media/use-structured-outputs/example_graph_treecover.png" alt-text="An example image showing a chart with the annual loss in thousand square kilometers of global tree cover across different climate zones." lightbox="../../media/use-structured-outputs/example_graph_treecover.png":::
306+
307+
We can define a generic schema that can be use to encode the information contained in the chart and then use it for further analysis. We use [Pyndatic objects](#use-pydantic-objects) as described before.
260308

261309
```python
262310
from pydantic import BaseModel
@@ -275,7 +323,18 @@ class Graph(BaseModel):
275323
data: list[DataPoint]
276324
```
277325

278-
Let's use structured outputs to extract the information:
326+
We can load the image as follows to pass it to the model:
327+
328+
```python
329+
from azure.ai.inference.models import ImageContentItem, ImageUrl
330+
331+
image_graph = ImageUrl.load(
332+
image_file="example_graph_treecover.png",
333+
image_format="png"
334+
)
335+
```
336+
337+
Use structured outputs to extract the information:
279338

280339
```python
281340
response = client.complete(
@@ -297,7 +356,7 @@ response = client.complete(
297356
)
298357
```
299358

300-
Let's check the outputs:
359+
It's always a good practice to validate the outputs and schemas:
301360

302361
```python
303362
import json
@@ -340,7 +399,7 @@ print(json.dumps(json_response_message, indent=4))
340399
}
341400
```
342401

343-
Let's use `matplotlib` and try to plot the data:
402+
We can see how much information the model was able to capture by plotting the data using `matplotlib`:
344403

345404
```python
346405
import matplotlib.pyplot as plt
@@ -372,59 +431,4 @@ plt.show()
372431

373432
:::image type="content" source="../../media/use-structured-outputs/graph_treecover_plot.png" alt-text="The resulting plot of the data contained in the structured output generated by the model." lightbox="../../media/use-structured-outputs/graph_treecover_plot.png":::
374433

375-
376-
### Supported schemas
377-
378-
There are some limitations that models may place in how schemas are defined. When using structure outputs, consider the following limitations. Notice that limitations may vary per model.
379-
380-
#### Optional fields
381-
382-
Some models may require all the fields to be in the `required` section of the schema. If you need to use optional fields, use unions with null types to express that a given field can be optional.
383-
384-
```python
385-
from pydantic import BaseModel
386-
from typing import Literal, Union
387-
388-
class Issue(BaseModel, extra="forbid"):
389-
title: str
390-
description: str
391-
type: Literal["Bug", "Feature", "Documentation", "Regression"]
392-
operating_system: Union[str, None]
393-
```
394-
395-
#### Nested types
396-
397-
Models may support indicating nesting types. You can compose complex structures as needed:
398-
399-
```python
400-
from pydantic import BaseModel
401-
from typing import Literal
402-
403-
class Project(BaseModel, extra="forbid"):
404-
name: str
405-
owner: str
406-
407-
class Issue(BaseModel, extra="forbid"):
408-
title: str
409-
description: str
410-
type: Literal["Bug", "Feature", "Documentation", "Regression"]
411-
operating_system: str
412-
project: Project
413-
```
414-
415-
Nested types also include recursive definition of types:
416-
417-
```python
418-
from pydantic import BaseModel
419-
from typing import Literal, List
420-
421-
class Issue(BaseModel, extra="forbid"):
422-
title: str
423-
description: str
424-
type: Literal["Bug", "Feature", "Documentation", "Regression"]
425-
operating_system: str
426-
related_issues: List[Issue]
427-
```
428-
429-
430-
[!INCLUDE [supported-schemas](supported-schemas.md)]
434+
While the information is not perfect, we can see the model was able to capture a good amount of information from the original chart.

articles/ai-foundry/model-inference/includes/use-structured-outputs/supported-schemas.md

Lines changed: 0 additions & 9 deletions
This file was deleted.

0 commit comments

Comments
 (0)