Skip to content

Commit c993266

Browse files
committed
Merge branch 'main' into release-public-preview-translator-v4
2 parents 3e3fb04 + 17dc68a commit c993266

File tree

17 files changed

+414
-89
lines changed

17 files changed

+414
-89
lines changed

.openpublishing.publish.config.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,12 @@
9292
"branch": "main",
9393
"branch_mapping": {}
9494
},
95+
{
96+
"path_to_root": "azure-functions-ai-services-agent-javascript",
97+
"url": "https://github.com/Azure-Samples/azure-functions-ai-services-agent-javascript",
98+
"branch": "main",
99+
"branch_mapping": {}
100+
},
95101
{
96102
"path_to_root": "samples-cognitive-services-speech-sdk",
97103
"url": "https://github.com/Azure-Samples/cognitive-services-speech-sdk",

articles/ai-services/agents/how-to/tools/azure-functions.md

Lines changed: 60 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ services: azure-ai-agent-service
66
manager: nitinme
77
ms.service: azure-ai-agent-service
88
ms.topic: how-to
9-
ms.date: 01/30/2025
9+
ms.date: 04/15/2025
1010
author: aahill
1111
ms.author: aahi
1212
ms.custom: azure-ai-agents
@@ -138,7 +138,9 @@ To use all features of function calling including parallel functions, you need t
138138

139139
## Define a function for your agent to call
140140

141-
Start by defining an Azure Function queue trigger function that will process function calls from the queue.
141+
Start by defining an Azure Function queue trigger function that will process AI function calls from the queue.
142+
143+
# [Python](#tab/python)
142144

143145
```python
144146
# Function to get the weather from an Azure Storage queue where the AI Agent will send function call information
@@ -172,12 +174,27 @@ def process_queue_message(msg: func.QueueMessage) -> None:
172174
logging.info(f"Sent message to output queue with message {result_message}")
173175
```
174176

177+
# [TypeScript](#tab/typescript)
178+
179+
:::code language="TypeScript" source="~/azure-functions-ai-services-agent-javascript/app/src/functions/queueGetWeather.ts" :::
180+
181+
# [REST API](#tab/rest)
182+
183+
No REST equivalent provided.
184+
185+
186+
---
187+
175188

176189
## Create an AI project client and agent
177190

178-
In the sample below we create a client and an agent that has the tools definition for the Azure Function
191+
In the sample below we create a client and an agent that has the AI tools definition for the Azure Function. The term `function` is used in two contexts within the AI tool definition:
192+
193+
* Azure Function: the type of tool. This is the Azure Functions app.
194+
* Function: the Http trigger function `GetWeather` within the Azure Function to call when the tool is invoked in the AI Project.
179195

180196
# [Python](#tab/python)
197+
181198
```python
182199
# Initialize the client and create agent for the tools Azure Functions that the agent can use
183200

@@ -231,7 +248,12 @@ agent = project_client.agents.create_agent(
231248
)
232249
```
233250

251+
# [TypeScript](#tab/typescript)
252+
253+
:::code language="TypeScript" source="~/azure-functions-ai-services-agent-javascript/app/src/azureProjectInit.ts" id="CreateAgent" :::
254+
234255
# [REST API](#tab/rest)
256+
235257
Follow the [REST API Quickstart](../../quickstart.md?pivots=rest-api) to set the right values for the environment variables `AZURE_AI_AGENTS_TOKEN` and `AZURE_AI_AGENTS_ENDPOINT`. Then create the agent using:
236258
```console
237259
curl $AZURE_AI_AGENTS_ENDPOINT/assistants?api-version=2024-12-01-preview \
@@ -276,6 +298,7 @@ curl $AZURE_AI_AGENTS_ENDPOINT/assistants?api-version=2024-12-01-preview \
276298
}'
277299
```
278300

301+
279302
---
280303

281304
## Create a thread for the agent
@@ -287,13 +310,18 @@ thread = project_client.agents.create_thread()
287310
print(f"Created thread, thread ID: {thread.id}")
288311
```
289312

313+
# [TypeScript](#tab/typescript)
314+
315+
:::code language="TypeScript" source="~/azure-functions-ai-services-agent-javascript/app/src/azureProjectInit.ts" id="CreateThread" :::
316+
290317
# [REST API](#tab/rest)
291318
```console
292319
curl $AZURE_AI_AGENTS_ENDPOINT/threads?api-version=2024-12-01-preview \
293320
-H "Authorization: Bearer $AZURE_AI_AGENTS_TOKEN" \
294321
-H "Content-Type: application/json" \
295322
-d ''
296323
```
324+
297325
---
298326

299327
## Create a run and check the output
@@ -320,6 +348,11 @@ while run.status in ["queued", "in_progress", "requires_action"]:
320348

321349
print(f"Run finished with status: {run.status}")
322350
```
351+
352+
# [TypeScript](#tab/typescript)
353+
354+
:::code language="TypeScript" source="~/azure-functions-ai-services-agent-javascript/app/src/functions/httpPrompt.ts" id="CreateMessage" :::
355+
323356
# [REST API](#tab/rest)
324357
```console
325358
curl $AZURE_AI_AGENTS_ENDPOINT/threads/thread_abc123/messages?api-version=2024-12-01-preview \
@@ -364,10 +397,34 @@ project_client.agents.delete_agent(agent.id)
364397
print("Deleted agent")
365398
```
366399

400+
# [TypeScript](#tab/typescript)
401+
402+
:::code language="TypeScript" source="~/azure-functions-ai-services-agent-javascript/app/src/functions/httpPrompt.ts" id="ListMessages" :::
403+
367404
# [REST API](#tab/rest)
368405
```console
369406
curl $AZURE_AI_AGENTS_ENDPOINT/threads/thread_abc123/messages?api-version=2024-12-01-preview \
370407
-H "Authorization: Bearer $AZURE_AI_AGENTS_TOKEN"
371408
```
372409

410+
---
411+
412+
## Troubleshooting
413+
414+
# [Python](#tab/python)
415+
416+
For any issues with the Python code, create an issue on the [sample code repository](https://github.com/Azure-Samples/azure-functions-ai-services-agent-python/issues)
417+
418+
# [TypeScript](#tab/typescript)
419+
420+
For any issues with the TypeScript code, create an issue on the [sample code repository](https://github.com/Azure-Samples/azure-functions-ai-services-agent-javascript/issues)
421+
422+
# [REST API](#tab/rest)
423+
424+
No REST equivalent provided.
425+
426+
---
427+
373428
::: zone-end
429+
430+

articles/ai-services/agents/how-to/virtual-networks.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -113,11 +113,11 @@ Azure AI Agent Service offers a standard agent configuration with private networ
113113

114114
| Template | Description | Autodeploy |
115115
| ------------------- | -----------------------------------------------| -----------------------|
116-
| `network-secured-agent.bicep` | Deploy a network secured agent setup that uses user-managed identity authentication on the Agent connections. | [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fazureai-samples%2Fmain%2Fscenarios%2FAgents%2Fsetup%2Fnetwork-secured-agent%2Fazuredeploy.json)
116+
| `network-secured-agent.bicep` | Deploy a network secured agent setup that uses user-managed identity authentication on the Agent connections. | [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fazureai-samples%2Fmain%2Fscenarios%2FAgents%2Fsetup%2Fnetwork-secured-agent-thread-storage%2Fazuredeploy.json)
117117

118118
### Option 2: manually deploy the bicep template
119119

120-
1. To manually run the bicep templates, [download the template from GitHub](https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/Agents/setup/network-secured-agent). Download the following from the `network-secured-agent` folder:
120+
1. To manually run the bicep templates, [download the template from GitHub](https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/Agents/setup/network-secured-agent-thread-storage). Download the following from the `network-secured-agent` folder:
121121
1. `main.bicep`
122122
1. `azuredeploy.parameters.json`
123123
1. `modules-network-secured folder`
Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
---
2+
title: Document Intelligence supported Markdown elements
3+
titleSuffix: Azure AI services
4+
description: Description of the supported Markdown elements returned as part of the Layout API response and how to use the response in your applications.
5+
author: laujan
6+
manager: nitinme
7+
ms.service: azure-ai-document-intelligence
8+
ms.topic: conceptual
9+
ms.date: 05/05/2025
10+
ms.author: tonyeiyalla
11+
12+
---
13+
14+
# Understanding Document Intelligence Layout API Markdown Output Format
15+
16+
Azure AI Document Intelligence Layout API can transform your documents into rich Markdown, preserving their original structure and formatting. Just specify `outputContentFormat=markdown` in your request to receive semantically structured content that maintains paragraphs, headings, tables, and other document elements in their proper hierarchy.
17+
18+
This Markdown output elegantly captures the document's original organization while providing standardized, easily consumable content for downstream applications. The preserved semantic structure enables more sophisticated document processing workflows without losing the context and relationships between document elements.
19+
20+
21+
## Markdown elements supported in Layout Analysis
22+
23+
The following Markdown elements are included in Layout API responses:
24+
25+
* Paragraph
26+
* Heading
27+
* Table
28+
* Figure
29+
* Selection Mark
30+
* Formula
31+
* Barcode
32+
* PageNumber/PageHeader/PageFooter
33+
* PageBreak
34+
* KeyValuePairs/Language/Style
35+
* Spans and Content
36+
37+
### Paragraph
38+
39+
Paragraphs represent cohesive blocks of text that belong together semantically. The Layout API maintains paragraph integrity by:
40+
41+
* Preserving paragraph boundaries with empty lines between separate paragraphs
42+
* Using line breaks within paragraphs to maintain the visual structure of the original document
43+
* Maintaining proper text flow that respects the original document's reading order
44+
45+
Here's an example:
46+
47+
``` md
48+
This is paragraph 1.
49+
This is still paragraph 1, even if in another Markdown line.
50+
51+
This is paragraph 2. There is a blank line between paragraph 1 and paragraph 2.
52+
```
53+
---
54+
55+
### Heading
56+
57+
Headings organize document content into a hierarchical structure to make navigation and understanding easier. The Layout API has the following capabilities:
58+
59+
* Uses standard Markdown heading syntax with 1-6 hash symbols (#) corresponding to heading levels.
60+
* Maintains proper spacing with two blank lines before each heading for improved readability.
61+
62+
Here's an example:
63+
64+
``` md
65+
# This is a title
66+
67+
## This is heading 1
68+
69+
### This is heading 2
70+
71+
#### This is heading 3
72+
```
73+
74+
---
75+
76+
### Table
77+
78+
Tables preserve complex structured data in a visually organized format. The Layout API uses HTML table syntax for maximum fidelity and compatibility:
79+
80+
* Implements full HTML table markup (`<table>`, `<tr>`, `<th>`, `<td>`) rather than standard Markdown tables
81+
* Preserves merged cell with HTML rowspan and colspan attributes.
82+
* Preserves table captions with the `<caption>` tag to maintain document context
83+
* Handles complex table structures including headers, cells, and footers
84+
* Maintains proper spacing with two blank lines before each table for improved readability
85+
* Preserves table footnotes as separate paragraph following the table
86+
87+
Here's an example:
88+
89+
``` md
90+
<table>
91+
<caption>Table 1. This is a demo table</caption>
92+
<tr><th>Header</th><th>Header</th></tr>
93+
<tr><td>Cell</td><td>Cell</td></tr>
94+
<tr><td>Cell</td><td>Cell</td></tr>
95+
<tr><td>Cell</td><td>Cell</td></tr>
96+
<tr><td>Footer</td><td>Footer</td></tr>
97+
</table>
98+
This is the footnote of the table.
99+
```
100+
101+
---
102+
103+
### Figure
104+
105+
The Layout API preserves figure elements:
106+
107+
* Encapsulates figure content in `<figure>` tags to maintain semantic distinction from surrounding text
108+
* Preserves figure captions with the `<figcaption>` tag to provide important context
109+
* Preserves figure footnotes as separate paragraphs following the figure container
110+
111+
Here's an example:
112+
113+
``` md
114+
<figure>
115+
<figcaption>Figure 2 This is a figure</figcaption>
116+
117+
Values
118+
300
119+
200
120+
100
121+
0
122+
123+
Jan Feb Mar Apr May Jun Months
124+
125+
</figure>
126+
127+
This is footnote if the figure have.
128+
```
129+
---
130+
131+
### Selection Mark
132+
133+
Selection marks represent checkbox-like elements in forms and documents. The Layout API:
134+
135+
* Uses Unicode characters for visual clarity: ☒ (checked) and ☐ (unchecked)
136+
* Filters out low-confidence checkbox detections (below 0.1 confidence) to improve reliability
137+
* Maintains the semantic relationship between selection marks and their associated text
138+
139+
140+
### Formula
141+
142+
Mathematical formulas are preserved with LaTeX-compatible syntax that allows for rendering of complex mathematical expressions:
143+
144+
* Inline formulas are enclosed in single dollar signs (`$...$`) to maintain text flow
145+
* Block formulas use double dollar signs (`$$...$$`) for standalone display
146+
* Multi-line formulas are represented as consecutive block formulas, preserving mathematical relationships
147+
* Original spacing and formatting are maintained to ensure accurate representation
148+
149+
Here's an example of inline formula, single-line formula block and multiple-lines formula block:
150+
151+
``` md
152+
The mass-energy equivalence formula $E = m c ^ { 2 }$ is an example of an inline formula
153+
154+
$$\frac { n ! } { k ! \left( n - k \right) ! } = \binom { n } { k }$$
155+
156+
$$\frac { p _ { j } } { p _ { 1 } } = \prod _ { k = 1 } ^ { j - 1 } e ^ { - \beta _ { k , k + 1 } \Delta E _ { k , k + 1 } }$$
157+
$$= \exp \left[ - \sum _ { k = 1 } ^ { j - 1 } \beta _ { k , k + 1 } \Delta E _ { k , k + 1 } \right] .$$
158+
```
159+
---
160+
161+
### Barcode
162+
163+
Barcodes and QR codes are represented using Markdown image syntax with added semantic information:
164+
165+
* Uses standard image Markdown syntax with descriptive attributes
166+
* Captures both the barcode type (QR code, barcode, etc.) and its encoded value
167+
* Preserves the semantic relationship between barcodes and surrounding content
168+
169+
Here's an example:
170+
171+
```
172+
![QRCode](barcodes/1.1 "https://www.microsoft.com")
173+
174+
![UPCA](barcodes/1.2 "012345678905")
175+
176+
![barcode type](barcodes/pagenumber.barcodenumber "barcode value/content")
177+
```
178+
---
179+
180+
### PageNumber/PageHeader/PageFooter
181+
182+
Page metadata elements provide context about document pagination but aren't meant to be displayed inline with the main content:
183+
184+
* Enclosed in HTML comments to preserve the information while keeping it hidden from standard Markdown rendering
185+
* Maintains original page structure information that might be valuable for document reconstruction
186+
* Enables applications to understand document pagination without disrupting the content flow
187+
188+
Here's an example:
189+
190+
``` md
191+
<!-- PageHeader="This is page header" -->
192+
193+
<!-- PageFooter="This is page footer" -->
194+
<!-- PageNumber="1" -->
195+
196+
```
197+
---
198+
199+
### PageBreak
200+
201+
To easily figure out which parts belong to which page base on the pure Markdown content, we introduced PageBreak as the delimiter of the pages
202+
203+
Here's an example:
204+
``` md
205+
<!-- PageBreak -->
206+
```
207+
---
208+
209+
### KeyValuePairs/Language/Style
210+
211+
For KeyValuePairs/Language/Style, we map them to Analytics JSON body and not in the Markdown content.
212+
213+
214+
> [!NOTE]
215+
> For more information on Markdown that is currently supported for user content on GitHub.com, *see* [GitHub Flavored Markdown Spec](https://github.github.com/gfm).
216+
217+
## Conclusion
218+
219+
Document Intelligence's Markdown elements provide a powerful way to represent the structure and content of analyzed documents. By understanding and properly utilizing these Markdown elements, you can enhance your document processing workflows and build more sophisticated content extraction applications.
220+
221+
## Next steps
222+
223+
* Try processing your documents with [Document Intelligence Studio](https://formrecognizer.appliedai.azure.com/studio).
224+
225+
* Complete a [Document Intelligence quickstart](../quickstarts/get-started-sdks-rest-api.md?view=doc-intel-3.0.0&preserve-view=true) and get started creating a document processing app in the development language of your choice.

0 commit comments

Comments
 (0)