You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> This capability is in preview and is subject to the
18
19
> [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
19
20
20
-
To integrate enterprise data and services with AI technologies, you can use the Azure OpenAI and Azure AI Search built-in connectors in automated Standard logic app workflows. These connectors support multiple authentication types, such as API keys, Microsoft Entra ID, and managed identities. They can also connect to Azure OpenAI Service and Azure AI Search endpoints behind firewalls so that your workflows securely connect to your AI resources in Azure.
21
+
To integrate enterprise data and services with AI technologies, you can use the **Azure OpenAI** and **Azure AI Search** built-in connectors in Standard logic app workflows. These connectors support multiple authentication types, such as API keys, Microsoft Entra ID, and managed identities. They also can connect to Azure OpenAI Service and Azure AI Search endpoints behind firewalls so that your workflows securely connect to your AI resources in Azure.
21
22
22
-
This guide provides an overview and examples for how to use the Azure OpenAI and Azure AI Search connector operations in your workflow.
23
+
This guide provides an overview and examples for how to use the **Azure OpenAI** and **Azure AI Search** connector operations in your workflow.
23
24
24
25
-[What is Azure OpenAI Service](../../ai-services/openai/overview.md)
25
26
-[What is Azure AI Search](../../search/search-what-is-azure-search.md)
26
27
27
-
## Why use Azure Logic Apps to integrate with AI services?
28
+
## Why use Azure Logic Apps with AI services?
28
29
29
30
Usually, building AI solutions involves several key steps and requires a few building blocks. Primarily, you need to have a dynamic ingestion pipeline and a chat interface that can communicate with large language models (LLMs) and vector databases.
30
31
32
+
> [!TIP]
33
+
>
34
+
> To learn more, you can ask Azure Copilot these questions:
35
+
>
36
+
> -*What is a dynamic ingestion pipeline in AI?*
37
+
> -*What is a vector database in AI?*
38
+
>
39
+
> To find Azure Copilot, on the [Azure portal](https://portal.azure.com) toolbar, select **Copilot**.
40
+
31
41
You can assemble various components, not only to perform data ingestion but also to provide a robust backend for the chat interface. This backend facilitates entering prompts and generates dependable responses during interactions. However, creating the code to manage and control all these elements can prove challenging, which is the case for most solutions.
32
42
33
43
Azure Logic Apps offers a low code approach and simplifies backend management by providing prebuilt connectors that you use as building blocks to streamline the backend process. This approach lets you focus on sourcing your data and making sure that search results provide current and relevant information. With these AI connectors, your workflow acts as an orchestration engine that transfers data between AI services and other components that you want to integrate.
@@ -42,15 +52,24 @@ For more information, see the following resources:
42
52
43
53
### Azure OpenAI
44
54
45
-
Azure OpenAI Service provides access to [OpenAI's language models](https://openai.com/product), which include GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and the Embeddings model series. With the Azure OpenAI connector, your workflow can connect to Azure OpenAI Service and get OpenAI embeddings for your data or generate chat completions.
55
+
Azure OpenAI Service provides access to [OpenAI's language models](https://openai.com/product), which include GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and the Embeddings model series. With the **Azure OpenAI** connector, your workflow can connect to Azure OpenAI Service and get OpenAI embeddings for your data or generate chat completions.
56
+
57
+
> [!TIP]
58
+
>
59
+
> To learn more, you can ask Azure Copilot these questions:
60
+
>
61
+
> -*What is an embedding in AI?*
62
+
> -*What is a chat completion in AI?*
63
+
>
64
+
> To find Azure Copilot, on the [Azure portal](https://portal.azure.com) toolbar, select **Copilot**.
46
65
47
66
| Logic app | Environment | Connector version |
48
67
|-----------|-------------|-------------------|
49
68
|**Standard**| Single-tenant Azure Logic Apps and App Service Environment v3 (Windows plans only) | Built-in connector, which appears in the connector gallery under **Runtime** > **In-App** and is [service provider-based](../custom-connector-overview.md#service-provider-interface-implementation). The built-in connector can directly access Azure virtual networks without using an on-premises data gateway. <br><br>For more information, see [Azure OpenAI built-in connector reference](/azure/logic-apps/connectors/built-in/reference/openai). |
50
69
51
70
### Azure AI Search
52
71
53
-
Azure AI Search is platform for AI-powered information retrieval that helps developers build rich search experiences and generative AI apps by combining large language models with enterprise data. With the Azure AI Search connector, your workflow can connect to Azure AI Search to index documents and perform vector searches on your data.
72
+
Azure AI Search is platform for AI-powered information retrieval that helps developers build rich search experiences and generative AI apps by combining large language models with enterprise data. With the **Azure AI Search** connector, your workflow can connect to Azure AI Search to index documents and perform vector searches on your data.
54
73
55
74
| Logic app | Environment | Connector version |
56
75
|-----------|-------------|-------------------|
@@ -145,12 +164,13 @@ Each step in this pattern makes sure that the AI seamlessly extracts all the cru
145
164
| 1 | Check for new data. |**When an HTTP request is received**| A trigger that either polls or waits for new data to arrive, either based on a scheduled recurrence or in response to specific events respectively. Such an event might be a new file that's uploaded to a specific storage system, such as SharePoint, OneDrive, or Azure Blob Storage. <br><br>In this example, the **Request** trigger operation waits for an HTTP or HTTPS request sent from another endpoint. The request includes the URL for a new uploaded document. |
146
165
| 2 | Get the data. |**HTTP**| An **HTTP** action that retrieves the uploaded document using the file URL from the trigger output. |
147
166
| 3 | Compose document details. |**Compose**| A **Data Operations** action that concatenates various items. <br><br>This example concatenates key-value information about the document. |
148
-
| 4 | Tokenize the data. |**HTTP**| An **HTTP** action that calls a custom Azure function that [batches and tokenizes](../../ai-services/openai/overview.md#tokens) the output from the **Compose** action. |
149
-
| 5 | Convert tokenized data to JSON. |**Parse JSON**| A **Data Operations** action that converts the tokenized string output into a JSON array. |
150
-
| 6 | Select JSON array items. |**Select**| A **Data Operations** action that selects multiple items from the JSON array. |
151
-
| 7 | Generate the embeddings. |**Get multiple embeddings**| An **Azure OpenAI** action that creates embeddings for each JSON array item. |
152
-
| 8 | Select embeddings and other information. |**Select**| A **Data Operations** action that selects embeddings and other document information. |
153
-
| 9 | Index the data. |**Index documents**| An **Azure AI Search** action that indexes the data based on each selected embedding. |
167
+
| 4 | Create token string. |**Parse a document**| A **Data Operations** action that produces a [token string](../../ai-services/openai/overview.md#tokens) using the output from the **Compose** action. |
168
+
| 5 | Create content chunks. |**Chunk text**| A **Data Operations** action that splits the token string into pieces, based on either the number of characters or tokens per content chunk. |
169
+
| 6 | Convert tokenized data to JSON. |**Parse JSON**| A **Data Operations** action that converts the token string chunks into a JSON array. |
170
+
| 7 | Select JSON array items. |**Select**| A **Data Operations** action that selects multiple items from the JSON array. |
171
+
| 8 | Generate the embeddings. |**Get multiple embeddings**| An **Azure OpenAI** action that creates embeddings for each JSON array item. |
172
+
| 9 | Select embeddings and other information. |**Select**| A **Data Operations** action that selects embeddings and other document information. |
173
+
| 10 | Index the data. |**Index documents**| An **Azure AI Search** action that indexes the data based on each selected embedding. |
Copy file name to clipboardExpand all lines: articles/logic-apps/parse-document-chunk-text.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,8 +3,8 @@ title: Parse document or chunk text
3
3
description: Parse a document or chunk text for Standard workflows in Azure Logic Apps.
4
4
services: logic-apps
5
5
ms.suite: integration
6
-
ms.reviewer: estfan, azla
7
6
ms.collection: ce-skilling-ai-copilot
7
+
ms.reviewer: estfan, azla
8
8
ms.topic: how-to
9
9
ms.date: 07/26/2024
10
10
# Customer intent: As a developer using Azure Logic Apps, I want to parse a document or chunk text that I want to use with Azure AI operations for my Standard workflow in Azure Logic Apps.
@@ -18,18 +18,18 @@ ms.date: 07/26/2024
18
18
> This capability is in preview and is subject to the
19
19
> [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
20
20
21
-
Sometimes you have to convert content into token formor break down a large document into smaller pieces before you can use this content with some actions. For example, the such as **Azure AI Search** or **Azure OpenAI** expect tokenized input and can handle only a limited number of tokens, which are words or chunks of characters.
21
+
Sometimes you have to convert content into token form, which are words or chunks of characters, or divide a large document into smaller pieces before you can use this content with some actions. For example, the **Azure AI Search** or **Azure OpenAI**actions expect tokenized input and can handle only a limited number of tokens.
22
22
23
-
For these scenarios, use the **Data Operations** actions named **Parse a document** and **Chunk text** in your Standard logic app workflow. These actions respectively convert content, such as a PDF document, CSV file, Excel file, and so on, into tokenized string output and then split the string into pieces, based on the number of tokens or characters. You can then reference and use these outputs with subsequent actions in your workflow.
23
+
For these scenarios, use the **Data Operations** actions named **Parse a document** and **Chunk text** in your Standard logic app workflow. These actions respectively transform content, such as a PDF document, CSV file, Excel file, and so on, into tokenized string output and then split the string into pieces, based on the number of tokens or characters. You can then reference and use these outputs with subsequent actions in your workflow.
24
24
25
25
> [!TIP]
26
26
>
27
27
> To learn more, you can ask Azure Copilot these questions:
28
28
>
29
-
> -*What is parsing in AI?*
30
29
> -*What is a token in AI?*
31
30
> -*What is tokenized input?*
32
31
> -*What is tokenized string output?*
32
+
> -*What is parsing in AI?*
33
33
> -*What is chunking in AI?*
34
34
>
35
35
> To find Azure Copilot, on the [Azure portal](https://portal.azure.com) toolbar, select **Copilot**.
@@ -76,7 +76,7 @@ If you use other content sources, such as Azure Blob Storage, SharePoint, OneDri
76
76
77
77
:::image type="content" source="media/parse-document-chunk-text/parse-document.png" alt-text="Screenshot shows sample workflow with Body output in the action named Parse a document." lightbox="media/parse-document-chunk-text/parse-document.png":::
78
78
79
-
1. Under the **Parse a document** action, add the actions that you want to work with the tokenized output string, for example, **Chunk text**, which this guide describes later.
79
+
1. Under the **Parse a document** action, add the actions that you want to work with the tokenized string output, for example, **Chunk text**, which this guide describes later.
80
80
81
81
## Parse a document - Reference
82
82
@@ -192,7 +192,7 @@ The following example includes other actions that create a complete workflow pat
192
192
| 2 | Get the content. |**HTTP**| An **HTTP** action that retrieves the uploaded document using the file URL from the trigger output. |
193
193
| 3 | Compose document details. |**Compose**| A **Data Operations** action that concatenates various items. <br><br>This example concatenates key-value information about the document. |
194
194
| 4 | Create token string. |**Parse a document**| A **Data Operations** action that produces a tokenized string using the output from the **Compose** action. |
195
-
| 5 | Create content chunks. |**Chunk text**| A **Data Operations** action that splits the token string into pieces, based on either the number of characters or tokens per content chunk. |
195
+
| 5 | Create content chunks. |**Chunk text**| A **Data Operations** action that splits the token string into pieces, based on either the number of characters or tokens per content chunk. |
196
196
| 6 | Convert tokenized and chunked text to JSON. |**Parse JSON**| A **Data Operations** action that converts the chunked output into a JSON array. |
197
197
| 7 | Select JSON array items. |**Select**| A **Data Operations** action that selects multiple items from the JSON array. |
198
198
| 8 | Generate the embeddings. |**Get multiple embeddings**| An **Azure OpenAI** action that creates embeddings for each JSON array item. |
0 commit comments