You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Learn how to build AI applications using logic apps](https://www.youtube.com/watch?v=tiU5yCvMW9o)
122
134
123
-
### Data ingestion workflow
135
+
### Ingest data workflow
124
136
125
137
To save considerable time and effort when you build an ingestion pipeline, implement the following pattern with any data source. This pattern encapsulates all the advantages and benefits currently offered by Standard workflows in single-tenant Azure Logic Apps.
126
138
127
-
Each step in this pattern not only provides faster performance when run in a stateless workflow, but also makes sure that the AI seamlessly extracts all the crucial information from your data files. This approach simplifies not only the coding aspect but also guarantees that your workflows have effective authentication, monitoring, and deployment processes in place.
139
+
Each step in this pattern makes sure that the AI seamlessly extracts all the crucial information from your data files. If run as a stateless workflow, this pattern also provides faster performance. This approach simplifies not only the coding aspect but also guarantees that your workflows have effective authentication, monitoring, and deployment processes in place.
128
140
129
141
:::image type="content" source="media/azure-ai/ingest-data-workflow.png" alt-text="Screenshot shows Azure portal, Standard workflow designer, and workflow operations that implement data ingestion functionality." lightbox="media/azure-ai/ingest-data-workflow.png":::
| 1 | Check for new data. |**When an HTTP request is received**| A trigger that polls or waits for new data to arrive, either based on a scheduled recurrence or in response to specific events respectively. For example, an event might be a new file uploaded to a specific storage system, such as SharePoint, OneDrive, or Azure Blob Storage. <br><br>In this example, a Request trigger operation waits for an HTTP call from another endpoint. |
134
-
| 2 | Get the data's location. |**HTTP**| An action that gets the URL for the document in the specified storage system. |
135
-
| 3 |Get the data. |**Compose**| A **Data Operations** action that concatenates various items. <br><br>This example concatenates key-value information about the document. |
136
-
| 4 | Tokenize the data. |**HTTP**| An action that [tokenizes the output from the **Compose** action](../../ai-services/openai/overview.md#tokens). |
137
-
| 5 | Convert tokenized data to JSON. |**Parse JSON**| A **Data Operations** action that converts the tokenized output into a JSON array. |
138
-
| 6 | Select JSON array items. |**Select**| A **Data Operations** action that selects items from the JSON array. |
145
+
| 1 | Check for new data. |**When an HTTP request is received**| A trigger that either polls or waits for new data to arrive, either based on a scheduled recurrence or in response to specific events respectively. Such an event might be a new file that's uploaded to a specific storage system, such as SharePoint, OneDrive, or Azure Blob Storage. <br><br>In this example, the **Request** trigger operation waits for an HTTP or HTTPS request sent from another endpoint. The request includes the URL for a new uploaded document. |
146
+
| 2 | Get the data. |**HTTP**| An **HTTP**action that retrieves the uploaded document using the file URL from the trigger output. |
147
+
| 3 |Compose document details. |**Compose**| A **Data Operations** action that concatenates various items. <br><br>This example concatenates key-value information about the document. |
148
+
| 4 | Tokenize the data. |**HTTP**| An **HTTP**action that calls a custom function that [tokenizes](../../ai-services/openai/overview.md#tokens) the output from the **Compose** action. |
149
+
| 5 | Convert tokenized data to JSON. |**Parse JSON**| A **Data Operations** action that converts the tokenized string output into a JSON array. |
150
+
| 6 | Select JSON array items. |**Select**| A **Data Operations** action that selects multiple items from the JSON array. |
139
151
| 7 | Generate the embeddings. |**Get multiple embeddings**| An **Azure OpenAI** action that creates embeddings for each JSON array item. |
140
-
| 8 | Select the embeddings. |**Select**| A **Data Operations** action that selects embeddings. |
152
+
| 8 | Select embeddings and other information. |**Select**| A **Data Operations** action that selects embeddings and other document information. |
141
153
| 9 | Index the data. |**Index documents**| An **Azure AI Search** action that indexes the data based on each selected embedding. |
142
154
143
155
### Chat workflow
@@ -150,16 +162,16 @@ The following pattern is only one example that shows how a chat workflow might l
| 1 |Capture the prompt. |**When an HTTP request is received**| A trigger that waits for the customer's question in JSON format. This example uses the Request trigger. |
154
-
| 2 |Train the model. |**Compose**| A **Data Operations** action that includes input to train the model. |
155
-
| 3 |Train the model. |**Compose**| A **Data Operations** action that includes sample customer questions input. |
156
-
| 4 |Train the model. |**Compose**| A **Data Operations** action that includes input for the search query. |
157
-
| 5 | Generate the query. |**Execute JavaScript Code**| An **Inline Code** action that uses JavaScript to create search queries for the vector database by using the outputs from the previous**Compose** actions. |
158
-
| 6 | Convert the query into an embedding. |**Get the chat completions**| An **Azure OpenAI** action that converts search queries into vector embeddings. |
159
-
| 7 | Get an embedding. |**Gets a single embedding**| An **Azure OpenAI** action that gets the vector embedding. |
160
-
| 8 |Run the vector search. |**Vector search**| An **Azure AI Search** action that executes searches in the vector database. |
161
-
| 9 | Create the prompt. |**Execute JavaScript Code**| An **Inline Code** action that uses JavaScript to build a prompt. |
162
-
| 10 | Perform chat completion. |**Get the chat completions**| An **Azure OpenAI** action that connects to the chat completion API, which guarantees reliable responses in chat conversations. |
165
+
| 1 |Wait for input prompt. |**When an HTTP request is received**| A trigger that either polls or waits for new data to arrive, either based on a scheduled recurrence or in response to specific events respectively. <br><br>In this example, the **Request** trigger waits for and captures the customer's question. |
166
+
| 2 |Input system message for the model. |**Compose**| A **Data Operations** action that provides input to train the model. |
167
+
| 3 |Input sample questions and responses. |**Compose**| A **Data Operations** action that provides sample customer questions and associated roles to train the model. |
168
+
| 4 |Input system message for search query. |**Compose**| A **Data Operations** action that provides search query input to train the model. |
169
+
| 5 | Generate search query. |**Execute JavaScript Code**| An **Inline Code** action that uses JavaScript to create a search query for the vector store, based on the outputs from the preceding**Compose** actions. |
170
+
| 6 | Convert query to embedding. |**Get chat completions**| An **Azure OpenAI** action that connects to the chat completion API, which guarantees reliable responses in chat conversations. <br><br>In this example, the action accepts search queries and roles as input to the model and returns vector embeddings as output. |
171
+
| 7 | Get an embedding. |**Get an embedding**| An **Azure OpenAI** action that gets a single vector embedding. |
172
+
| 8 |Search the vector database. |**Search vectors**| An **Azure AI Search** action that executes searches in the vector store. |
173
+
| 9 | Create prompt. |**Execute JavaScript Code**| An **Inline Code** action that uses JavaScript to build prompts. |
174
+
| 10 | Perform chat completion. |**Get chat completions**| An **Azure OpenAI** action that connects to the chat completion API, which guarantees reliable responses in chat conversations. <br><br>In this example, the action accepts prompts and roles as input to the model and returns model-generated responses as output. |
163
175
| 11 | Return a response. |**Response**| A **Request** action that returns the results to the caller when you use the **Request** trigger. |
0 commit comments