Skip to content

Commit 1d0c1b0

Browse files
Merge pull request #270155 from aahill/elasticsearch-data-source
new include section
2 parents 4477709 + 48af74e commit 1d0c1b0

File tree

2 files changed

+23
-10
lines changed

2 files changed

+23
-10
lines changed

articles/ai-services/openai/concepts/use-your-data.md

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,7 @@ Mapping these fields correctly helps ensure the model has better response and ci
132132

133133
If you want to implement additional value-based criteria for query execution, you can set up a [search filter](/azure/search/search-filters) using the `filter` parameter in the [REST API](../references/azure-search.md).
134134

135+
[!INCLUDE [ai-search-ingestion](../includes/ai-search-ingestion.md)]
135136

136137
# [Azure Cosmos DB for MongoDB vCore](#tab/mongo-db)
137138

@@ -211,12 +212,15 @@ To modify the schedule, you can use the [Azure portal](https://portal.azure.com/
211212

212213
1. Select **Save**.
213214

215+
[!INCLUDE [ai-search-ingestion](../includes/ai-search-ingestion.md)]
216+
214217
# [Upload files (preview)](#tab/file-upload)
215218

216219
Using Azure OpenAI Studio, you can upload files from your machine to try Azure OpenAI On Your Data, and optionally creating a new Azure Blob Storage account and Azure AI Search resource. The service then stores the files to an Azure storage container and performs ingestion from the container. You can use the [quickstart](../use-your-data-quickstart.md) article to learn how to use this data source option.
217220

218221
:::image type="content" source="../media/quickstarts/add-your-data-source.png" alt-text="A screenshot showing options for selecting a data source in Azure OpenAI Studio." lightbox="../media/quickstarts/add-your-data-source.png":::
219222

223+
[!INCLUDE [ai-search-ingestion](../includes/ai-search-ingestion.md)]
220224

221225
# [URL/Web address (preview)](#tab/web-pages)
222226

@@ -294,16 +298,6 @@ Along with using Elasticsearch databases in Azure OpenAI Studio, you can also us
294298

295299
---
296300

297-
### How data is ingested into Azure AI search
298-
299-
Data is ingested into Azure AI search using the following process:
300-
301-
1. Ingestion assets are created in Azure AI Search resource and Azure storage account. Currently these assets are: indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) in the search resource, and a container (later called the chunks container) in the Azure storage account. You can specify the input Azure storage container using the [Azure OpenAI studio](https://oai.azure.com/), or the [ingestion API (preview)](/rest/api/azureopenai/ingestion-jobs).
302-
303-
2. Data is read from the input container, contents are opened and chunked into small chunks with a maximum of 1,024 tokens each. If vector search is enabled, the service calculates the vector representing the embeddings on each chunk. The output of this step (called the "preprocessed" or "chunked" data) is stored in the chunks container created in the previous step.
304-
305-
3. The preprocessed data is loaded from the chunks container, and indexed in the Azure AI Search index.
306-
307301
## Deploy to a copilot (preview) or web app
308302

309303
After you connect Azure OpenAI to your data, you can deploy it using the **Deploy to** button in Azure OpenAI studio.
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
manager: nitinme
3+
ms.service: azure-ai-studio
4+
ms.custom:
5+
ms.topic: include
6+
ms.date: 03/25/2024
7+
ms.author: aahi
8+
author: aahill
9+
---
10+
11+
### How data is ingested into Azure AI search
12+
13+
Data is ingested into Azure AI search using the following process:
14+
15+
1. Ingestion assets are created in Azure AI Search resource and Azure storage account. Currently these assets are: indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) in the search resource, and a container (later called the chunks container) in the Azure storage account. You can specify the input Azure storage container using the [Azure OpenAI studio](https://oai.azure.com/), or the [ingestion API (preview)](/rest/api/azureopenai/ingestion-jobs).
16+
17+
2. Data is read from the input container, contents are opened and chunked into small chunks with a maximum of 1,024 tokens each. If vector search is enabled, the service calculates the vector representing the embeddings on each chunk. The output of this step (called the "preprocessed" or "chunked" data) is stored in the chunks container created in the previous step.
18+
19+
3. The preprocessed data is loaded from the chunks container, and indexed in the Azure AI Search index.

0 commit comments

Comments
 (0)