Merge pull request #2222 from HeidiSteen/heidist-freshness

[azure search] Updates on vector dimensions, RAG quickstart-tutorial roles and regions

Author: prmerger-automator[bot]

articles/search/cognitive-search-attach-cognitive-services.md

@@ -9,7 +9,7 @@ ms.custom:
   - ignite-2023
   - ignite-2024
 ms.topic: how-to
-ms.date: 11/19/2024
+ms.date: 01/09/2025
 ---
 
 # Attach an Azure AI multi-service resource to a skillset in Azure AI Search
@@ -57,6 +57,8 @@ Using the Azure portal or newer preview REST APIs and beta SDK packages, you can
 
 As with keys, the details you provide about the Azure AI Services resource are used for billing, not connections.  All API requests made by Azure AI Search to Azure AI services for built-in skills processing continue to be internal and managed by Microsoft.
 
+The subdomain URL must include a unique name (for example, `https://hereismyuniquename.cognitiveservices.azure.com`). If the service was created through the Azure portal, a unique subdomain is automatically generated as part of your service setup. Ensure that your service includes a unique subdomain before using it with the Azure AI Search integration.
+
 ### Example: system-assigned managed identity
 
 Identity is set to null.

articles/search/cognitive-search-skill-azure-openai-embedding.md

@@ -9,7 +9,7 @@ ms.custom:
   - ignite-2023
   - build-2024
 ms.topic: reference
-ms.date: 10/16/2024
+ms.date: 01/09/2025
 ---
 
 #	Azure OpenAI Embedding skill
@@ -46,7 +46,7 @@ Parameters are case-sensitive.
 | `deploymentId`   | The name of the deployed Azure OpenAI embedding model. The model should be an embedding model, such as text-embedding-ada-002. See the [List of Azure OpenAI models](/azure/ai-services/openai/concepts/models) for supported models.|
 | `authIdentity`   | A user-managed identity used by the search service for connecting to Azure OpenAI. You can use either a [system or user managed identity](search-howto-managed-identities-data-sources.md). To use a system manged identity, leave `apiKey` and `authIdentity` blank. The system-managed identity is used automatically. A managed identity must have [Cognitive Services OpenAI User](/azure/ai-services/openai/how-to/role-based-access-control#azure-openai-roles) permissions to send text to Azure OpenAI. |
 | `modelName` | This property is required if your skillset is created using the 2024-05-01-preview or 2024-07-01 REST API. Set this property to the deployment name of an Azure OpenAI embedding model deployed on the provider specified through `resourceUri` and identified through `deploymentId`. Currently, the supported values are `text-embedding-ada-002`, `text-embedding-3-large`, and `text-embedding-3-small`.  |
-| `dimensions` | (Optional, introduced in the 2024-05-01-preview REST API). The dimensions of embeddings that you would like to generate if the model supports reducing the embedding dimensions. Supported ranges are listed below. Defaults to the maximum dimensions for each model if not specified. For skillsets created using the 2023-10-01-preview, dimensions are fixed at 1536. |
+| `dimensions` | Optional, starting in the 2024-05-01-preview REST API, the dimensions of embeddings that you would like to generate, assuming the model supports a range of dimensions. Supported ranges are listed below, and currently only apply to the text-embedding-3 model series. The default is the maximum dimensions for each model. For skillsets created using earlier RESt API versions dating back to the 2023-10-01-preview, dimensions are fixed at 1536. When setting the dimensions property on a skill, make sure to set the `dimensions` property on the [vector field definition](vector-search-how-to-create-index.md#add-a-vector-field-to-the-fields-collection) to the same value. |
 
 ## Supported dimensions by `modelName`
 

articles/search/search-get-started-rag.md

@@ -8,7 +8,7 @@ ms.service: azure-ai-search
 ms.custom:
   - ignite-2024
 ms.topic: quickstart
-ms.date: 01/07/2025
+ms.date: 01/09/2025
 ---
 
 # Quickstart: Generative search (RAG) with grounding data from Azure AI Search
@@ -17,13 +17,25 @@ This quickstart shows you how to send queries to a chat completion model for a c
 
 ## Prerequisites
 
-- An Azure subscription. [Create one for free](https://azure.microsoft.com/free/).
+- [Visual Studio Code](https://code.visualstudio.com/download) with the [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python) and the [Jupyter package](https://pypi.org/project/jupyter/). For more information, see [Python in Visual Studio Code](https://code.visualstudio.com/docs/languages/python).
 
-- [Azure AI Search](search-create-service-portal.md), Basic tier or higher so that you can [enable semantic ranker](semantic-how-to-enable-disable.md). Region must be the same one used for Azure OpenAI.
+- An Azure subscription with permissions to assign roles. [Create one for free](https://azure.microsoft.com/free/).
 
-- [Azure OpenAI](https://aka.ms/oai/access) resource with a deployment of `gpt-4o`, `gpt-4o-mini`, or equivalent chat completion model, in the same region as Azure AI Search.
+- [Azure OpenAI](/azure/ai-services/openai/how-to/create-resource)
 
-- [Visual Studio Code](https://code.visualstudio.com/download) with the [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python) and the [Jupyter package](https://pypi.org/project/jupyter/). For more information, see [Python in Visual Studio Code](https://code.visualstudio.com/docs/languages/python).
+  - [Choose a region](/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#global-standard-model-availability) that supports the chat completion model you want to use (gpt-4o, gpt-4o-mini, or equivalent model). 
+  - [Deploy the chat completion model](/azure/ai-studio/how-to/deploy-models-openai) in Azure AI Foundry or [use another approach](/azure/ai-services/openai/how-to/working-with-models).
+
+- [Azure AI Search](search-create-service-portal.md)
+
+  - Same region as Azure OpenAI.
+  - Basic tier or higher is recommended.
+  - [Enable semantic ranking](semantic-how-to-enable-disable.md).
+  - [Enable role-based access control (see below)](#configure-access).
+
+To meet the same-region requirement, start by reviewing the [regions for the chat model](/azure/ai-services/openai/concepts/models#model-summary-table-and-region-availability) you want to use. Once you identify a region, confirm that Azure AI Search is available in the [same region](search-region-support.md#azure-public-regions).
+
+Make sure you know the name of the deployed model, and have the endpoints for both Azure resources at hand. You'll provide this information in the steps that follow.
 
 ## Download file
 
@@ -37,20 +49,12 @@ Requests to the search endpoint must be authenticated and authorized. You can us
 
 You're setting up two clients, so you need permissions on both resources.
 
-Azure AI Search is receiving the query request from your local system. Assign yourself the **Search Index Data Reader** role assignment for that task. If you're also creating and loading the hotel sample index, add **Search Service Contributor** and **Search Index Data Contributor** roles as well.
+Azure AI Search is receiving the query request from your local system. Assign yourself the **Search Index Data Reader** role assignment if the hotels sample index already exists. If it doesn't exist, assign yourself **Search Service Contributor** and **Search Index Data Contributor** roles so that you can create and query the index.
 
-Azure OpenAI is receiving the (query) "Can you recommend a few hotels" from your local system, plus its receiving the search results (source) from the search service. Assign yourself and the search service the **Cognitive Services OpenAI User** role.
+Azure OpenAI is receiving the query and the search results from your local system. Assign yourself the **Cognitive Services OpenAI User** role on Azure OpenAI.
 
 1. Sign in to the [Azure portal](https://portal.azure.com).
 
-1. Configure Azure AI Search to use a system-assigned managed identity so that you can you give it role assignments:
-
-    1. In the Azure portal, [find your search service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices).
- 
-    1. On the left menu, select **Settings** > **Identity**.
-
-    1. On the System assigned tab, set status to **On**.
-
 1. Configure Azure AI Search for role-based access:
 
     1. In the Azure portal, find your Azure AI Search service.
@@ -61,20 +65,20 @@ Azure OpenAI is receiving the (query) "Can you recommend a few hotels" from your
 
     1. On the left menu, select **Access control (IAM)**.
 
-    1. On Azure AI Search, make sure you have permissions to create, load, and query a search index:
+    1. On Azure AI Search, select these roles to create, load, and query a search index, and then assign them to your Microsoft Entra ID user identity:
 
        - **Search Index Data Contributor**
        - **Search Service Contributor**
 
-    1. On Azure OpenAI, select **Access control (IAM)** to assign yourself and the search service identity permissions on Azure OpenAI. The code for this quickstart runs locally. Requests to Azure OpenAI originate from your system. Also, search results from the search engine are passed to Azure OpenAI. For these reasons, both you and the search service need permissions on Azure OpenAI.
+    1. On Azure OpenAI, select **Access control (IAM)** to assign this role to yourself on Azure OpenAI:
 
        - **Cognitive Services OpenAI User**
 
 It can take several minutes for permissions to take effect.
 
 ## Create an index
 
-We recommend the hotels-sample-index, which can be created in minutes and runs on any search service tier. This index is created using built-in sample data.
+A search index provides grounding data for the chat model. We recommend the hotels-sample-index, which can be created in minutes and runs on any search service tier. This index is created using built-in sample data.
 
 1. In the Azure portal, [find your search service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices).
 
@@ -211,6 +215,34 @@ In the remaining sections, you set up API calls to Azure OpenAI and Azure AI Sea
 
 1. On the **Overview** home page, select the link to view the endpoints. Copy the URL. An example endpoint might look like `https://example.openai.azure.com/`.
 
+## Create a virtual environment
+
+In this step, switch back to your local system and Visual Studio Code. We recommend that you create a virtual environment so that you can install the dependencies in isolation.
+
+1. In Visual Studio Code, open the folder containing Quickstart-RAG.ipynb.
+
+1. Press Ctrl-shift-P to open the command palette, search for "Python: Create Environment", and then select `Venv` to create a virtual environment in the current workspace.
+
+1. Select Quickstart-RAG\requirements.txt for the dependencies.
+
+It takes several minutes to create the environment. When the environment is ready, continue to the next step.
+
+## Sign in to Azure
+
+You're using Microsoft Entra ID and role assignments for the connection. Make sure you're logged in to the same tenant and subscription as Azure AI Search and Azure OpenAI. You can use the Azure CLI on the command line to show current properties, change properties, and to sign in. For more information, see [Connect without keys](search-get-started-rbac.md). 
+
+Run each of the following commands in sequence.
+
+```azure-cli
+az account show
+
+az account set --subscription <PUT YOUR SUBSCRIPTION ID HERE>
+
+az login --tenant <PUT YOUR TENANT ID HERE>
+```
+
+You should now be logged in to Azure from your local device.
+
 ## Set up the query and chat thread
 
 This section uses Visual Studio Code and Python to call the chat completion APIs on Azure OpenAI.
@@ -271,18 +303,20 @@ This section uses Visual Studio Code and Python to call the chat completion APIs
     Sources:\n{sources}
     """
 
-    # Query is the question being asked. It's sent to the search engine and the LLM.
+    # Query is the question being asked. It's sent to the search engine and the chat model
     query="Can you recommend a few hotels with complimentary breakfast?"
 
-    # Set up the search results and the chat thread.
-    # Retrieve the selected fields from the search index related to the question.
+    # Search results are created by the search client
+    # Search results are composed of the top 5 results and the fields selected from the search index
+    # Search results include the top 5 matches to your query
     search_results = search_client.search(
         search_text=query,
         top=5,
         select="Description,HotelName,Tags"
     )
     sources_formatted = "\n".join([f'{document["HotelName"]}:{document["Description"]}:{document["Tags"]}' for document in search_results])
 
+    # Send the search results and the query to the LLM to generate a response based on the prompt.
     response = openai_client.chat.completions.create(
         messages=[
             {
@@ -293,6 +327,7 @@ This section uses Visual Studio Code and Python to call the chat completion APIs
         model=AZURE_DEPLOYMENT_MODEL
     )
 
+    # Here is the response from the chat model.
     print(response.choices[0].message.content)
     ```
 
@@ -321,6 +356,8 @@ This section uses Visual Studio Code and Python to call the chat completion APIs
 
     If you get an **Authorization failed** error message, wait a few minutes and try again. It can take several minutes for role assignments to become operational.
 
+    If you get a **Resource not found** error message, check the resource URIs and make sure the API version on the chat model is valid.
+
     Otherwise, to experiment further, change the query and rerun the last step to better understand how the model works with the grounding data.
 
     You can also modify the prompt to change the tone or structure of the output.
@@ -331,7 +368,7 @@ This section uses Visual Studio Code and Python to call the chat completion APIs
 
 Azure AI Search supports [complex types](search-howto-complex-data-types.md) for nested JSON structures. In the hotels-sample-index, `Address` is an example of a complex type, consisting of `Address.StreetAddress`, `Address.City`, `Address.StateProvince`, `Address.PostalCode`, and `Address.Country`. The index also has complex collection of `Rooms` for each hotel.
 
-If your index has complex types, your query can provide those fields if you first convert the search results output to JSON, and then pass the JSON to the LLM. The following example adds complex types to the request. The formatting instructions include a JSON specification.
+If your index has complex types, your query can provide those fields if you first convert the search results output to JSON, and then pass the JSON to the chat model. The following example adds complex types to the request. The formatting instructions include a JSON specification.
 
 ```python
 import json

articles/search/tutorial-rag-build-solution-index-schema.md

@@ -106,6 +106,8 @@ A minimal index for LLM is designed to store chunks of content. It typically inc
 
    Like the basic schema, it's organized around chunks. The `chunk_id` uniquely identifies each chunk. The `text_vector` field is an embedding of the chunk. The nonvector `chunk` field is a readable string. The `title` maps to a unique metadata storage path for the blobs. The `parent_id` is the only parent-level field, and it's a base64-encoded version of the parent file URI. 
 
+   In integrated vectorization workloads like the one used in this tutorial series, the `dimensions` property on your vector fields should be identical to the number of `dimensions` generated by the embedding skill used to vectorize your data. In this series, we use the Azure OpenAI embedding skill, which calls the text-embedding-3-large model on Azure OpenAI. The skill is specified in the next tutorial. We set dimensions to 1024 in both the vector field and in the skill definition.
+
    The schema also includes a `locations` field for storing generated content that's created by the [indexing pipeline](tutorial-rag-build-solution-pipeline.md).
 
    ```python

articles/search/tutorial-rag-build-solution-pipeline.md

@@ -10,7 +10,7 @@ ms.service: azure-ai-search
 ms.custom:
   - ignite-2024
 ms.topic: tutorial
-ms.date: 11/19/2024
+ms.date: 01/09/2025
 ---
 
 # Tutorial: Build an indexing pipeline for RAG on Azure AI Search
@@ -51,7 +51,7 @@ If you don't have an Azure subscription, create a [free account](https://azure.m
 
 Open or create a Jupyter notebook (`.ipynb`) in Visual Studio Code to contain the scripts that comprise the pipeline. Initial steps install packages and collect variables for the connections. After you complete the setup steps, you're ready to begin with the components of the indexing pipeline. 
 
-Let's start with the index schema from the [previous tutorial](tutorial-rag-build-solution-index-schema.md). It's organized around vectorized and nonvectorized chunks. It includes a `locations` field that stores AI-generated content created by the skillset.  
+Let's start with the index schema from the [previous tutorial](tutorial-rag-build-solution-index-schema.md). It's organized around vectorized and nonvectorized chunks. It includes a `locations` field that stores AI-generated content created by the skillset.
 
 ```python
 from azure.identity import DefaultAzureCredential

articles/search/tutorial-rag-build-solution-query.md

@@ -10,7 +10,7 @@ ms.service: azure-ai-search
 ms.custom:
   - ignite-2024
 ms.topic: tutorial
-ms.date: 10/04/2024
+ms.date: 01/09/2025
 ---
 
 # Tutorial: Search your data using a chat model (RAG in Azure AI Search)

articles/search/vector-search-how-to-create-index.md

@@ -9,7 +9,7 @@ ms.service: azure-ai-search
 ms.custom:
   - ignite-2024
 ms.topic: how-to
-ms.date: 08/05/2024
+ms.date: 01/09/2025
 ---
 
 # Create a vector index
@@ -281,7 +281,7 @@ Vector fields are characterized by [their data type](/rest/api/searchservice/sup
 1. Define a vector field with the following attributes. You can store one generated embedding per field. For each vector field:
 
    + `type` must be a [vector data types](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields). `Collection(Edm.Single)` is the most common for embedding models.
-   + `dimensions` is the number of dimensions generated by the embedding model. For text-embedding-ada-002, it's 1536.
+   + `dimensions` is the number of dimensions generated by the embedding model. For text-embedding-ada-002, it's fixed at 1536. For the text-embedding-3 model series, there's a range of values. If you're using integrated vectorization and an embedding skill to generate vectors, make sure this property is set to the [same dimensions value](cognitive-search-skill-azure-openai-embedding.md#supported-dimensions-by-modelname) used by the embedding skill.
    + `vectorSearchProfile` is the name of a profile defined elsewhere in the index.
    + `searchable` must be true.
    + `retrievable` can be true or false. True returns the raw vectors (1536 of them) as plain text and consumes storage space. Set to true if you're passing a vector result to a downstream app.

articles/search/vector-search-vectorizer-ai-services-vision.md

@@ -38,7 +38,7 @@ The Azure AI Vision vectorizer supports `text`, `imageUrl`, and `imageBinary` ve
 
 ## Expected field dimensions
 
-A field configured with the Azure AI Vision vectorizer should have a dimensions value of 1024.
+A vector field configured with the Azure AI Vision vectorizer should have a dimensions value of 1024.
 
 ## Sample definition
 

articles/search/vector-search-vectorizer-azure-machine-learning-ai-studio-catalog.md

@@ -58,7 +58,7 @@ Which vector query types are supported by the Azure AI Foundry model catalog vec
 
 ## Expected field dimensions
 
-The expected field dimensions for a field configured with an Azure AI Foundry model catalog vectorizer depend on the `modelName` that is configured.
+The expected field dimensions for a vector field configured with an Azure AI Foundry model catalog vectorizer depend on the `modelName` that is configured.
 
 | `modelName` | Expected dimensions |
 |--------------------|-------------|