Merge branch 'main' into release-azure-search

Author: v-alje

articles/ai-services/content-safety/whats-new.md

@@ -47,9 +47,7 @@ The groundedness detection API includes a correction feature that automatically
 
 The Prompt Shields API and groundedness detection API are now generally available (GA). Follow a quickstart to try them out.
 * [Prompt Shields quickstart](/azure/ai-services/content-safety/quickstart-jailbreak)
-* [Groundedness detection quickstart](/azure/ai-services/content-safety/quickstart-groundedness?tabs=curl)
-
-
+* [Protected Material quickstart](/azure/ai-services/content-safety/quickstart-protected-material?tabs=curl)
 ## July 2024
 
 ### Custom categories (standard) API public preview

articles/ai-services/openai/concepts/content-filter.md

@@ -43,13 +43,15 @@ Text and image models support Drugs as an additional classification. This catego
 | Sexual  | Sexual describes language related to anatomical organs and genitals, romantic relationships and sexual acts, acts portrayed in erotic or affectionate terms, including those portrayed as an assault or a forced sexual violent act against one’s will. <br><br> This includes but is not limited to:<ul><li>Vulgar content</li><li>Prostitution</li><li>Nudity and Pornography</li><li>Abuse</li><li>Child exploitation, child abuse, child grooming</li></ul>   |
 | Violence  | Violence describes language related to physical actions intended to hurt, injure, damage, or kill someone or something; describes weapons, guns and related entities. <br><br>This includes, but isn't limited to:  <ul><li>Weapons</li><li>Bullying and intimidation</li><li>Terrorist and violent extremism</li><li>Stalking</li></ul>  |
 | Self-Harm  | Self-harm describes language related to physical actions intended to purposely hurt, injure, damage one’s body or kill oneself. <br><br> This includes, but isn't limited to: <ul><li>Eating Disorders</li><li>Bullying and intimidation</li></ul>  |
-| Protected Material for Text<sup>*</sup> | Protected material text describes known text content (for example, song lyrics, articles, recipes, and selected web content) that can be outputted by large language models.
+| Protected Material for Text<sup>1</sup> | Protected material text describes known text content (for example, song lyrics, articles, recipes, and selected web content) that can be outputted by large language models.
 | Protected Material for Code | Protected material code describes source code that matches a set of source code from public repositories, which can be outputted by large language models without proper citation of source repositories.
 |User Prompt Attacks |User prompt attacks are User Prompts designed to provoke the Generative AI model into exhibiting behaviors it was trained to avoid or to break the rules set in the System Message. Such attacks can vary from intricate roleplay to subtle subversion of the safety objective. |
 |Indirect Attacks |Indirect Attacks, also referred to as Indirect Prompt Attacks or Cross-Domain Prompt Injection Attacks, are a potential vulnerability where third parties place malicious instructions inside of documents that the Generative AI system can access and process. Requires [document embedding and formatting](#embedding-documents-in-your-prompt). |
+| Groundedness<sup>2</sup> | Groundedness detection flags whether the text responses of large language models (LLMs) are grounded in the source materials provided by the users. Ungrounded material refers to instances where the LLMs produce information that is non-factual or inaccurate from what was present in the source materials. Requires [document embedding and formatting](#embedding-documents-in-your-prompt). |
 
-<sup>*</sup> If you're an owner of text material and want to submit text content for protection, [file a request](https://aka.ms/protectedmaterialsform).
+<sup>1</sup> If you're an owner of text material and want to submit text content for protection, [file a request](https://aka.ms/protectedmaterialsform).
 
+<sup>2</sup> Not available in non-streaming scenarios; only available for streaming scenarios. The following regions support Groundedness Detection: Central US, East US, France Central, and Canada East 
 
 [!INCLUDE [severity-levels text, four-level](../../content-safety/includes/severity-levels-text-four.md)]
 
@@ -328,24 +330,27 @@ When annotations are enabled as shown in the code snippets below, the following
 |indirect attacks|detected (true or false), </br>filtered (true or false)|
 |protected material text|detected (true or false), </br>filtered (true or false)|
 |protected material code|detected (true or false), </br>filtered (true or false), </br>Example citation of public GitHub repository where code snippet was found, </br>The license of the repository|
+|Groundedness | detected (true or false)</br>filtered (true or false) </br>details (`completion_end_offset`, `completion_start_offset`) |
 
 When displaying code in your application, we strongly recommend that the application also displays the example citation from the annotations. Compliance with the cited license may also be required for Customer Copyright Commitment coverage.
 
 See the following table for the annotation availability in each API version:
 
-|Category |2024-02-01 GA| 2024-04-01-preview | 2023-10-01-preview | 2023-06-01-preview| 
+|Category |2024-10-01-preview|2024-02-01 GA| 2024-04-01-preview | 2023-10-01-preview | 2023-06-01-preview| 
 |--|--|--|--|
-| Hate | ✅ |✅ |✅ |✅ |
-| Violence | ✅ |✅ |✅ |✅ |
-| Sexual |✅ |✅ |✅ |✅ |
-| Self-harm |✅ |✅ |✅ |✅ |
-| Prompt Shield for user prompt attacks|✅ |✅ |✅ |✅ |
-|Prompt Shield for indirect attacks|  | ✅ | | |
-|Protected material text|✅ |✅ |✅ |✅ |
-|Protected material code|✅ |✅ |✅ |✅ |
-|Profanity blocklist|✅ |✅ |✅ |✅ |
-|Custom blocklist| | ✅ |✅ |✅ |
-
+| Hate | ✅|✅ |✅ |✅ |✅ |
+| Violence | ✅|✅ |✅ |✅ |✅ |
+| Sexual |✅ |✅|✅ |✅ |✅ |
+| Self-harm |✅|✅|✅ |✅ |✅ |
+| Prompt Shield for user prompt attacks|✅|✅|✅ |✅ |✅ |
+|Prompt Shield for indirect attacks|   | | ✅ | | |
+|Protected material text|✅|✅ |✅ |✅ |✅ |
+|Protected material code|✅|✅ |✅ |✅ |✅ |
+|Profanity blocklist|✅|✅ |✅ |✅ |✅ |
+|Custom blocklist|✅| | ✅ |✅ |✅ |
+|Groundedness<sup>1</sup>|✅| |  | |  |
+
+<sup>1</sup> Not available in non-streaming scenarios; only available for streaming scenarios. The following regions support Groundedness Detection: Central US, East US, France Central, and Canada East 
 
 # [OpenAI Python 1.x](#tab/python-new)
 
@@ -710,6 +715,39 @@ violence  : @{filtered=False; severity=safe}
 
 For details on the inference REST API endpoints for Azure OpenAI and how to create Chat and Completions, follow [Azure OpenAI Service REST API reference guidance](../reference.md). Annotations are returned for all scenarios when using any preview API version starting from `2023-06-01-preview`, as well as the GA API version `2024-02-01`.
 
+### Groundedness
+
+#### Annotate only 
+
+Returns offsets referencing the ungrounded completion content. 
+
+```json
+{ 
+  "ungrounded_material": { 
+    "details": [ 
+       { 
+         "completion_end_offset": 127, 
+         "completion_start_offset": 27 
+       } 
+   ], 
+    "detected": true, 
+    "filtered": false 
+ } 
+} 
+```
+
+#### Annotate and filter 
+
+Blocks completion content when ungrounded completion content was detected. 
+
+```json
+{ "ungrounded_material": { 
+    "detected": true, 
+    "filtered": true 
+  } 
+} 
+```
+
 ### Example scenario: An input prompt containing content that is classified at a filtered category and severity level is sent to the completions API
 
 ```json

articles/ai-services/speech-service/regions.md

@@ -54,7 +54,7 @@ The following regions are supported for Speech service features such as speech t
 | Qatar         | Qatar Central        | `qatarcentral`<sup>3,8</sup>            |
 | US            | Central US           | `centralus`                             |
 | US            | East US              | `eastus` <sup>1,2,4,5,7,9,11,12</sup>   |
-| US            | East US 2            | `eastus2` <sup>1,2,4,5,12</sup>         |
+| US            | East US 2            | `eastus2` <sup>1,2,4,5,10,12</sup>         |
 | US            | North Central US     | `northcentralus` <sup>4,6,12</sup>      |
 | US            | South Central US     | `southcentralus` <sup>1,2,4,5,6,7,10,12</sup> |
 | US            | West Central US      | `westcentralus` <sup>3,5</sup>          |

articles/ai-services/speech-service/text-to-speech-avatar/what-is-text-to-speech-avatar.md

@@ -69,11 +69,11 @@ Sample code for text to speech avatar is available on [GitHub](https://github.co
 
 - Throughout an avatar real-time session or batch content creation, the text-to-speech, speech-to-text, Azure OpenAI, or other Azure services are charged separately.
 - Refer to [text to speech avatar pricing note](../text-to-speech.md#text-to-speech-avatar) to learn how billing works for the text-to-speech avatar feature.
-- For the detailed pricing, see [Speech service pricing](https://azure.microsoft.com/pricing/details/cognitive-services/speech-services/). Note that avatar pricing will only be visible for service regions where the feature is available, including Southeast Asia, North Europe, West Europe, Sweden Central, South Central US, and West US 2.
+- For the detailed pricing, see [Speech service pricing](https://azure.microsoft.com/pricing/details/cognitive-services/speech-services/). Note that avatar pricing will only be visible for service regions where the feature is available, including Southeast Asia, North Europe, West Europe, Sweden Central, South Central US, East US 2, and West US 2.
 
 ## Available locations
 
-The text to speech avatar feature is only available in the following service regions: Southeast Asia, North Europe, West Europe, Sweden Central, South Central US, and West US 2.
+The text to speech avatar feature is only available in the following service regions: Southeast Asia, North Europe, West Europe, Sweden Central, South Central US, East US 2, and West US 2.
 
 ### Responsible AI
 

articles/ai-studio/how-to/develop/llama-index.md

@@ -45,7 +45,10 @@ To run this tutorial, you need:
     ```bash
     pip install -U llama-index-llms-azure-inference
     pip install -U llama-index-embeddings-azure-inference
-    ``` 
+    ```
+
+    > [!IMPORTANT]
+    > Using the [Azure AI model inference service](https://aka.ms/aiservices/inference) requires version `0.2.4` for `llama-index-llms-azure-inference` or `llama-index-embeddings-azure-inference`.
 
 ## Configure the environment
 
@@ -67,7 +70,7 @@ export AZURE_INFERENCE_ENDPOINT="<your-model-endpoint-goes-here>"
 export AZURE_INFERENCE_CREDENTIAL="<your-key-goes-here>"
 ```
 
-Once configured, create a client to connect to the endpoint. The parameter `model_name` in the constructor is not required for endpoints serving a single model, like serverless endpoints.
+Once configured, create a client to connect to the endpoint.
 
 ```python
 import os
@@ -80,7 +83,20 @@ llm = AzureAICompletionsModel(
 ```
 
 > [!TIP]
-> If your model is an OpenAI model deployed to Azure OpenAI service or AI services resource, configure the client as indicated at [Azure OpenAI models](#azure-openai-models).
+> If your model is an OpenAI model deployed to Azure OpenAI service or AI services resource, configure the client as indicated at [Azure OpenAI models and Azure AI model inference service](#azure-openai-models-and-azure-ai-model-infernece-service).
+
+If your endpoint is serving more than one model, like with the [Azure AI model inference service](../../ai-services/model-inference.md) or [GitHub Models](https://github.com/marketplace/models), you have to indicate `model_name` parameter:
+
+```python
+import os
+from llama_index.llms.azure_inference import AzureAICompletionsModel
+
+llm = AzureAICompletionsModel(
+    endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
+    credential=os.environ["AZURE_INFERENCE_CREDENTIAL"],
+    model_name="mistral-large-2407",
+)
+```
 
 Alternatively, if your endpoint support Microsoft Entra ID, you can use the following code to create the client:
 
@@ -112,22 +128,23 @@ llm = AzureAICompletionsModel(
 )
 ```
 
-### Azure OpenAI models
+### Azure OpenAI models and Azure AI model infernece service
 
-If you are using Azure OpenAI models with key-based authentication, you need to pass the authentication key in the header `api-key`, which is the one expected in the Azure OpenAI service and in Azure AI Services. This configuration is not required if you are using Microsoft Entra ID (formerly known as Azure AD). The following example shows how to configure the client:
+If you are using Azure OpenAI models or [Azure AI model inference service](../../ai-services/model-inference.md), ensure you have at least version `0.2.4` of the LlamaIndex integration. Use `api_version` parameter in case you need to select a specific `api_version`. For the [Azure AI model inference service](../../ai-services/model-inference.md), you need to pass `model_name` parameter:
 
 ```python
-import os
 from llama_index.llms.azure_inference import AzureAICompletionsModel
 
 llm = AzureAICompletionsModel(
     endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
-    credential="",
-    client_kwargs={"headers" : { "api-key": os.environ["AZURE_INFERENCE_CREDENTIAL"] } }
+    credential=os.environ["AZURE_INFERENCE_CREDENTIAL"],
+    model_name="gpt-4o",
+    api_version="2024-05-01-preview",
 )
 ```
 
-Notice that `credentials` is still being passed with an empty value since it's a required parameter.
+> [!TIP]
+> Using a wrong `api_version` or one not supported by the model results in a `ResourceNotFound` exception.
 
 ### Inference parameters
 

articles/machine-learning/concept-customer-managed-keys.md

@@ -124,20 +124,17 @@ Data that previously was stored in Azure Cosmos DB in your subscription, is stor
 
 Pipelines metadata that previously was stored in a storage account in a managed resource group, is now stored on the storage account in your subscription that is associated to the Azure Machine Learning workspace. Since this Azure Storage resource is managed separately in your subscription, you're responsible to configure encryption settings on it.
 
-To opt in for this preview, set the `enableServiceSideCMKEncryption` on a REST API or in your Bicep or Resource Manager template. You can also use Azure portal. Preview availability varies by [workspace kind](concept-workspace.md):
-
-| Kind | Supported |
-| ----- | ----- |
-| Default | Yes |
-| Hub | No |
-| Project | No |
+To opt in for this preview, set the `enableServiceSideCMKEncryption` on a REST API or in your Bicep or Resource Manager template. You can also use Azure portal.
 
 :::image type="content" source="./media/concept-customer-managed-keys/cmk-service-side-encryption.png" alt-text="Screenshot of the encryption tab with the option for server side encryption selected." lightbox="./media/concept-customer-managed-keys/cmk-service-side-encryption.png":::
 
 > [!NOTE]
 > During this preview key rotation and data labeling capabilities are not supported. Server-side encryption is currently not supported in reference to an Azure Key Vault for storing your encryption key that has public network access disabled.
 
-For a template that creates a workspace with service-side encryption of metadata, see [https://github.com/azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.machinelearningservices/machine-learning-workspace-cmk-service-side-encryption](https://github.com/azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.machinelearningservices/machine-learning-workspace-cmk-service-side-encryption).
+For templates that create a workspace with service-side encryption of metadata, see 
+
+- [Bicep template for creating default workspace](https://github.com/azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.machinelearningservices/machine-learning-workspace-cmk-service-side-encryption).
+- [Bicep template for creating hub workspace](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.machinelearningservices/aistudio-cmk-service-side-encryption).
 
 ## hbi_workspace flag
 

articles/search/includes/quickstarts/java.md

@@ -5,7 +5,7 @@ ms.service: azure-ai-search
 ms.custom:
   - ignite-2023
 ms.topic: include
-ms.date: 10/07/2024
+ms.date: 11/01/2024
 ---
 
 Build a Java console application using the [Azure.Search.Documents](/java/api/overview/azure/search) library to create, load, and query a search index. 
@@ -65,12 +65,12 @@ Use the following tools to create this quickstart.
         <dependency>
           <groupId>com.azure</groupId>
           <artifactId>azure-search-documents</artifactId>
-          <version>11.5.2</version>
+          <version>11.7.3</version>
         </dependency>
         <dependency>
           <groupId>com.azure</groupId>
           <artifactId>azure-core</artifactId>
-          <version>1.34.0</version>
+          <version>1.53.0</version>
         </dependency>
         <dependency>
           <groupId>junit</groupId>

articles/search/keyless-connections.md

@@ -1,11 +1,10 @@
 ---
 title: Use keyless connections with Azure AI Search
-description: Use keyless connections with an Azure Identity library for authentication and authorization with Azure AI Search.
+description: Use keyless connections with an Azure Identity library for Microsoft Entra ID authentication and authorization with Azure AI Search.
 ms.topic: how-to
-ms.date: 06/05/2024
+ms.date: 10/30/2024
 author: HeidiSteen
 ms.author: heidist
-ms.reviewer: scaddie
 ms.custom: devx-track-dotnet, devx-track-extended-java, devx-track-js, devx-track-python, Keyless-dotnet, Keyless-java, Keyless-js, Keyless-python, build-2024-intelligent-apps
 #customer intent: As a developer, I want to use keyless connections so that I don't leak secrets.
 ---
@@ -34,7 +33,7 @@ Before continuing with this article, you need an Azure AI Search resource to wor
 
 ### Install Azure Identity client library
 
-Before working locally without keyless, update your AI Search enabled code with the Azure Identity client library.
+To use a keyless approach, update your AI Search enabled code with the Azure Identity client library.
 
 #### [.NET](#tab/csharp)
 
@@ -211,22 +210,21 @@ search_index_client = SearchIndexClient(
 
 ---
 
-
 ## Local development
 
-Local development without keyless includes these steps:
+Local development using roles includes these steps:
 
-- Assign your personal identity with RBAC roles on the specific resource.
-- Use a tool to authenticate with Azure.
+- Assign your personal identity to RBAC roles on the specific resource.
+- Use a tool like the Azure CLI or Azure PowerShell to authenticate with Azure.
 - Establish environment variables for your resource.
 
 ### Roles for local development
 
-As a local developer, your Azure identity needs full control of your service. This control is provided with RBAC roles. To manage your resource during development, these are the suggested roles:
+As a local developer, your Azure identity needs full control over data plane operations. These are the suggested roles:
 
-- Search Service Contributor
-- Search Index Data Contributor
-- Search Index Data Reader
+- Search Service Contributor, create and manage objects
+- Search Index Data Contributor, load an index
+- Search Index Data Reader, query an index
 
 Find your personal identity with one of the following tools. Use that identity as the `<identity-id>` value.
 
@@ -253,7 +251,7 @@ Find your personal identity with one of the following tools. Use that identity a
         --assignee "<identity-id>" \
         --scope "/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>"
     ```
-    
+
 #### [Azure PowerShell](#tab/azure-powershell)
 
 1. Sign in with PowerShell.
@@ -277,13 +275,12 @@ Find your personal identity with one of the following tools. Use that identity a
 #### [Azure portal](#tab/portal)
 
 1. Use the steps found here: [find the user object ID](/partner-center/find-ids-and-domain-names#find-the-user-object-id) in the Azure portal.
-    
-2. Use the steps found at [open the Add role assignment page](search-security-rbac.md) in the Azure portal.
-    
+
+1. Use the steps found at [open the Add role assignment page](search-security-rbac.md) in the Azure portal.
+
 ---
-    
-Where applicable, replace `<identity-id>`, `<subscription-id>`, and `<resource-group-name>` with your actual values. 
 
+Where applicable, replace `<identity-id>`, `<subscription-id>`, and `<resource-group-name>` with your actual values. 
 
 ### Authentication for local development