Merge branch 'main' of https://github.com/MicrosoftDocs/azure-docs-pr into auth-recs

Author: pauljewellmsft

.openpublishing.redirection.json

@@ -3430,6 +3430,12 @@
             "redirect_url": "/azure/ai-services/anomaly-detector/quickstarts/detect-data-anomalies-python",
             "redirect_document_id": false
         },
+        {
+            "source_path_from_root": "/articles/ai-studio/how-to/model-catalog.md",
+            "redirect_url": "/azure/ai-studio/how-to/model-catalog-overview",
+            "redirect_document_id": false
+        },
+
         {
             "source_path_from_root": "/articles/service-fabric/upgrade-managed-disks.md",
             "redirect_url": "/azure/service-fabric/service-fabric-scale-up-primary-node-type",
@@ -4024,6 +4030,11 @@
             "source_path_from_root":"/articles/aks/generation-2-vm-windows.md",
             "redirect_url":"/azure/aks/generation-2-vm",
             "redirect_document_id":false
+        },
+        {
+            "source_path_from_root":"/articles/cosmos-db/high-availability.md",
+            "redirect_url":"/azure/reliability/reliability-cosmos-db-nosql.md",
+            "redirect_document_id":false
         }
     ]
 }

articles/active-directory-b2c/custom-policies-series-sign-up-or-sign-in-federation.md

@@ -225,7 +225,7 @@ Notice the claims transformations we defined in [step 3.2](#step-32---define-cla
 
 Just like in sign-in with a local account, you need to configure the [Microsoft Entra Technical Profiles](active-directory-technical-profile.md), which you use to connect to Microsoft Entra ID storage, to store or read a user social account. 
 
-1. In the `ContosoCustomPolicy.XML` file, locate the `AAD-UserRead` technical profile and then add a new technical profile by using the following code: 
+1. In the `ContosoCustomPolicy.XML` file, locate the `AAD-UserRead` technical profile and then add a new technical profile below it by using the following code: 
 
     ```xml
         <TechnicalProfile Id="AAD-UserWriteUsingAlternativeSecurityId">
@@ -517,6 +517,7 @@ Use the following steps to add a combined local and social account:
     ```xml
         <OutputClaim ClaimTypeReferenceId="authenticationSource" DefaultValue="localIdpAuthentication" AlwaysUseDefaultValue="true" />
     ```
+   Make sure you also add the `authenticationSource` claim in the output claims collection of the `UserSignInCollector` self-asserted technical profile.
 
 1. In the `UserJourneys` section, add a new user journey, `LocalAndSocialSignInAndSignUp` by using the following code: 
 

articles/ai-services/openai/assistants-reference-runs.md

@@ -5,7 +5,7 @@ description: Learn how to use Azure OpenAI's Python & REST API runs with Assista
 manager: nitinme
 ms.service: azure-ai-openai
 ms.topic: conceptual
-ms.date: 02/01/2024
+ms.date: 04/16/2024
 author: mrbullwinkle
 ms.author: mbullwin
 recommendations: false
@@ -585,3 +585,115 @@ Represent a step in execution of a run.
 | `failed_at`| integer or null | The Unix timestamp (in seconds) for when the run step failed.|
 | `completed_at`| integer or null | The Unix timestamp (in seconds) for when the run step completed.|
 | `metadata`| map | Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format. Keys can be a maximum of 64 characters long and values can be a maximum of 512 characters long.|
+
+## Stream a run result (preview)
+
+Stream the result of executing a Run or resuming a Run after submitting tool outputs. You can stream events after:
+* [Create Thread and Run](#create-thread-and-run) 
+* [Create Run](#create-run)
+* [Submit Tool Outputs](#submit-tool-outputs-to-run) 
+
+To stream a result, pass `"stream": true` while creating a run. The response will be a [Server-Sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events) stream.
+
+### Streaming example
+
+```python
+from typing_extensions import override
+from openai import AssistantEventHandler
+ 
+# First, we create a EventHandler class to define
+# how we want to handle the events in the response stream.
+ 
+class EventHandler(AssistantEventHandler):    
+  @override
+  def on_text_created(self, text) -> None:
+    print(f"\nassistant > ", end="", flush=True)
+      
+  @override
+  def on_text_delta(self, delta, snapshot):
+    print(delta.value, end="", flush=True)
+      
+  def on_tool_call_created(self, tool_call):
+    print(f"\nassistant > {tool_call.type}\n", flush=True)
+  
+  def on_tool_call_delta(self, delta, snapshot):
+    if delta.type == 'code_interpreter':
+      if delta.code_interpreter.input:
+        print(delta.code_interpreter.input, end="", flush=True)
+      if delta.code_interpreter.outputs:
+        print(f"\n\noutput >", flush=True)
+        for output in delta.code_interpreter.outputs:
+          if output.type == "logs":
+            print(f"\n{output.logs}", flush=True)
+ 
+# Then, we use the `create_and_stream` SDK helper 
+# with the `EventHandler` class to create the Run 
+# and stream the response.
+ 
+with client.beta.threads.runs.stream(
+  thread_id=thread.id,
+  assistant_id=assistant.id,
+  instructions="Please address the user as Jane Doe. The user has a premium account.",
+  event_handler=EventHandler(),
+) as stream:
+  stream.until_done()
+```
+
+
+## Message delta object
+
+Represents a message delta. For example any changed fields on a message during streaming.
+
+|Name | Type | Description |
+|---  |---   |---         |
+| `id` | string | The identifier of the message, which can be referenced in API endpoints. |
+| `object` | string | The object type, which is always `thread.message.delta`. |
+| `delta` | object | The delta containing the fields that have changed on the Message. |
+
+## Run step delta object
+
+Represents a run step delta. For example any changed fields on a run step during streaming.
+
+|Name | Type | Description |
+|---  |---   |---         |
+| `id` | string | The identifier of the run step, which can be referenced in API endpoints. |
+| `object` | string | The object type, which is always `thread.run.step.delta`. |
+| `delta` | object | The delta containing the fields that have changed on the run step.
+
+## Assistant stream events
+
+Represents an event emitted when streaming a Run. Each event in a server-sent events stream has an event and data property:
+
+```json
+event: thread.created
+data: {"id": "thread_123", "object": "thread", ...}
+```
+
+Events are emitted whenever a new object is created, transitions to a new state, or is being streamed in parts (deltas). For example, `thread.run.created` is emitted when a new run is created, `thread.run.completed` when a run completes, and so on. When an Assistant chooses to create a message during a run, we emit a `thread.message.created` event, a `thread.message.in_progress` event, many thread.`message.delta` events, and finally a `thread.message.completed` event.
+
+|Name | Type | Description |
+|---  |---   |---         |
+| `thread.created` | `data` is a thread. | Occurs when a new thread is created. |
+| `thread.run.created` | `data` is a run. | Occurs when a new run is created. |
+| `thread.run.queued` | `data` is a run. | Occurs when a run moves to a queued status. |
+| `thread.run.in_progress` | `data` is a run. | Occurs when a run moves to an in_progress status. |
+| `thread.run.requires_action` | `data` is a run. | Occurs when a run moves to a `requires_action` status. |
+| `thread.run.completed` | `data` is a run. | Occurs when a run is completed. |
+| `thread.run.failed` | `data` is a run. | Occurs when a run fails. |
+| `thread.run.cancelling` | `data` is a run. | Occurs when a run moves to a `cancelling` status. |
+| `thread.run.cancelled` | `data` is a run. | Occurs when a run is cancelled. |
+| `thread.run.expired` | `data` is a run. | Occurs when a run expires. |
+| `thread.run.step.created` | `data` is a run step. | Occurs when a run step is created. |
+| `thread.run.step.in_progress` | `data` is a run step. | Occurs when a run step moves to an `in_progress` state. | 
+| `thread.run.step.delta` | `data` is a run step delta. | Occurs when parts of a run step are being streamed. |
+| `thread.run.step.completed` | `data` is a run step. | Occurs when a run step is completed. |
+| `thread.run.step.failed` | `data` is a run step. | Occurs when a run step fails. |
+| `thread.run.step.cancelled` | `data` is a run step. | Occurs when a run step is cancelled. |
+| `thread.run.step.expired` | `data` is a run step. | Occurs when a run step expires. |
+| `thread.message.created` | `data` is a message. | Occurs when a message is created. |
+| `thread.message.in_progress` | `data` is a message. | Occurs when a message moves to an in_progress state. | 
+| `thread.message.delta` | `data` is a message delta. | Occurs when parts of a Message are being streamed. |
+| `thread.message.completed` | `data` is a message. | Occurs when a message is completed. |
+| `thread.message.incomplete` | `data` is a message. | Occurs when a message ends before it is completed. |
+| `error` | `data` is an error. | Occurs when an error occurs. This can happen due to an internal server error or a timeout. |
+| `done` | `data` is `[DONE]` | Occurs when a stream ends. |

articles/ai-services/openai/concepts/use-your-data.md

@@ -62,59 +62,15 @@ The [Integrated Vector Database in vCore-based Azure Cosmos DB for MongoDB](/azu
 
 For some data sources such as uploading files from your local machine (preview) or data contained in a blob storage account (preview), Azure AI Search is used. When you choose the following data sources, your data is ingested into an Azure AI Search index.
 
->[!TIP]
->If you use Azure Cosmos DB (except for its vCore-based API for MongoDB), you may be eligible for the [Azure AI Advantage offer](/azure/cosmos-db/ai-advantage), which provides the equivalent of up to $6,000 in Azure Cosmos DB throughput credits.
-
-|Data source  | Description  |
+|Data ingested through Azure AI Search  | Description  |
 |---------|---------|
 | [Azure AI Search](/azure/search/search-what-is-azure-search)  | Use an existing Azure AI Search index with Azure OpenAI On Your Data.      |
-| [Azure Cosmos DB](/azure/cosmos-db/introduction)  | Azure Cosmos DB's API for Postgres and vCore-based API for MongoDB offer natively integrated vector indexing; therefore, they don't require Azure AI Search. However, its other APIs do require Azure AI Search for vector indexing. Azure Cosmos DB for NoSQL's natively integrated vector database debuts in mid-2024.     |
 |Upload files (preview)      | Upload files from your local machine to be stored in an Azure Blob Storage database, and ingested into Azure AI Search.         |
 |URL/Web address (preview)        | Web content from the URLs is stored in Azure Blob Storage.         |
 |Azure Blob Storage (preview) | Upload files from Azure Blob Storage to be ingested into an Azure AI Search index.         |
 
 :::image type="content" source="../media/use-your-data/azure-databases-and-ai-search.png" lightbox="../media/use-your-data/azure-databases-and-ai-search.png" alt-text="Diagram of vector indexing services.":::
 
-# [Vector Database in Azure Cosmos DB for MongoDB](#tab/mongo-db)
-
-### Prerequisites
-* [vCore-based Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/introduction) account
-* A deployed [embedding model](../concepts/understand-embeddings.md)
-
-### Limitations
-* Only vCore-based Azure Cosmos DB for MongoDB is supported.
-* The search type is limited to [Integrated Vector Database in Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/vector-search) with an Azure OpenAI embedding model.
-* This implementation works best on unstructured and spatial data.
-
-  
-### Data preparation
-
-Use the script provided on [GitHub](https://github.com/microsoft/sample-app-aoai-chatGPT/tree/main/scripts#data-preparation) to prepare your data.
-
-<!--### Add your data source in Azure OpenAI Studio
-
-To add vCore-based Azure Cosmos DB for MongoDB as a data source, you'll need an existing Azure Cosmos DB for MongoDB index containing your data, and a deployed Azure OpenAI Ada embeddings model that will be used for vector search.
-
-1. In the [Azure OpenAI portal](https://oai.azure.com/portal) chat playground, select **Add your data**. In the panel that appears, select ** vCore-based Azure Cosmos DB for MongoDB** as the data source. 
-1. Select your Azure subscription and database account, then connect to your Azure Cosmos DB account by providing your Azure Cosmos DB account username and password.
-    
-    :::image type="content" source="../media/use-your-data/add-mongo-data-source.png" alt-text="A screenshot showing the screen for adding Mongo DB as a data source in Azure OpenAI Studio." lightbox="../media/use-your-data/add-mongo-data-source.png":::
-
-1. **Select Database**. In the dropdown menus, select the database name, database collection, and index name that you want to use as your data source. Select the embedding model deployment you would like to use for vector search on this data source, and acknowledge that you'll incur charges for using vector search. Then select **Next**.
-
-    :::image type="content" source="../media/use-your-data/select-mongo-database.png" alt-text="A screenshot showing the screen for adding Mongo DB settings in Azure OpenAI Studio." lightbox="../media/use-your-data/select-mongo-database.png":::
--->
-
-### Index field mapping
-
-When you add your vCore-based Azure Cosmos DB for MongoDB data source, you can specify data fields to properly map your data for retrieval.
-
-* Content data (required): One or more provided fields to be used to ground the model on your data. For multiple fields, separate the values with commas, with no spaces.
-* File name/title/URL: Used to display more information when a document is referenced in the chat.
-* Vector fields (required): Select the field in your database that contains the vectors.
-
-:::image type="content" source="../media/use-your-data/mongo-index-mapping.png" alt-text="A screenshot showing the index field mapping options for Mongo DB." lightbox="../media/use-your-data/mongo-index-mapping.png":::
-
 # [Azure AI Search](#tab/ai-search)
 
 You might want to consider using an Azure AI Search index when you either want to:
@@ -179,6 +135,33 @@ If you want to implement additional value-based criteria for query execution, yo
 
 [!INCLUDE [ai-search-ingestion](../includes/ai-search-ingestion.md)]
 
+
+# [Vector Database in Azure Cosmos DB for MongoDB](#tab/mongo-db)
+
+### Prerequisites
+* [vCore-based Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/introduction) account
+* A deployed [embedding model](../concepts/understand-embeddings.md)
+
+### Limitations
+* Only vCore-based Azure Cosmos DB for MongoDB is supported.
+* The search type is limited to [Integrated Vector Database in Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/vector-search) with an Azure OpenAI embedding model.
+* This implementation works best on unstructured and spatial data.
+
+  
+### Data preparation
+
+Use the script provided on [GitHub](https://github.com/microsoft/sample-app-aoai-chatGPT/tree/main/scripts#data-preparation) to prepare your data.
+
+### Index field mapping
+
+When you add your vCore-based Azure Cosmos DB for MongoDB data source, you can specify data fields to properly map your data for retrieval.
+
+* Content data (required): One or more provided fields to be used to ground the model on your data. For multiple fields, separate the values with commas, with no spaces.
+* File name/title/URL: Used to display more information when a document is referenced in the chat.
+* Vector fields (required): Select the field in your database that contains the vectors.
+
+:::image type="content" source="../media/use-your-data/mongo-index-mapping.png" alt-text="A screenshot showing the index field mapping options for Mongo DB." lightbox="../media/use-your-data/mongo-index-mapping.png":::
+
 # [Azure Blob Storage (preview)](#tab/blob-storage)
 
 You might want to use Azure Blob Storage as a data source if you want to connect to existing Azure Blob Storage and use files stored in your containers.

articles/ai-services/openai/concepts/use-your-image-data.md

@@ -8,7 +8,7 @@ ms.service: azure-ai-openai
 ms.topic: quickstart
 author: aahill
 ms.author: aahi
-ms.date: 11/02/2023
+ms.date: 05/09/2024
 recommendations: false
 ---
 
@@ -17,7 +17,7 @@ recommendations: false
 Use this article to learn how to provide your own image data for GPT-4 Turbo with Vision, Azure OpenAI’s vision model. GPT-4 Turbo with Vision on your data allows the model to generate more customized and targeted answers using Retrieval Augmented Generation based on your own images and image metadata. 
 
 > [!IMPORTANT]
-> This article is for using your data on the GPT-4 Turbo with Vision model. If you are interested in using your data for text-based models, see [Use your text data](./use-your-data.md).  
+> Once the GPT4-Turbo with vision preview model is deprecated, you will no longer be able to use Azure OpenAI On your image data. To implement a Retrieval Augmented Generation (RAG) solution with image data, see the following sample on [github](https://github.com/Azure-Samples/azure-search-openai-demo/).
 
 ## Prerequisites 
 

articles/ai-services/openai/how-to/use-web-app.md

@@ -7,7 +7,7 @@ ms.service: azure-ai-openai
 ms.topic: how-to
 author: aahill
 ms.author: aahi
-ms.date: 03/27/2024
+ms.date: 05/09/2024
 recommendations: false
 ---
 
@@ -22,7 +22,7 @@ Along with Azure OpenAI Studio, APIs and SDKs, you can also use the available st
 
 - Publishing creates an Azure App Service in your subscription. It might incur costs depending on the [pricing plan](https://azure.microsoft.com/pricing/details/app-service/windows/) you select. When you're done with your app, you can delete it from the Azure portal.
 - By default, the app will be deployed with the Microsoft identity provider already configured, restricting access to the app to members of your Azure tenant. To add or modify authentication:
-
+- gpt-4 vision-preview models are not supported.
     1. Go to the [Azure portal](https://portal.azure.com/#home) and search for the app name you specified during publishing. Select the web app, and go to the **Authentication** tab on the left navigation menu. Then select **Add an identity provider**. 
 
         :::image type="content" source="../media/quickstarts/web-app-authentication.png" alt-text="Screenshot of the authentication page in the Azure portal." lightbox="../media/quickstarts/web-app-authentication.png":::
@@ -77,7 +77,7 @@ Consider either clicking the **watch** or **star** buttons on the web app's [Git
 
 You can enable chat history for your users of the web app. When you enable the feature, your users will have access to their individual previous queries and responses. 
 
-To enable chat history, deploy or redeploy your model as a web app using [Azure OpenAI Studio](https://oai.azure.com/portal)
+To enable chat history, deploy or redeploy your model as a web app using [Azure OpenAI Studio](https://oai.azure.com/portal).
 
 :::image type="content" source="../media/use-your-data/enable-chat-history.png" alt-text="A screenshot of the chat history enablement button on Azure OpenAI studio." lightbox="../media/use-your-data/enable-chat-history.png":::
 
@@ -94,4 +94,4 @@ Deleting your web app does not delete your Cosmos DB instance automatically. To
 
 ## Next steps
 * [Prompt engineering](../concepts/prompt-engineering.md)
-* [Azure openAI on your data](../concepts/use-your-data.md)
+* [Azure OpenAI on your data](../concepts/use-your-data.md)

articles/ai-services/openai/includes/create-resource-portal.md

@@ -89,7 +89,9 @@ As an option, you can add a private endpoint for access to your resource. Select
 
 1. Confirm your configuration settings, and select **Create**.
 
-The Azure portal displays a notification when the new resource is available.
+1. The Azure portal displays a notification when the new resource is available. Select **Go to resource**.
+
+   :::image type="content" source="../media/create-resource/create-resource-go-to-resource.png" alt-text="Screenshot showing the Go to resource button in the Azure portal.":::
 
 ## Deploy a model
 

articles/ai-services/openai/media/create-resource/create-resource-go-to-resource.png

@@ -159,7 +159,7 @@ items:
         href: ../cognitive-services-virtual-networks.md?context=/azure/ai-services/openai/context/context
       - name: Encryption of data at rest
         href: encrypt-data-at-rest.md
-      - name: Manage identity
+      - name: Managed identity
         href: ./how-to/managed-identity.md
       - name: Use Azure OpenAI on your data securely
         href: ./how-to/use-your-data-securely.md