Merge pull request #274934 from MicrosoftDocs/main

5/9/2024 PM Publish

Author: Taojunshen

.openpublishing.redirection.json

@@ -3430,6 +3430,12 @@
             "redirect_url": "/azure/ai-services/anomaly-detector/quickstarts/detect-data-anomalies-python",
             "redirect_document_id": false
         },
+        {
+            "source_path_from_root": "/articles/ai-studio/how-to/model-catalog.md",
+            "redirect_url": "/azure/ai-studio/how-to/model-catalog-overview",
+            "redirect_document_id": false
+        },
+
         {
             "source_path_from_root": "/articles/service-fabric/upgrade-managed-disks.md",
             "redirect_url": "/azure/service-fabric/service-fabric-scale-up-primary-node-type",
@@ -4024,6 +4030,11 @@
             "source_path_from_root":"/articles/aks/generation-2-vm-windows.md",
             "redirect_url":"/azure/aks/generation-2-vm",
             "redirect_document_id":false
+        },
+        {
+            "source_path_from_root":"/articles/cosmos-db/high-availability.md",
+            "redirect_url":"/azure/reliability/reliability-cosmos-db-nosql.md",
+            "redirect_document_id":false
         }
     ]
 }

articles/ai-services/content-moderator/includes/tool-deprecation.md

@@ -13,7 +13,7 @@ ms.author: pafarley
 
 
 > [!IMPORTANT]
-> Azure Content Moderator is being deprecated in February 2024, and will be retired by February 2027. It is being replaced by [Azure AI Content Safety](/azure/ai-services/content-safety/overview), which offers advanced AI features and enhanced performance.
+> Azure Content Moderator is deprecated as of February 2024 and will be retired by February 2027. It is replaced by [Azure AI Content Safety](/azure/ai-services/content-safety/overview), which offers advanced AI features and enhanced performance.
 >
 > Azure AI Content Safety is a comprehensive solution designed to detect harmful user-generated and AI-generated content in applications and services. Azure AI Content Safety is suitable for many scenarios such as online marketplaces, gaming companies, social messaging platforms, enterprise media companies, and K-12 education solution providers. Here's an overview of its features and capabilities:
 > 

articles/ai-services/content-moderator/index.yml

@@ -1,7 +1,7 @@
 ### YamlMime:Landing
 
 title: Content Moderator documentation # < 60 chars
-summary: "The Azure Content Moderator API checks text, image, and video content for material that is potentially offensive, risky, or otherwise undesirable. Content Moderator is being deprecated in February 2024, and will be retired by February 2027. It is being replaced by Azure AI Content Safety, which offers advanced AI features and enhanced performance. Azure AI Content Safety is a comprehensive solution designed to detect harmful user-generated and AI-generated content in applications and services. Azure AI Content Safety is suitable for many scenarios such as online marketplaces, gaming companies, social messaging platforms, enterprise media companies, and K-12 education solution providers." # < 160 chars
+summary: "Azure Content Moderator is deprecated as of February 2024 and will be retired by February 2027. It is replaced by Azure AI Content Safety, which offers advanced AI features and enhanced performance. Azure AI Content Safety is a comprehensive solution designed to detect harmful user-generated and AI-generated content in applications and services. Azure AI Content Safety is suitable for many scenarios such as online marketplaces, gaming companies, social messaging platforms, enterprise media companies, and K-12 education solution providers." # < 160 chars
 
 metadata:
   title: Content Moderator Documentation - Quickstarts, Tutorials, API Reference - Azure AI services  | Microsoft Docs

articles/ai-services/openai/assistants-reference-runs.md

@@ -5,7 +5,7 @@ description: Learn how to use Azure OpenAI's Python & REST API runs with Assista
 manager: nitinme
 ms.service: azure-ai-openai
 ms.topic: conceptual
-ms.date: 02/01/2024
+ms.date: 04/16/2024
 author: mrbullwinkle
 ms.author: mbullwin
 recommendations: false
@@ -585,3 +585,115 @@ Represent a step in execution of a run.
 | `failed_at`| integer or null | The Unix timestamp (in seconds) for when the run step failed.|
 | `completed_at`| integer or null | The Unix timestamp (in seconds) for when the run step completed.|
 | `metadata`| map | Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format. Keys can be a maximum of 64 characters long and values can be a maximum of 512 characters long.|
+
+## Stream a run result (preview)
+
+Stream the result of executing a Run or resuming a Run after submitting tool outputs. You can stream events after:
+* [Create Thread and Run](#create-thread-and-run) 
+* [Create Run](#create-run)
+* [Submit Tool Outputs](#submit-tool-outputs-to-run) 
+
+To stream a result, pass `"stream": true` while creating a run. The response will be a [Server-Sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events) stream.
+
+### Streaming example
+
+```python
+from typing_extensions import override
+from openai import AssistantEventHandler
+ 
+# First, we create a EventHandler class to define
+# how we want to handle the events in the response stream.
+ 
+class EventHandler(AssistantEventHandler):    
+  @override
+  def on_text_created(self, text) -> None:
+    print(f"\nassistant > ", end="", flush=True)
+      
+  @override
+  def on_text_delta(self, delta, snapshot):
+    print(delta.value, end="", flush=True)
+      
+  def on_tool_call_created(self, tool_call):
+    print(f"\nassistant > {tool_call.type}\n", flush=True)
+  
+  def on_tool_call_delta(self, delta, snapshot):
+    if delta.type == 'code_interpreter':
+      if delta.code_interpreter.input:
+        print(delta.code_interpreter.input, end="", flush=True)
+      if delta.code_interpreter.outputs:
+        print(f"\n\noutput >", flush=True)
+        for output in delta.code_interpreter.outputs:
+          if output.type == "logs":
+            print(f"\n{output.logs}", flush=True)
+ 
+# Then, we use the `create_and_stream` SDK helper 
+# with the `EventHandler` class to create the Run 
+# and stream the response.
+ 
+with client.beta.threads.runs.stream(
+  thread_id=thread.id,
+  assistant_id=assistant.id,
+  instructions="Please address the user as Jane Doe. The user has a premium account.",
+  event_handler=EventHandler(),
+) as stream:
+  stream.until_done()
+```
+
+
+## Message delta object
+
+Represents a message delta. For example any changed fields on a message during streaming.
+
+|Name | Type | Description |
+|---  |---   |---         |
+| `id` | string | The identifier of the message, which can be referenced in API endpoints. |
+| `object` | string | The object type, which is always `thread.message.delta`. |
+| `delta` | object | The delta containing the fields that have changed on the Message. |
+
+## Run step delta object
+
+Represents a run step delta. For example any changed fields on a run step during streaming.
+
+|Name | Type | Description |
+|---  |---   |---         |
+| `id` | string | The identifier of the run step, which can be referenced in API endpoints. |
+| `object` | string | The object type, which is always `thread.run.step.delta`. |
+| `delta` | object | The delta containing the fields that have changed on the run step.
+
+## Assistant stream events
+
+Represents an event emitted when streaming a Run. Each event in a server-sent events stream has an event and data property:
+
+```json
+event: thread.created
+data: {"id": "thread_123", "object": "thread", ...}
+```
+
+Events are emitted whenever a new object is created, transitions to a new state, or is being streamed in parts (deltas). For example, `thread.run.created` is emitted when a new run is created, `thread.run.completed` when a run completes, and so on. When an Assistant chooses to create a message during a run, we emit a `thread.message.created` event, a `thread.message.in_progress` event, many thread.`message.delta` events, and finally a `thread.message.completed` event.
+
+|Name | Type | Description |
+|---  |---   |---         |
+| `thread.created` | `data` is a thread. | Occurs when a new thread is created. |
+| `thread.run.created` | `data` is a run. | Occurs when a new run is created. |
+| `thread.run.queued` | `data` is a run. | Occurs when a run moves to a queued status. |
+| `thread.run.in_progress` | `data` is a run. | Occurs when a run moves to an in_progress status. |
+| `thread.run.requires_action` | `data` is a run. | Occurs when a run moves to a `requires_action` status. |
+| `thread.run.completed` | `data` is a run. | Occurs when a run is completed. |
+| `thread.run.failed` | `data` is a run. | Occurs when a run fails. |
+| `thread.run.cancelling` | `data` is a run. | Occurs when a run moves to a `cancelling` status. |
+| `thread.run.cancelled` | `data` is a run. | Occurs when a run is cancelled. |
+| `thread.run.expired` | `data` is a run. | Occurs when a run expires. |
+| `thread.run.step.created` | `data` is a run step. | Occurs when a run step is created. |
+| `thread.run.step.in_progress` | `data` is a run step. | Occurs when a run step moves to an `in_progress` state. | 
+| `thread.run.step.delta` | `data` is a run step delta. | Occurs when parts of a run step are being streamed. |
+| `thread.run.step.completed` | `data` is a run step. | Occurs when a run step is completed. |
+| `thread.run.step.failed` | `data` is a run step. | Occurs when a run step fails. |
+| `thread.run.step.cancelled` | `data` is a run step. | Occurs when a run step is cancelled. |
+| `thread.run.step.expired` | `data` is a run step. | Occurs when a run step expires. |
+| `thread.message.created` | `data` is a message. | Occurs when a message is created. |
+| `thread.message.in_progress` | `data` is a message. | Occurs when a message moves to an in_progress state. | 
+| `thread.message.delta` | `data` is a message delta. | Occurs when parts of a Message are being streamed. |
+| `thread.message.completed` | `data` is a message. | Occurs when a message is completed. |
+| `thread.message.incomplete` | `data` is a message. | Occurs when a message ends before it is completed. |
+| `error` | `data` is an error. | Occurs when an error occurs. This can happen due to an internal server error or a timeout. |
+| `done` | `data` is `[DONE]` | Occurs when a stream ends. |

articles/ai-services/openai/concepts/use-your-data.md

@@ -62,59 +62,15 @@ The [Integrated Vector Database in vCore-based Azure Cosmos DB for MongoDB](/azu
 
 For some data sources such as uploading files from your local machine (preview) or data contained in a blob storage account (preview), Azure AI Search is used. When you choose the following data sources, your data is ingested into an Azure AI Search index.
 
->[!TIP]
->If you use Azure Cosmos DB (except for its vCore-based API for MongoDB), you may be eligible for the [Azure AI Advantage offer](/azure/cosmos-db/ai-advantage), which provides the equivalent of up to $6,000 in Azure Cosmos DB throughput credits.
-
-|Data source  | Description  |
+|Data ingested through Azure AI Search  | Description  |
 |---------|---------|
 | [Azure AI Search](/azure/search/search-what-is-azure-search)  | Use an existing Azure AI Search index with Azure OpenAI On Your Data.      |
-| [Azure Cosmos DB](/azure/cosmos-db/introduction)  | Azure Cosmos DB's API for Postgres and vCore-based API for MongoDB offer natively integrated vector indexing; therefore, they don't require Azure AI Search. However, its other APIs do require Azure AI Search for vector indexing. Azure Cosmos DB for NoSQL's natively integrated vector database debuts in mid-2024.     |
 |Upload files (preview)      | Upload files from your local machine to be stored in an Azure Blob Storage database, and ingested into Azure AI Search.         |
 |URL/Web address (preview)        | Web content from the URLs is stored in Azure Blob Storage.         |
 |Azure Blob Storage (preview) | Upload files from Azure Blob Storage to be ingested into an Azure AI Search index.         |
 
 :::image type="content" source="../media/use-your-data/azure-databases-and-ai-search.png" lightbox="../media/use-your-data/azure-databases-and-ai-search.png" alt-text="Diagram of vector indexing services.":::
 
-# [Vector Database in Azure Cosmos DB for MongoDB](#tab/mongo-db)
-
-### Prerequisites
-* [vCore-based Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/introduction) account
-* A deployed [embedding model](../concepts/understand-embeddings.md)
-
-### Limitations
-* Only vCore-based Azure Cosmos DB for MongoDB is supported.
-* The search type is limited to [Integrated Vector Database in Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/vector-search) with an Azure OpenAI embedding model.
-* This implementation works best on unstructured and spatial data.
-
-  
-### Data preparation
-
-Use the script provided on [GitHub](https://github.com/microsoft/sample-app-aoai-chatGPT/tree/main/scripts#data-preparation) to prepare your data.
-
-<!--### Add your data source in Azure OpenAI Studio
-
-To add vCore-based Azure Cosmos DB for MongoDB as a data source, you'll need an existing Azure Cosmos DB for MongoDB index containing your data, and a deployed Azure OpenAI Ada embeddings model that will be used for vector search.
-
-1. In the [Azure OpenAI portal](https://oai.azure.com/portal) chat playground, select **Add your data**. In the panel that appears, select ** vCore-based Azure Cosmos DB for MongoDB** as the data source. 
-1. Select your Azure subscription and database account, then connect to your Azure Cosmos DB account by providing your Azure Cosmos DB account username and password.
-    
-    :::image type="content" source="../media/use-your-data/add-mongo-data-source.png" alt-text="A screenshot showing the screen for adding Mongo DB as a data source in Azure OpenAI Studio." lightbox="../media/use-your-data/add-mongo-data-source.png":::
-
-1. **Select Database**. In the dropdown menus, select the database name, database collection, and index name that you want to use as your data source. Select the embedding model deployment you would like to use for vector search on this data source, and acknowledge that you'll incur charges for using vector search. Then select **Next**.
-
-    :::image type="content" source="../media/use-your-data/select-mongo-database.png" alt-text="A screenshot showing the screen for adding Mongo DB settings in Azure OpenAI Studio." lightbox="../media/use-your-data/select-mongo-database.png":::
--->
-
-### Index field mapping
-
-When you add your vCore-based Azure Cosmos DB for MongoDB data source, you can specify data fields to properly map your data for retrieval.
-
-* Content data (required): One or more provided fields to be used to ground the model on your data. For multiple fields, separate the values with commas, with no spaces.
-* File name/title/URL: Used to display more information when a document is referenced in the chat.
-* Vector fields (required): Select the field in your database that contains the vectors.
-
-:::image type="content" source="../media/use-your-data/mongo-index-mapping.png" alt-text="A screenshot showing the index field mapping options for Mongo DB." lightbox="../media/use-your-data/mongo-index-mapping.png":::
-
 # [Azure AI Search](#tab/ai-search)
 
 You might want to consider using an Azure AI Search index when you either want to:
@@ -179,6 +135,33 @@ If you want to implement additional value-based criteria for query execution, yo
 
 [!INCLUDE [ai-search-ingestion](../includes/ai-search-ingestion.md)]
 
+
+# [Vector Database in Azure Cosmos DB for MongoDB](#tab/mongo-db)
+
+### Prerequisites
+* [vCore-based Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/introduction) account
+* A deployed [embedding model](../concepts/understand-embeddings.md)
+
+### Limitations
+* Only vCore-based Azure Cosmos DB for MongoDB is supported.
+* The search type is limited to [Integrated Vector Database in Azure Cosmos DB for MongoDB](/azure/cosmos-db/mongodb/vcore/vector-search) with an Azure OpenAI embedding model.
+* This implementation works best on unstructured and spatial data.
+
+  
+### Data preparation
+
+Use the script provided on [GitHub](https://github.com/microsoft/sample-app-aoai-chatGPT/tree/main/scripts#data-preparation) to prepare your data.
+
+### Index field mapping
+
+When you add your vCore-based Azure Cosmos DB for MongoDB data source, you can specify data fields to properly map your data for retrieval.
+
+* Content data (required): One or more provided fields to be used to ground the model on your data. For multiple fields, separate the values with commas, with no spaces.
+* File name/title/URL: Used to display more information when a document is referenced in the chat.
+* Vector fields (required): Select the field in your database that contains the vectors.
+
+:::image type="content" source="../media/use-your-data/mongo-index-mapping.png" alt-text="A screenshot showing the index field mapping options for Mongo DB." lightbox="../media/use-your-data/mongo-index-mapping.png":::
+
 # [Azure Blob Storage (preview)](#tab/blob-storage)
 
 You might want to use Azure Blob Storage as a data source if you want to connect to existing Azure Blob Storage and use files stored in your containers.

articles/ai-services/openai/concepts/use-your-image-data.md

@@ -8,7 +8,7 @@ ms.service: azure-ai-openai
 ms.topic: quickstart
 author: aahill
 ms.author: aahi
-ms.date: 11/02/2023
+ms.date: 05/09/2024
 recommendations: false
 ---
 
@@ -17,7 +17,7 @@ recommendations: false
 Use this article to learn how to provide your own image data for GPT-4 Turbo with Vision, Azure OpenAI’s vision model. GPT-4 Turbo with Vision on your data allows the model to generate more customized and targeted answers using Retrieval Augmented Generation based on your own images and image metadata. 
 
 > [!IMPORTANT]
-> This article is for using your data on the GPT-4 Turbo with Vision model. If you are interested in using your data for text-based models, see [Use your text data](./use-your-data.md).  
+> Once the GPT4-Turbo with vision preview model is deprecated, you will no longer be able to use Azure OpenAI On your image data. To implement a Retrieval Augmented Generation (RAG) solution with image data, see the following sample on [github](https://github.com/Azure-Samples/azure-search-openai-demo/).
 
 ## Prerequisites