Merge pull request #262750 from MicrosoftDocs/main

1/9 11:00 AM IST Publish

Author: Saisang

articles/ai-services/openai/api-version-deprecation.md

@@ -0,0 +1,53 @@
+---
+title: Azure OpenAI Service API version retirement
+description: Learn more about API version retirement in Azure OpenAI Services
+services: cognitive-services
+manager: nitinme
+ms.service: azure-ai-openai
+ms.topic: conceptual 
+ms.date: 01/08/2024
+author: mrbullwinkle
+ms.author: mbullwin
+recommendations: false
+ms.custom:
+---
+
+# Azure OpenAI API preview lifecycle
+
+This article is to help you understand the support lifecycle for the Azure OpenAI API previews.
+
+## Latest preview API release
+
+Azure OpenAI API version 2023-12-01-preview is currently the latest preview release.
+
+This version contains support for all the latest Azure OpenAI features including:
+
+- [Fine-tuning](./how-to/fine-tuning.md) `gpt-35-turbo`, `babbage-002`, and `davinci-002` models.[**Added in 2023-10-01-preview**]
+- [Whisper](./whisper-quickstart.md). [**Added in 2023-09-01-preview**]
+- [Function calling](./how-to/function-calling.md)  [**Added in 2023-07-01-preview**]
+- [DALL-E](./dall-e-quickstart.md)  [**Added in 2023-06-01-preview**]
+- [Retrieval augmented generation with the on your data feature](./use-your-data-quickstart.md).  [**Added in 2023-06-01-preview**]
+
+## Retiring soon
+
+On April 2, 2024 the following API preview releases will be retired and will stop accepting API requests:
+
+- 2023-03-15-preview
+- 2023-06-01-preview
+- 2023-07-01-preview
+- 2023-08-01-preview
+
+To avoid service disruptions, you must update to use the latest preview version prior to the retirement date.
+
+## Updating API versions
+
+We recommend first testing the upgrade to new API versions to confirm there is no impact to your application from the API update prior to making the change globally across your environment.
+
+If you are using the OpenAI Python client library or the REST API, you will need to update your code directly to the latest preview API version.
+
+If you are using one of the Azure OpenAI SDKs for C#, Go, Java, or JavaScript you will instead need to update to the latest version of the SDK. Each SDK release is hardcoded to work with specific versions of the Azure OpenAI API.
+
+## Next steps
+
+- [Learn more about Azure OpenAI](overview.md)
+- [Learn about working with Azure OpenAI models](./how-to/working-with-models.md)

articles/ai-services/openai/concepts/content-filter.md

@@ -628,6 +628,174 @@ For details on the inference REST API endpoints for Azure OpenAI and how to crea
 }
 ```
 
+## Streaming
+
+Azure OpenAI Service includes a content filtering system that works alongside core models. The following section describes the AOAI streaming experience and options in the context of content filters. 
+
+### Default
+
+The content filtering system is integrated and enabled by default for all customers. In the default streaming scenario, completion content is buffered, the content filtering system runs on the buffered content, and – depending on content filtering configuration – content is either returned to the user if it does not violate the content filtering policy (Microsoft default or custom user configuration), or it’s immediately blocked which returns a content filtering error, without returning harmful completion content. This process is repeated until the end of the stream. Content was fully vetted according to the content filtering policy before returned to the user. Content is not returned token-by-token in this case, but in “content chunks” of the respective buffer size. 
+
+### Asynchronous modified filter
+
+Customers who have been approved for modified content filters can choose Asynchronous Modified Filter as an additional option, providing a new streaming experience. In this case, content filters are run asynchronously, completion content is returned immediately with a smooth token-by-token streaming experience. No content is buffered, the content filters run asynchronously, which allows for zero latency in this context.  
+
+> [!NOTE]
+> Customers must be aware that while the feature improves latency, it can bring a trade-off in terms of the safety and real-time vetting of smaller sections of model output. Because content filters are run asynchronously, content moderation messages and the content filtering signal in case of a policy violation are delayed, which means some sections of harmful content that would otherwise have been filtered immediately could be displayed to the user. 
+ 
+**Annotations**: Annotations and content moderation messages are continuously returned during the stream. We strongly recommend to consume annotations and implement additional AI content safety mechanisms, such as redacting content or returning additional safety information to the user. 
+
+**Content filtering signal**: The content filtering error signal is delayed; in case of a policy violation, it’s returned as soon as it’s available, and the stream is stopped. The content filtering signal is guaranteed within ~1,000-character windows in case of a policy violation. 
+
+Approval for Modified Content Filtering is required for access to Streaming – Asynchronous Modified Filter. The application can be found [here](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xURE01NDY1OUhBRzQ3MkQxMUhZSE1ZUlJKTiQlQCN0PWcu). To enable it via Azure OpenAI Studio please follow the instructions [here](/azure/ai-services/openai/how-to/content-filters) to create a new content filtering configuration, and select “Asynchronous Modified Filter” in the Streaming section, as shown in the below screenshot. 
+
+### Overview tbd
+
+| | Streaming - Default | Streaming - Asynchronous Modified Filter |
+|---|---|---|
+|Status |GA |Public Preview |
+| Access | Enabled by default, no action needed |Customers approved for Modified Content Filtering can configure directly via Azure OpenAI Studio (as part of a content filtering configuration; applied on deployment-level) |
+| Eligibility |All customers |Customers approved for Modified Content Filtering |
+|Modality and Availability |Text; all GPT-models |Text; all GPT-models except gpt-4-vision |
+|Streaming experience |Content is buffered and returned in chunks |Zero latency (no buffering, filters run asynchronously) |
+|Content filtering signal |Immediate filtering signal |Delayed filtering signal (in up to ~1,000 char increments) |
+|Content filtering configurations |Supports default and any customer-defined filter setting (including optional models) |Supports default and any customer-defined filter setting (including optional models) | 
+
+### Annotations and sample response stream
+
+#### Prompt annotation message
+
+This is the same as default annotations.
+
+```json
+data: { 
+    "id": "", 
+    "object": "", 
+    "created": 0, 
+    "model": "", 
+    "prompt_filter_results": [ 
+        { 
+            "prompt_index": 0, 
+            "content_filter_results": { ... } 
+        } 
+    ], 
+    "choices": [], 
+    "usage": null 
+} 
+```
+
+#### Completion token message
+
+Completion messages are forwarded immediately. No moderation is performed first, and no annotations are provided initially. 
+
+```json
+data: { 
+    "id": "chatcmpl-7rAJvsS1QQCDuZYDDdQuMJVMV3x3N", 
+    "object": "chat.completion.chunk", 
+    "created": 1692905411, 
+    "model": "gpt-35-turbo", 
+    "choices": [ 
+        { 
+            "index": 0, 
+            "finish_reason": null, 
+            "delta": { 
+                "content": "Color" 
+            } 
+        } 
+    ], 
+    "usage": null 
+} 
+```
+
+#### Annotation message
+
+The text field will always be an empty string, indicating no new tokens. Annotations will only be relevant to already-sent tokens. There may be multiple Annotation Messages referring to the same tokens.  
+
+“start_offset” and “end_offset” are low-granularity offsets in text (with 0 at beginning of prompt) which the annotation is relevant to. 
+
+“check_offset” represents how much text has been fully moderated. It is an exclusive lower bound on the end_offsets of future annotations. It is nondecreasing. 
+
+```json
+data: { 
+    "id": "", 
+    "object": "", 
+    "created": 0, 
+    "model": "", 
+    "choices": [ 
+        { 
+            "index": 0, 
+            "finish_reason": null, 
+            "content_filter_results": { ... }, 
+            "content_filter_raw": [ ... ], 
+            "content_filter_offsets": { 
+                "check_offset": 44, 
+                "start_offset": 44, 
+                "end_offset": 198 
+            } 
+        } 
+    ], 
+    "usage": null 
+} 
+```
+
+
+### Sample response stream
+
+Below is a real chat completion response using  Asynchronous Modified Filter. Note how prompt annotations are not changed; completion tokens are sent without annotations; and new annotation messages are sent without tokens, instead associated with certain content filter offsets. 
+
+`{"temperature": 0, "frequency_penalty": 0, "presence_penalty": 1.0, "top_p": 1.0, "max_tokens": 800, "messages": [{"role": "user", "content": "What is color?"}], "stream": true}`
+
+```
+data: {"id":"","object":"","created":0,"model":"","prompt_annotations":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"choices":[],"usage":null} 
+
+data: {"id":"chatcmpl-7rCNsVeZy0PGnX3H6jK8STps5nZUY","object":"chat.completion.chunk","created":1692913344,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"role":"assistant"}}],"usage":null} 
+
+data: {"id":"chatcmpl-7rCNsVeZy0PGnX3H6jK8STps5nZUY","object":"chat.completion.chunk","created":1692913344,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":"Color"}}],"usage":null} 
+
+data: {"id":"chatcmpl-7rCNsVeZy0PGnX3H6jK8STps5nZUY","object":"chat.completion.chunk","created":1692913344,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":" is"}}],"usage":null} 
+
+data: {"id":"chatcmpl-7rCNsVeZy0PGnX3H6jK8STps5nZUY","object":"chat.completion.chunk","created":1692913344,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":" a"}}],"usage":null} 
+
+... 
+
+data: {"id":"","object":"","created":0,"model":"","choices":[{"index":0,"finish_reason":null,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"content_filter_offsets":{"check_offset":44,"start_offset":44,"end_offset":198}}],"usage":null} 
+
+... 
+
+data: {"id":"chatcmpl-7rCNsVeZy0PGnX3H6jK8STps5nZUY","object":"chat.completion.chunk","created":1692913344,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":"stop","delta":{}}],"usage":null} 
+
+data: {"id":"","object":"","created":0,"model":"","choices":[{"index":0,"finish_reason":null,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"content_filter_offsets":{"check_offset":506,"start_offset":44,"end_offset":571}}],"usage":null} 
+
+data: [DONE] 
+```
+
+### Sample response stream (blocking)
+
+`{"temperature": 0, "frequency_penalty": 0, "presence_penalty": 1.0, "top_p": 1.0, "max_tokens": 800, "messages": [{"role": "user", "content": "Tell me the lyrics to \"Hey Jude\"."}], "stream": true}`
+
+```
+data: {"id":"","object":"","created":0,"model":"","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"choices":[],"usage":null} 
+
+data: {"id":"chatcmpl-8JCbt5d4luUIhYCI7YH4dQK7hnHx2","object":"chat.completion.chunk","created":1699587397,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"role":"assistant"}}],"usage":null} 
+
+data: {"id":"chatcmpl-8JCbt5d4luUIhYCI7YH4dQK7hnHx2","object":"chat.completion.chunk","created":1699587397,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":"Hey"}}],"usage":null} 
+
+data: {"id":"chatcmpl-8JCbt5d4luUIhYCI7YH4dQK7hnHx2","object":"chat.completion.chunk","created":1699587397,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":" Jude"}}],"usage":null} 
+
+data: {"id":"chatcmpl-8JCbt5d4luUIhYCI7YH4dQK7hnHx2","object":"chat.completion.chunk","created":1699587397,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":","}}],"usage":null} 
+
+... 
+
+data: {"id":"chatcmpl-8JCbt5d4luUIhYCI7YH4dQK7hnHx2","object":"chat.completion.chunk","created":1699587397,"model":"gpt-35- 
+
+turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":" better"}}],"usage":null} 
+
+data: {"id":"","object":"","created":0,"model":"","choices":[{"index":0,"finish_reason":null,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"content_filter_offsets":{"check_offset":65,"start_offset":65,"end_offset":1056}}],"usage":null} 
+
+data: {"id":"","object":"","created":0,"model":"","choices":[{"index":0,"finish_reason":"content_filter","content_filter_results":{"protected_material_text":{"detected":true,"filtered":true}},"content_filter_offsets":{"check_offset":65,"start_offset":65,"end_offset":1056}}],"usage":null} 
+
+data: [DONE] 
+```
 ## Best practices
 
 As part of your application design, consider the following best practices to deliver a positive experience with your application while minimizing potential harms:

articles/ai-services/openai/toc.yml

@@ -65,6 +65,8 @@ items:
       href: ./concepts/use-your-image-data.md
 - name: How-to 
   items:
+  - name: API version lifecycle
+    href: ./api-version-deprecation.md
   - name: Completions & chat completions
     items:
     - name: GPT-35-Turbo & GPT-4 

articles/ai-services/translator/custom-translator/how-to/create-manage-workspace.md

@@ -5,7 +5,7 @@ description: How to create and manage workspaces
 author: laujan
 manager: nitinme
 ms.service: azure-ai-translator
-ms.date: 07/18/2023
+ms.date: 01/08/2024
 ms.author: lajanuar
 ms.topic: how-to
 
@@ -15,6 +15,11 @@ ms.topic: how-to
 
  Workspaces are places to manage your documents, projects, and models. When you create a workspace, you can choose to use the workspace independently, or share it with teammates to divide up the work.
 
+  > [!NOTE]
+  >
+  > * [Custom Translator Portal](https://portal.customtranslator.azure.ai/) access can only be enabled through a public network.
+  > * For information on how to use selected networks and private endpoints, see [Enable Custom Translator through Azure Virtual Network](enable-vnet-service-endpoint.md).
+
 ## Create workspace
 
 1. After you sign in to Custom Translator, you'll be asked for permission to read your profile from the Microsoft identity platform to request your user access token and refresh token. Both tokens are needed for authentication and to ensure that you aren't signed out during your live session or while training your models. </br>Select **Yes**.

articles/aks/concepts-network.md

@@ -24,19 +24,27 @@ This article introduces the core concepts that provide networking to your applic
 * [Network policies](#network-policies)
 
 ## Kubernetes basics
+Kubernetes employs a virtual networking layer to manage access within and between your applications or their components. This involves the following key aspects:
 
-To allow access to your applications or between application components, Kubernetes provides an abstraction layer to virtual networking. Kubernetes nodes connect to a virtual network, providing inbound and outbound connectivity for pods. The *kube-proxy* component runs on each node to provide these network features.
+- **Kubernetes nodes and virtual network**: Kubernetes nodes are connected to a virtual network. This setup enables pods (basic units of deployment in Kubernetes) to have both inbound and outbound connectivity.
 
-In Kubernetes:
+- **Kube-proxy component**: Running on each node, kube-proxy is responsible for providing the necessary network features.
 
-* *Services* logically group pods to allow for direct access on a specific port via an IP address or DNS name.
-* *ServiceTypes* allow you to specify what kind of Service you want.
-* You can distribute traffic using a *load balancer*.
-* Layer 7 routing of application traffic can also be achieved with *ingress controllers*.
-* You can *control outbound (egress) traffic* for cluster nodes.
-* Security and filtering of the network traffic for pods is possible with *network policies*.
+Regarding specific Kubernetes functionalities:
 
-The Azure platform also simplifies virtual networking for AKS clusters. When you create a Kubernetes load balancer, you also create and configure the underlying Azure load balancer resource. As you open network ports to pods, the corresponding Azure network security group rules are configured. For HTTP application routing, Azure can also configure *external DNS* as new Ingress routes are configured.
+- **Services**: These are used to logically group pods, allowing direct access to them through a specific IP address or DNS name on a designated port.
+- **Service types**: This feature lets you specify the kind of Service you wish to create.
+- **Load balancer**: You can use a load balancer to distribute network traffic evenly across various resources.
+- **Ingress controllers**: These facilitate Layer 7 routing, which is essential for directing application traffic.
+- **Egress traffic control**: Kubernetes allows you to manage and control outbound traffic from cluster nodes.
+- **Network policies**: These policies enable security measures and filtering for network traffic in pods.
+
+In the context of the Azure platform:
+
+- Azure streamlines virtual networking for AKS (Azure Kubernetes Service) clusters.
+- Creating a Kubernetes load balancer on Azure simultaneously sets up the corresponding Azure load balancer resource.
+- As you open network ports to pods, Azure automatically configures the necessary network security group rules.
+- Azure can also manage external DNS configurations for HTTP application routing as new Ingress routes are established.
 
 ## Services
 

articles/aks/free-standard-pricing-tiers.md

@@ -1,12 +1,12 @@
 ---
-title: Azure Kubernetes Service (AKS) Free Standard and Premium pricing tiers for cluster management
+title: Azure Kubernetes Service (AKS) Free, Standard and Premium pricing tiers for cluster management
 description: Learn about the Azure Kubernetes Service (AKS) Free, Standard, and Premium pricing plans and what features, deployment patterns, and recommendations to consider between each plan.
 ms.topic: conceptual
 ms.date: 04/07/2023
 ms.custom: references_regions, devx-track-azurecli
 ---
 
-# Free Standard and Premium pricing tiers for Azure Kubernetes Service (AKS) cluster management
+# Free, Standard and Premium pricing tiers for Azure Kubernetes Service (AKS) cluster management
 
 Azure Kubernetes Service (AKS) is now offering three pricing tiers for cluster management: the **Free tier**, the **Standard tier** and the **Premium tier**. All tiers are in the **Base** sku.