Merge branch 'main' of https://github.com/MicrosoftDocs/azure-docs-pr into service-tags-links

Author: asudbring

articles/active-directory-b2c/custom-policies-series-store-user.md

@@ -10,7 +10,7 @@ ms.service: active-directory
 
 ms.topic: how-to
 ms.custom: b2c-docs-improvements
-ms.date: 01/11/2024
+ms.date: 05/11/2024
 ms.author: kengaderdus
 ms.reviewer: yoelh
 ms.subservice: B2C
@@ -407,15 +407,14 @@ To configure a display control, use the following steps:
 
 1. Use the procedure in [step 6](#step-6---upload-policy) and [step 7](#step-7---test-policy) to upload your policy file, and test it. This time, you must verify your email address before a user account is created.  
 
-<a name='update-user-account-by-using-azure-ad-technical-profile'></a>
 
 ## Update user account by using Microsoft Entra ID technical profile
 
-You can configure a Microsoft Entra ID technical profile to update a user account instead of attempting to create a new one. To do so, set the Microsoft Entra ID technical profile to throw an error if the specified user account doesn't already exist in the `Metadata` collection by using the following code. The *Operation* needs to be set to *Write*:
+You can configure a Microsoft Entra ID technical profile to update a user account instead of attempting to create a new one. To do so, set the Microsoft Entra ID technical profile to throw an error if the specified user account doesn't already exist in the metadata collection by using the following code. Also, remove the `Key="UserMessageIfClaimsPrincipalAlreadyExists` metadata entry. The *Operation* needs to be set to *Write*:
 
 ```xml
     <Item Key="Operation">Write</Item>
-    <Item Key="RaiseErrorIfClaimsPrincipalDoesNotExist">true</Item>
+    <Item Key="RaiseErrorIfClaimsPrincipalDoesNotExist">false</Item>
 ``` 
 
 ## Use custom attributes 

articles/ai-services/language-service/conversational-language-understanding/concepts/best-practices.md

@@ -216,3 +216,38 @@ curl --request POST \
 "targetResourceRegion": "<target-region>" 
 }'
 ```
+
+
+## Addressing out of domain utterances
+
+Customers can use the new recipe version '2024-06-01-preview' in case the model has poor AIQ on out of domain utterances. An example of this with the default recipe can be like the below where the model has 3 intents Sports, QueryWeather and Alarm. The test utterances are out of domain utterances and the model classifies them as InDomain with a relatively high confidence score.
+
+| Text |	Predicted intent |	Confidence score |
+|----|----|----|
+| "*Who built the Eiffel Tower?*" |	 `Sports` | 0.90 |
+| "*Do I look good to you today?*" | `QueryWeather` |	1.00 |
+| "*I hope you have a good evening.*" | `Alarm` | 0.80 |
+
+To address this, use the `2024-06-01-preview` configuration version that is built specifically to address this issue while also maintaining reasonably good quality on In Domain utterances.
+
+```console
+curl --location 'https://<your-resource>.cognitiveservices.azure.com/language/authoring/analyze-conversations/projects/<your-project>/:train?api-version=2022-10-01-preview' \
+--header 'Ocp-Apim-Subscription-Key: <your subscription key>' \
+--header 'Content-Type: application/json' \
+--data '{
+      "modelLabel": "<modelLabel>",
+      "trainingMode": "advanced",
+      "trainingConfigVersion": "2024-06-01-preview",
+      "evaluationOptions": {
+            "kind": "percentage",
+            "testingSplitPercentage": 0,
+            "trainingSplitPercentage": 100
+      }
+}
+```
+
+Once the request is sent, you can track the progress of the training job in Language Studio as usual.
+
+Caveats:
+- The None Score threshold for the app (confidence threshold below which the topIntent is marked as None) when using this recipe should be set to 0. This is because this new recipe attributes a certain portion of the in domain probabiliities to out of domain so that the model is not incorrectly overconfident about in domain utterances. As a result, users may see slightly reduced confidence scores for in domain utterances as compared to the prod recipe.
+- This recipe is not recommended for apps with just two (2) intents, such as IntentA and None, for example.

articles/ai-studio/how-to/model-benchmarks.md

@@ -0,0 +1,66 @@
+---
+title: Explore model benchmarks in Azure AI Studio
+titleSuffix: Azure AI Studio
+description: This article introduces benchmarking capabilities and the model benchmarks experience in Azure AI Studio.
+manager: scottpolly
+ms.service: azure-ai-studio
+ms.custom:
+ms.topic: how-to
+ms.date: 5/6/2024
+ms.reviewer: jcioffi
+ms.author: jcioffi
+author: jesscioffi
+---
+
+# Model benchmarks
+
+[!INCLUDE [Azure AI Studio preview](../includes/preview-ai-studio.md)]
+
+In Azure AI Studio, you can compare benchmarks across models and datasets available in the industry to assess which one meets your business scenario. You can find Model benchmarks under **Get started** in the left side menu in Azure AI Studio.
+
+:::image type="content" source="../media/explore/model-benchmarks-dashboard-view.png" alt-text="Screenshot of dashboard view graph of model accuracy." lightbox="../media/explore/model-benchmarks-dashboard-view.png":::
+
+Model benchmarks help you make informed decisions about the sustainability of models and datasets prior to initiating any job. The benchmarks are a curated list of the best performing models for a given task, based on a comprehensive comparison of benchmarking metrics. Currently, Azure AI Studio provides benchmarks based on quality, via the metrics listed below.
+
+| Metric       | Description |
+|--------------|-------|
+| Accuracy     |Accuracy scores are available at the dataset and the model levels. At the dataset level, the score is the average value of an accuracy metric computed over all examples in the dataset. The accuracy metric used is exact-match in all cases except for the *HumanEval* dataset that uses a `pass@1` metric. Exact match simply compares model generated text with the correct answer according to the dataset, reporting one if the generated text matches the answer exactly and zero otherwise. `Pass@1` measures the proportion of model solutions that pass a set of unit tests in a code generation task. At the model level, the accuracy score is the average of the dataset-level accuracies for each model.|
+| Coherence    |Coherence evaluates how well the language model can produce output that flows smoothly, reads naturally, and resembles human-like language.|
+| Fluency      |Fluency evaluates the language proficiency of a generative AI's predicted answer. It assesses how well the generated text adheres to grammatical rules, syntactic structures, and appropriate usage of vocabulary, resulting in linguistically correct and natural-sounding responses.|
+| GPTSimilarity|GPTSimilarity is a measure that quantifies the similarity between a ground truth sentence (or document) and the prediction sentence generated by an AI model. It is calculated by first computing sentence-level embeddings using the embeddings API for both the ground truth and the model's prediction. These embeddings represent high-dimensional vector representations of the sentences, capturing their semantic meaning and context.|
+
+The benchmarks are updated regularly as new metrics and datasets are added to existing models, and as new models are added to the model catalog.
+
+### How the scores are calculated
+
+The benchmark results originate from public datasets that are commonly used for language model evaluation. In most cases, the data is hosted in GitHub repositories maintained by the creators or curators of the data. Azure AI evaluation pipelines download data from their original sources, extract prompts from each example row, generate model responses, and then compute relevant accuracy metrics.
+
+Prompt construction follows best practice for each dataset, set forth by the paper introducing the dataset and industry standard. In most cases, each prompt contains several examples of complete questions and answers, or "shots," to prime the model for the task. The evaluation pipelines create shots by sampling questions and answers from a portion of the data that is held out from evaluation.
+
+### View options in the model benchmarks
+
+These benchmarks encompass both a dashboard view and a list view of the data for ease of comparison, and helpful information that explains what the calculated metrics mean.
+
+Dashboard view allows you to compare the scores of multiple models across datasets and tasks. You can view models side by side (horizontally along the x-axis) and compare their scores (vertically along the y-axis) for each metric.
+
+You can filter the dashboard view by task, model collection, model name, dataset, and metric.
+
+You can switch from dashboard view to list view by following these quick steps:
+1. Select the models you want to compare.
+2. Select **List** on the right side of the page.
+
+:::image type="content" source="../media/explore/model-benchmarks-dashboard-filtered.png" alt-text="Screenshot of dashboard view graph with question answering filter applied and 'List' button identified." lightbox="../media/explore/model-benchmarks-dashboard-filtered.png":::
+
+In list view you can find the following information:
+- Model name, description, version, and aggregate scores.
+- Benchmark datasets (such as AGIEval) and tasks (such as question answering) that were used to evaluate the model.
+- Model scores per dataset.
+
+You can also filter the list view by task, model collection, model name, dataset, and metric.
+
+:::image type="content" source="../media/explore/model-benchmarks-list-view.png" alt-text="Screenshot of list view table displaying accuracy metrics in an ordered list." lightbox="../media/explore/model-benchmarks-list-view.png":::
+
+## Next steps
+
+- [Explore Azure AI foundation models in Azure AI Studio](models-foundation-azure-ai.md)
+- [View and compare benchmarks in AI Studio](https://ai.azure.com/explore/benchmarks)

articles/ai-studio/media/explore/model-benchmarks-dashboard-filtered.png

@@ -41,6 +41,8 @@ items:
     items:
     - name: Model catalog
       href: how-to/model-catalog.md
+    - name: Model benchmarks
+      href: how-to/model-benchmarks.md
     - name: Cohere models
       items:
       - name: Deploy Cohere Command models

articles/ai-studio/media/explore/model-benchmarks-dashboard-view.png

@@ -33,6 +33,23 @@ The Federal Information Processing Standard (FIPS) 140-2 is a US government stan
 >
 > FIPS-enabled node images may have different version numbers, such as kernel version, than images that aren't FIPS-enabled. The update cycle for FIPS-enabled node pools and node images may differ from node pools and images that aren't FIPS-enabled.
 
+## Supported OS Versions
+You can create FIPS-enabled node pools on all supported OS types, Linux and Windows. However, not all OS versions support FIPS-enabled nodepools. After a new OS version is released, there is typically a waiting period before it is FIPS compliant.
+
+The below table includes the supported OS versions:
+
+|OS Type|OS SKU|FIPS Compliance|
+|--|--|--|
+|Linux|Ubuntu|Supported|
+|Linux|Azure Linux| Supported|
+|Windows|Windows Server 2019| Supported|
+|Windows| Windows Server 2022| Supported|
+
+When requesting FIPS enabled Ubuntu, if the default Ubuntu version does not support FIPS, AKS will default to the most recent FIPS-supported version of Ubuntu. For example, Ubuntu 22.04 is default for Linux node pools. Since 22.04 does not currently support FIPS, AKS will default to Ubuntu 20.04 for Linux FIPS-enabled nodepools.
+
+> [!NOTE]
+ > Previously, you could use the GetOSOptions API to determine whether a given OS supported FIPS. The GetOSOptions API is now deprecated and it will no longer be included in new AKS API versions starting with 2024-05-01. 
+
 ## Create a FIPS-enabled Linux node pool
 
 1. Create a FIPS-enabled Linux node pool using the [`az aks nodepool add`][az-aks-nodepool-add] command with the `--enable-fips-image` parameter.

articles/ai-studio/media/explore/model-benchmarks-list-view.png

@@ -0,0 +1,23 @@
+---
+title: 'Deploy an application that uses OpenAI on Azure App Service'
+description: Get started with OpenAI on Azure App Service
+author: jefmarti
+ms.author: jefmarti
+ms.date: 04/10/2024
+ms.topic: article
+zone_pivot_groups: app-service-openai
+---
+
+# Deploy an application that uses OpenAI on Azure App Service
+
+::: zone pivot="openai-dotnet"
+[!INCLUDE [deploy-intelligent-apps-linux-dotnet-pivot.md](includes/deploy-intelligent-apps/deploy-intelligent-apps-linux-dotnet-pivot.md)]
+::: zone-end  
+
+::: zone pivot="openai-python"
+[!INCLUDE [deploy-intelligent-apps-linux-python-pivot.md](includes/deploy-intelligent-apps/deploy-intelligent-apps-linux-python-pivot.md)]
+::: zone-end
+
+:::zone pivot="openai-java"
+[!INCLUDE [deploy-intelligent-apps-linux-java-pivot.md](includes/deploy-intelligent-apps/deploy-intelligent-apps-linux-java-pivot.md)]
+::: zone-end