Merge branch 'main' into release-preview-eval-redteaming

v-alje · v-alje · commit 6c7866d2c6a5 · 2025-04-01T17:03:42.000Z
diff --git a/.github/policies/disallow-edits.yml b/.github/policies/disallow-edits.yml
@@ -19,14 +19,18 @@ configuration:
                 @${issueAuthor} - You tried to add an index file to this repository; this is not permitted so your pull request will be closed automatically.
           - closePullRequest
 
-      - description: Close PRs to the "ai-services/personalizer" folder where the author isn't a member of the MicrosoftDocs org (i.e. PRs in public repo).
+      - description: Close PRs to the "ai-services/personalizer" and "ai-services/responsible-ai" folders where the author isn't a member of the MicrosoftDocs org (i.e. PRs in public repo).
         if:
           - payloadType: Pull_Request
           - isAction:
               action: Opened
-          - filesMatchPattern:
-              matchAny: true
-              pattern: articles/ai-services/personalizer/*
+          - or:
+            - filesMatchPattern:
+                matchAny: true
+                pattern: articles/ai-services/personalizer/*
+            - filesMatchPattern:
+                matchAny: true
+                pattern: articles/ai-services/responsible-ai/*
           - not:
               activitySenderHasAssociation:
                 association: Member
@@ -65,3 +69,55 @@ configuration:
                 - mrbullwinkle
               replyTemplate: ${mentionees} - Please review this PR and sign off when you're ready to merge it.
               assignMentionees: True # This part probably won't work since the bot doesn't have write perms.
+          - addLabel:
+              label: needs-human-review
+
+      - description: \@mention specific people when a PR is opened in the "ai-services/responsible-ai" folder.
+        if:
+          - payloadType: Pull_Request
+          - isAction:
+              action: Opened
+          - filesMatchPattern:
+              matchAny: true
+              pattern: articles/ai-services/responsible-ai/*
+          - activitySenderHasAssociation:
+              association: Member
+          - not:
+              or:
+                - isActivitySender:
+                    user: eric-urban
+                - isActivitySender:
+                    user: nitinme
+                - isActivitySender:
+                    user: mrbullwinkle
+                - isActivitySender:
+                    user: aahill
+                - isActivitySender:
+                    user: laujan
+                - isActivitySender:
+                    user: patrickfarley
+                - isActivitySender:
+                    user: jboback
+                - isActivitySender:
+                    user: heidisteen
+                - isActivitySender:
+                    user: haileytap
+        then:
+          - addReply:
+              reply: >-
+                @${issueAuthor} - Please don't sign off on this PR. The area owners will sign off once they've reviewed your contribution.
+          - mentionUsers:
+              mentionees:
+                - eric-urban
+                - nitinme
+                - mrbullwinkle
+                - aahill
+                - laujan
+                - patrickfarley
+                - jboback
+                - heidisteen
+                - haileytap
+              replyTemplate: ${mentionees} - Please review this PR and sign off when you're ready to merge it.
+              assignMentionees: True # This part probably won't work since the bot doesn't have write perms.
+          - addLabel:
+              label: needs-human-review
diff --git a/articles/ai-foundry/concepts/trace.md b/articles/ai-foundry/concepts/trace.md
@@ -7,7 +7,7 @@ ms.service: azure-ai-foundry
 ms.custom:
   - ignite-2024
 ms.topic: conceptual
-ms.date: 03/12/2025
+ms.date: 03/31/2025
 ms.reviewer: truptiparkar
 ms.author: lagayhar
 author: lgayhardt
diff --git a/articles/ai-foundry/how-to/develop/trace-local-sdk.md b/articles/ai-foundry/how-to/develop/trace-local-sdk.md
@@ -8,7 +8,7 @@ ms.custom:
   - build-2024
   - ignite-2024
 ms.topic: how-to
-ms.date: 03/12/2025
+ms.date: 03/31/2025
 ms.reviewer: truptiparkar
 ms.author: lagayhar
 author: lgayhardt
diff --git a/articles/ai-foundry/how-to/evaluate-results.md b/articles/ai-foundry/how-to/evaluate-results.md
@@ -9,7 +9,7 @@ ms.custom:
   - build-2024
   - ignite-2024
 ms.topic: how-to
-ms.date: 12/18/2024
+ms.date: 03/31/2025
 ms.reviewer: wenxwei
 ms.author: lagayhar
 author: lgayhardt
diff --git a/articles/ai-foundry/how-to/flow-develop-evaluation.md b/articles/ai-foundry/how-to/flow-develop-evaluation.md
@@ -8,7 +8,7 @@ ms.custom:
   - ignite-2023
   - build-2024
 ms.topic: how-to
-ms.date: 5/21/2024
+ms.date: 3/31/2025
 ms.reviewer: mithigpe
 ms.author: lagayhar
 author: lgayhardt
@@ -31,12 +31,12 @@ In prompt flow, you can customize or create your own evaluation flow tailored to
 There are two ways to develop your own evaluation methods:
 
 - **Customize a Built-in Evaluation Flow:** Modify a built-in evaluation flow. 
-  1. Under *Tools* select **Prompt flow**.
+  1. Under *Build and customize* select **Prompt flow**.
   2. Select **Create** to open the flow creation wizard.
   3. In the flow gallery under *Explore gallery* select **Evaluation flow** to filter by that type. Pick a sample and select **Clone** to do customization.
 
 - **Create a New Evaluation Flow from Scratch:** Develop a brand-new evaluation method from the ground up. 
-  1. Under *Tools* select **Prompt flow**.
+  1. Under *Build and customize* select **Prompt flow**.
   2. Select **Create** to open the flow creation wizard.
   3. In the flow gallery under *Create by type* in the "Evaluation flow" box select **Create** then you can see a template of evaluation flow.
 
diff --git a/articles/ai-foundry/how-to/flow-tune-prompts-using-variants.md b/articles/ai-foundry/how-to/flow-tune-prompts-using-variants.md
@@ -8,7 +8,7 @@ ms.custom:
   - ignite-2023
   - build-2024
 ms.topic: how-to
-ms.date: 5/21/2024
+ms.date: 3/31/2025
 ms.reviewer: none
 ms.author: lagayhar
 author: lgayhardt
@@ -53,7 +53,7 @@ Benefits of using variants include:
 In this article, we use **Web Classification** sample flow as example.
 
 1. Open the sample flow and remove the **prepare_examples** node as a start.
-	  1. Under *Tools* select **Prompt flow**.
+	  1. Under *Build and customize* select **Prompt flow**.
 	  2. Select **Create** to open the flow creation wizard.
 	  3. In the flow gallery under *Explore gallery* in the "Web Classification" box select **Clone**.
 	  4. In the flow tab, delete the **prepare_examples** node.
@@ -67,7 +67,7 @@ The classification will be based on the url, the webpage text content summary, o
 For a given URL : {{url}}, and text content: {{text_content}}.
 Classify above url to complete the category and indicate evidence.
 
-The output shoule be in this format: {"category": "App", "evidence": "Both"} 
+The output should be in this format: {"category": "App", "evidence": "Both"} 
 OUTPUT:
 ```
 
@@ -147,7 +147,7 @@ When you run the variants with a few single pieces of data and check the results
 You can submit a batch run, which allows you test the variants with a large amount of data and evaluate them with metrics, to help you find the best fit.
 
 1. First you need to prepare a dataset, which is representative enough of the real-world problem you want to solve with Prompt flow. In this example, it's a list of URLs and their classification ground truth. We use accuracy to evaluate the performance of variants.
-2. Select **Evaluate** on the top right of the page.
+2. Select **Evaluate** on the top right of the page then select **Custom Evaluation**.
 3. A wizard for **Batch run & Evaluate** occurs. The first step is to select a node to run all its variants.
   
     To test how different variants work for each node in a flow, you need to run a batch run for each node with variants one by one. This helps you avoid the influence of other nodes' variants and focus on the results of this node's variants. This follows the rule of the controlled experiment, which means that you only change one thing at a time and keep everything else the same.
diff --git a/articles/ai-foundry/model-inference/quotas-limits.md b/articles/ai-foundry/model-inference/quotas-limits.md
@@ -32,11 +32,11 @@ Azure uses quotas and limits to prevent budget overruns due to fraud, and to hon
 | -------------------- | ------------------- | ----------- |
 | Tokens per minute    | Azure OpenAI models | Varies per model and SKU. See [limits for Azure OpenAI](../../ai-services/openai/quotas-limits.md). |
 | Requests per minute  | Azure OpenAI models | Varies per model and SKU. See [limits for Azure OpenAI](../../ai-services/openai/quotas-limits.md). |
-| Tokens per minute    | DeepSeek-R1         | 5.000.000 |
-| Requests per minute  | DeepSeek-R1         | 5.000     |
+| Tokens per minute    | DeepSeek-R1         | 5,000,000 |
+| Requests per minute  | DeepSeek-R1         | 5,000     |
 | Concurrent requests  | DeepSeek-R1         | 300       |
-| Tokens per minute    | Rest of models      | 200.000   |
-| Requests per minute  | Rest of models      | 1.000     |
+| Tokens per minute    | Rest of models      | 400,000   |
+| Requests per minute  | Rest of models      | 1,000     |
 | Concurrent requests  | Rest of models      | 300       |
 
 You can [request increases to the default limits](#request-increases-to-the-default-limits). Due to high demand, limit increase requests can be submitted and evaluated per request.
diff --git a/articles/ai-services/openai/how-to/stored-completions.md b/articles/ai-services/openai/how-to/stored-completions.md
@@ -115,7 +115,7 @@ curl $AZURE_OPENAI_ENDPOINT/openai/deployments/gpt-4o/chat/completions?api-versi
   -H "Authorization: Bearer $AZURE_OPENAI_AUTH_TOKEN" \
   -d '{
     "model": "gpt-4o",
-    "store": True,
+    "store": true,
     "messages": [
       {
         "role": "system",
@@ -137,7 +137,7 @@ curl $AZURE_OPENAI_ENDPOINT/openai/deployments/gpt-4o/chat/completions?api-versi
   -H "api-key: $AZURE_OPENAI_API_KEY" \
   -d '{
     "model": "gpt-4o",
-    "store": True,
+    "store": true,
     "messages": [
       {
         "role": "system",
diff --git a/articles/machine-learning/concept-responsible-ai-scorecard.md b/articles/machine-learning/concept-responsible-ai-scorecard.md
@@ -1,35 +1,34 @@
 ---
 title: Share Responsible AI insights and make data-driven decisions with Azure Machine Learning Responsible AI scorecard
 titleSuffix: Azure Machine Learning
-description: Learn about how to use the Responsible AI scorecard to share responsible AI insights from your machine learning models and make data-driven decisions with non-technical and technical stakeholders.
+description: Learn about how to use the Responsible AI scorecard to share responsible AI insights from your machine learning models and make data-driven decisions with nontechnical and technical stakeholders.
 services: machine-learning
 ms.service: azure-machine-learning
 ms.subservice: responsible-ai
 ms.topic:  conceptual
 ms.author: lagayhar
 author: lgayhardt
 ms.reviewer: mesameki
-ms.date: 02/27/2024
+ms.date: 03/31/2025
 ms.custom: responsible-ml, build-2023, build-2023-dataai
 ---
 
 # Share Responsible AI insights using the Responsible AI scorecard (preview)
 
 Our Responsible AI dashboard is designed for machine learning professionals and data scientists to explore and evaluate model insights and inform their data-driven decisions. While it can help you implement Responsible AI practically in your machine learning lifecycle, there are some needs left unaddressed:
 
-- There often exists a gap between the technical Responsible AI tools (designed for machine-learning professionals) and the ethical, regulatory, and business requirements that define the production environment.
-- While an end-to-end machine learning life cycle includes both technical and non-technical stakeholders in the loop, there's little support to enable an effective multi-stakeholder alignment, helping technical experts get timely feedback and direction from the non-technical stakeholders.
-- AI regulations make it essential to be able to share model and data insights with auditors and risk officers for auditability purposes.
+- The gap between the technical Responsible AI tools (designed for machine learning professionals) and the ethical, regulatory, and business requirements that define the production environment.
+- The need for effective multi-stakeholder alignment in an end-to-end machine learning lifecycle, ensuring technical experts receive timely feedback and direction from nontechnical stakeholders.
+- The ability to share model and data insights with auditors and risk officers for auditability purposes, as required by AI regulations.
 
-One of the biggest benefits of using the Azure Machine Learning ecosystem is related to the archival of model and data insights in the Azure Machine Learning Run History (for quick reference in future). As a part of that infrastructure and to accompany machine learning models and their corresponding Responsible AI dashboards, we introduce the Responsible AI scorecard to empower ML professionals to generate and share their data and model health records easily.
+One of the biggest benefits of using the Azure Machine Learning ecosystem is the ability to archive model and data insights in the Azure Machine Learning Run History for quick reference in the future. As part of this infrastructure, and to complement machine learning models and their corresponding Responsible AI dashboards, we introduce the Responsible AI scorecard. This scorecard empowers machine learning professionals to easily generate and share their data and model health records.
 
 [!INCLUDE [machine-learning-preview-generic-disclaimer](includes/machine-learning-preview-generic-disclaimer.md)]
 
 ## Who should use a Responsible AI scorecard?
 
-- If you're a data scientist or a machine learning professional, after training your model and generating its corresponding Responsible AI dashboard(s) for assessment and decision-making purposes, you can extract those learnings via our PDF scorecard and share the report easily with your technical and non-technical stakeholders to build trust and gain their approval for deployment.  
-
-- If you're a product manager, business leader, or an accountable stakeholder on an AI product, you can pass your desired model performance and fairness target values such as your target accuracy, target error rate, etc., to your data science team, asking them to generate this scorecard with respect to your identified target values and whether your model meets them. That can provide guidance into whether the model should be deployed or further improved.
+- **Data scientists and machine learning professionals**: After training your model and generating its corresponding Responsible AI dashboard for assessment and decision-making purposes, you can extract those learnings via our PDF scorecard. This allows you to easily share the report with your technical and nontechnical stakeholders, building trust and gaining their approval for deployment.
+- **Product managers, business leaders, and accountable stakeholders on an AI product**: You can provide your desired model performance and fairness target values, such as target accuracy and target error rate, to your data science team. They can then generate the scorecard based on these target values to determine whether the model meets them. This helps guide decisions on whether the model should be deployed or further improved.
 
 ## Next steps
 
diff --git a/articles/machine-learning/concept-secure-code-best-practice.md b/articles/machine-learning/concept-secure-code-best-practice.md
@@ -9,15 +9,15 @@ ms.topic: conceptual
 ms.author: larryfr
 author:  Blackmist
 ms.reviewer: deeikele
-ms.date: 04/02/2024
+ms.date: 04/01/2025
 ---
 
 # Best practices for secure code
 
 In Azure Machine Learning, you can upload files and content from any source into Azure. Content within Jupyter notebooks or scripts that you load can potentially read data from your sessions, access sensitive data within your organization in Azure, or run malicious processes on your behalf.
 
 > [!IMPORTANT]
-> Only run notebooks or scripts from trusted sources. For example, where you or your security team have reviewed the notebook or script.
+> Only run notebooks or scripts from trusted sources. For example, where you or your security team reviewed the notebook or script.
 
 ## Potential threats
 
diff --git a/articles/machine-learning/how-to-troubleshoot-managed-network.md b/articles/machine-learning/how-to-troubleshoot-managed-network.md
@@ -9,7 +9,7 @@ ms.reviewer: None
 ms.author: larryfr
 author: Blackmist
 ms.topic: troubleshooting
-ms.date: 04/30/2024
+ms.date: 04/01/2025
 ms.custom: build-2023
 ---
 
@@ -29,11 +29,11 @@ To use an Azure Virtual Network when creating a workspace through the Azure port
 1. In the __Workspace Outbound access__ section, select __Use my own virtual network__.
 1. Continue to create the workspace as normal.
 
-## Does not have authorization to perform action 'Microsoft.MachineLearningServices<br/> /workspaces/privateEndpointConnections/read'
+## Doesn't have authorization to perform action 'Microsoft.MachineLearningServices<br/> /workspaces/privateEndpointConnections/read'
 
 When you create a managed virtual network, the operation can fail with an error similar to the following text:
 
-"The client '\<GUID\>' with object id '\<GUID\>' doesn't have authorization to perform action 'Microsoft.MachineLearningServices/workspaces/privateEndpointConnections/read' over scope '/subscriptions/\<GUID\>/resourceGroups/\<resource-group-name\>/providers/Microsoft.MachineLearningServices/workspaces/\<workspace-name\>' or the scope is invalid."
+"The client '\<GUID\>' with object ID '\<GUID\>' doesn't have authorization to perform action 'Microsoft.MachineLearningServices/workspaces/privateEndpointConnections/read' over scope '/subscriptions/\<GUID\>/resourceGroups/\<resource-group-name\>/providers/Microsoft.MachineLearningServices/workspaces/\<workspace-name\>' or the scope is invalid."
 
 This error occurs when the Azure identity used to create the managed virtual network doesn't have the following Azure role-based access control permissions:
 
diff --git a/articles/machine-learning/v1/how-to-debug-pipelines.md b/articles/machine-learning/v1/how-to-debug-pipelines.md
@@ -5,9 +5,9 @@ description: How to troubleshoot when you get errors running a machine learning
 services: machine-learning
 ms.service: azure-machine-learning
 ms.subservice: mlops
-ms.author: zhanxia
-author: xiaoharper
-ms.date: 11/04/2022
+ms.author: lagayhar
+author: lgayhardt
+ms.date: 03/31/2025
 ms.topic: troubleshooting
 ms.custom: UpdateFrequency5, troubleshooting, sdkv1
 #Customer intent: As a data scientist, I want to figure out why my pipeline doesn't run so that I can fix it.
diff --git a/articles/search/search-security-rbac.md b/articles/search/search-security-rbac.md
@@ -8,7 +8,7 @@ author: HeidiSteen
 ms.author: heidist
 ms.service: azure-ai-search
 ms.topic: how-to
-ms.date: 01/27/2025
+ms.date: 03/31/2025
 ms.custom: subject-rbac-steps, devx-track-azurepowershell
 ---
 
@@ -90,7 +90,7 @@ Combine these roles to get sufficient permissions for your use case.
 |List all objects on the resource |❌|❌|✅|✅|✅|
 |Access quotas and service statistics |❌|❌|✅|✅|❌|
 |Read/query an index |✅|✅|❌|❌|❌|
-|Upload data for indexing |❌|✅|❌|✅|❌|
+|Upload data for indexing <sup>1</sup>|❌|✅|❌|❌|❌|
 |Create or edit indexes/aliases |❌|❌|✅|✅|❌|
 |Create, edit and run indexers/data sources/skillsets |❌|❌|✅|✅|❌|
 |Create or edit synonym maps |❌|❌|✅|✅|❌|
@@ -103,6 +103,8 @@ Combine these roles to get sufficient permissions for your use case.
 |Configure private connections |❌|❌|✅|✅|❌|
 |Configure network security |❌|❌|✅|✅|❌|
 
+<sup>1</sup> In the Azure portal, an Owner or Contributor can run the Import data wizards that create and load indexes, even though they can't upload documents in other clients. Data connections in the wizard are made by the search service itself and not individual users. The wizards have the `Microsoft.Search/searchServices/indexes/documents/*` permission necessary for completing this task.
+
 Owners and Contributors grant the same permissions, except that only Owners can assign roles.
 
 <!-- Owners and Contributors can create, read, update, and delete objects in the Azure portal *if API keys are enabled*. the Azure portal uses keys on internal calls to data plane APIs. In you subsequently configure Azure AI Search to use "roles only", then Owner and Contributor won't be able to manage objects in the Azure portal using just those role assignments. The solution is to assign more roles, such as Search Index Data Reader, Search Index Data Contributor, and Search Service Contributor. -->
