Merge pull request #6170 from MicrosoftDocs/main

Auto Publish – main to live - 2025-07-22 22:09 UTC

Author: learn-build-service-prod[bot]

articles/ai-foundry/concepts/evaluation-evaluators/agent-evaluators.md

@@ -47,7 +47,7 @@ load_dotenv()
 
 model_config = AzureOpenAIModelConfiguration(
     azure_endpoint=os.environ["AZURE_ENDPOINT"],
-    api_key=os.environ.get["AZURE_API_KEY"],
+    api_key=os.environ.get("AZURE_API_KEY"),
     azure_deployment=os.environ.get("AZURE_DEPLOYMENT_NAME"),
     api_version=os.environ.get("AZURE_API_VERSION"),
 )
@@ -83,7 +83,7 @@ intent_resolution(
 
 ### Intent resolution output
 
-The numerical score on a Likert scale (integer 1 to 5) and a higher score is better. Given a numerical threshold (default to 3), we also output "pass" if the score >= threshold, or "fail" otherwise. Using the reason and additional fields can help you understand why the score is high or low.
+The numerical score is on a Likert scale (integer 1 to 5) and a higher score is better. Given a numerical threshold (default to 3), we also output "pass" if the score >= threshold, or "fail" otherwise. Using the reason and additional fields can help you understand why the score is high or low.
 
 ```python
 {
@@ -104,14 +104,17 @@ The numerical score on a Likert scale (integer 1 to 5) and a higher score is bet
 
 ```
 
-If you're building agents outside of Azure AI Agent Serice, this evaluator accepts a schema typical for agent messages. To learn more, see our sample notebook for [Intent Resolution](https://aka.ms/intentresolution-sample).
+If you're building agents outside of Azure AI Agent Service, this evaluator accepts a schema typical for agent messages. To learn more, see our sample notebook for [Intent Resolution](https://aka.ms/intentresolution-sample).
 
 ## Tool call accuracy
 
-`ToolCallAccuracyEvaluator` measures an agent's ability to select appropriate tools, extract, and process correct parameters from previous steps of the agentic workflow. It detects whether each tool call made is accurate (binary) and reports back the average scores, which can be interpreted as a passing rate across tool calls made.
+`ToolCallAccuracyEvaluator` measures the accuracy and efficiency of tool calls made by an agent in a run. It provides a 1-5 score based on: 
+- the relevance and helpfulness of the tool invoked;
+- the correctness of parameters used in tool calls;
+- the counts of missing or excessive calls.
 
 > [!NOTE]
-> `ToolCallAccuracyEvaluator` only supports Azure AI Agent's Function Tool evaluation, but doesn't support Built-in Tool evaluation. The agent messages must have at least one Function Tool actually called to be evaluated.    
+> `ToolCallAccuracyEvaluator` only supports Azure AI Agent's Function Tool evaluation, but doesn't support Built-in Tool evaluation. The agent run must have at least one Function Tool call and no Built-in Tool calls made to be evaluated.
 
 ### Tool call accuracy example
 
@@ -150,20 +153,35 @@ tool_call_accuracy(
 
 ### Tool call accuracy output
 
-The numerical score (passing rate of correct tool calls) is 0-1 and a higher score is better. Given a numerical threshold (default to 3), we also output "pass" if the score >= threshold, or "fail" otherwise. Using the reason and tool call detail fields can help you understand why the score is high or low.
+The numerical score is on a Likert scale (integer 1 to 5) and a higher score is better. Given a numerical threshold (default to 3), we also output "pass" if the score >= threshold, or "fail" otherwise. Using the reason and tool call detail fields can help you understand why the score is high or low.
 
 ```python
 {
-    "tool_call_accuracy": 1.0,
+    "tool_call_accuracy": 5,
     "tool_call_accuracy_result": "pass",
-    "tool_call_accuracy_threshold": 0.8,
-    "per_tool_call_details": [
-        {
-            "tool_call_accurate": True,
-            "tool_call_accurate_reason": "The input Data should get a Score of 1 because the TOOL CALL is directly relevant to the user's question about the weather in Seattle, includes appropriate parameters that match the TOOL DEFINITION, and the parameter values are correct and relevant to the user's query.",
-            "tool_call_id": "call_CUdbkBfvVBla2YP3p24uhElJ"
+    "tool_call_accuracy_threshold": 3,
+    "details": {
+        "tool_calls_made_by_agent": 1,
+        "correct_tool_calls_made_by_agent": 1,
+        "per_tool_call_details": [
+            {
+                "tool_name": "fetch_weather",
+                "total_calls_required": 1,
+                "correct_calls_made_by_agent": 1,
+                "correct_tool_percentage": 1.0,
+                "tool_call_errors": 0,
+                "tool_success_result": "pass"
+            }
+        ],
+        "excess_tool_calls": {
+            "total": 0,
+            "details": []
+        },
+        "missing_tool_calls": {
+            "total": 0,
+            "details": []
         }
-    ]
+    }
 }
 ```
 
@@ -187,7 +205,7 @@ task_adherence(
 
 ### Task adherence output
 
-The numerical score on a Likert scale (integer 1 to 5) and a higher score is better. Given a numerical threshold (default to 3), we also output "pass" if the score >= threshold, or "fail" otherwise. Using the reason field can help you understand why the score is high or low.
+The numerical score is on a Likert scale (integer 1 to 5) and a higher score is better. Given a numerical threshold (default to 3), we also output "pass" if the score >= threshold, or "fail" otherwise. Using the reason field can help you understand why the score is high or low.
 
 ```python
 {

articles/ai-foundry/concepts/resource-types.md

@@ -7,7 +7,7 @@ ms.reviewer: deeikele
 manager: scottpolly
 author: sgilley
 ms.author: sgilley
-ms.date: 05/18/2025
+ms.date: 07/22/2025
 ms.service: azure-ai-foundry
 ms.topic: concept-article
 ms.custom:
@@ -19,15 +19,15 @@ ms.custom:
 
 An Azure resource is required to use and manage services in Azure. It defines the scope for configuring, securing, and monitoring the tools or capabilities you want to use—like AI models, agents, or storage.
 
-AI Foundry Portal and SDK clients support multiple distinct Azure resource types, each designed to serve different development and operational needs. This article explains which use case requires which type.
+Azure AI Foundry portal and SDK clients support multiple distinct Azure resource types, each designed to serve different development and operational needs. This article explains which use case requires which type.
 
 ## Resource Types supported with AI Foundry
 
-* **Azure AI Foundry** – The primary resource type for designing, deploying, and managing generative AI applications and agents. It provides access to agent service, models that are hosted using a serverless hosting model, evaluations, and Azure OpenAI service. This is the recommended resource type for most applications built in Azure AI Foundry. 
+* **Azure AI Foundry** – The primary resource type for designing, deploying, and managing generative AI applications and agents. It provides access to agent service, models that are hosted using a serverless hosting model, evaluations, and Azure OpenAI service. Azure AI Foundry is the recommended resource type for most applications built in Azure AI Foundry. 
 
   Get started by [creating a first AI Foundry resource](../../ai-services/multi-service-resource.md?context=/azure/ai-foundry/context/context).
 
-* **Azure AI Hub** – Use this resource type in combination with Azure AI Foundry to additionally access open-source model hosting and fine-tuning capabilities, as well as Azure Machine Learning capabilities. When you create an AI Hub, an Azure AI Foundry resource is automatically provisioned. Hub resources can be used in both AI Foundry Portal and Machine Learning Studio.
+* **Azure AI Hub** – Use this resource type in combination with Azure AI Foundry to additionally access open-source model hosting and fine-tuning capabilities, as well as Azure Machine Learning capabilities. When you create an AI Hub, an Azure AI Foundry resource is automatically provisioned. Hub resources can be used in both Azure AI Foundry portal and Machine Learning Studio.
 
 * **Azure AI Search** – A resource used to index and retrieve data for grounding AI applications. It can be [connected](../how-to/connections-add.md) to Azure AI Foundry agents to enable retrieval-augmented generation (RAG) and semantic search experiences.
 

articles/ai-foundry/how-to/create-resource-template.md

@@ -6,7 +6,7 @@ ms.author: sgilley
 author: sdgilley
 manager: scottpolly
 reviewer: deeikele
-ms.date: 05/18/2025
+ms.date: 07/22/2025
 ms.service: azure-ai-foundry
 ms.topic: quickstart-bicep
 ms.custom:
@@ -18,8 +18,6 @@ ms.custom:
 
 # Quickstart: Create an Azure AI Foundry resource using a Bicep file
 
-[!INCLUDE [hub-only-alt](../includes/uses-hub-only-alt.md)]
-
 Use a [Microsoft Bicep](/azure/azure-resource-manager/bicep/overview) file (template) to create an [Azure AI Foundry](https://ai.azure.com/?cid=learnDocs) resource. A template makes it easy to create resources as a single, coordinated operation. A Bicep file is a text document that defines the resources that are needed for a deployment. It might also specify deployment parameters. Parameters are used to provide input values when using the file to deploy resources.
 
 ## Prerequisites
@@ -64,14 +62,14 @@ Deploy the Bicep file using either the Azure CLI or Azure PowerShell.
 
 ```azurecli
 az group create --name exampleRG --location eastus
-az deployment group create --resource-group exampleRG --template-file main.bicep --parameters aiServicesName=myai aiProjectName=myai-proj 
+az deployment group create --resource-group exampleRG --template-file main.bicep --parameters aiFoundryName=myai aiProjectName=myai-proj 
 ```
 
 # [Azure PowerShell](#tab/powershell)
 
 ```azurepowershell
 New-AzResourceGroup -Name exampleRG -Location eastus
-New-AzResourceGroupDeployment -ResourceGroupName exampleRG -TemplateFile main.bicep -aiHubName myai -aiProjectName myai-proj
+New-AzResourceGroupDeployment -ResourceGroupName exampleRG -TemplateFile main.bicep -aiFoundryName myai -aiProjectName myai-proj
 ```
 
 ---