Skip to content

Commit f85abdf

Browse files
authored
Merge pull request #3944 from MicrosoftDocs/main
04/04/2025 PM Publishing
2 parents 6c10af8 + eb73709 commit f85abdf

File tree

90 files changed

+322
-3044
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

90 files changed

+322
-3044
lines changed

articles/ai-foundry/.openpublishing.redirection.ai-studio.json

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,25 @@
11
{
22
"redirections": [
3+
{
4+
"source_path_from_root": "/articles/ai-foundry/azure-openai-in-ai-foundry.md",
5+
"redirect_url": "/azure/ai-foundry/azure-openai-in-azure-ai-foundry",
6+
"redirect_document_id": false
7+
},
8+
{
9+
"source_path_from_root": "/articles/ai-foundry/what-is-ai-foundry.md",
10+
"redirect_url": "/azure/ai-foundry/what-is-azure-ai-foundry",
11+
"redirect_document_id": false
12+
},
13+
{
14+
"source_path_from_root": "/articles/ai-foundry/whats-new-ai-foundry.md",
15+
"redirect_url": "/azure/ai-foundry/whats-new-azure-ai-foundry",
16+
"redirect_document_id": false
17+
},
18+
{
19+
"source_path_from_root": "/articles/ai-foundry/concepts/rbac-ai-foundry.md",
20+
"redirect_url": "/azure/ai-foundry/concepts/rbac-azure-ai-foundry",
21+
"redirect_document_id": false
22+
},
323
{
424
"source_path_from_root": "/articles/ai-studio/how-to/access-on-premises-resources.md",
525
"redirect_url": "/azure/ai-foundry/how-to/access-on-premises-resources",
@@ -644,7 +664,7 @@
644664
},
645665
{
646666
"source_path_from_root": "/articles/ai-studio/concepts/rbac-ai-studio.md",
647-
"redirect_url": "/azure/ai-foundry/concepts/rbac-ai-foundry",
667+
"redirect_url": "/azure/ai-foundry/concepts/rbac-azure-ai-foundry",
648668
"redirect_document_id": true
649669
},
650670
{
@@ -669,7 +689,7 @@
669689
},
670690
{
671691
"source_path_from_root": "/articles/ai-studio/azure-openai-in-ai-studio.md",
672-
"redirect_url": "/azure/ai-foundry/azure-openai-in-ai-foundry",
692+
"redirect_url": "/azure/ai-foundry/azure-openai-in-azure-ai-foundry",
673693
"redirect_document_id": true
674694
},
675695
{
@@ -739,12 +759,12 @@
739759
},
740760
{
741761
"source_path_from_root": "/articles/ai-studio/what-is-ai-studio.md",
742-
"redirect_url": "/azure/ai-foundry/what-is-ai-foundry",
762+
"redirect_url": "/azure/ai-foundry/what-is-azure-ai-foundry",
743763
"redirect_document_id": true
744764
},
745765
{
746766
"source_path_from_root": "/articles/ai-studio/whats-new-ai-studio.md",
747-
"redirect_url": "/azure/ai-foundry/whats-new-ai-foundry",
767+
"redirect_url": "/azure/ai-foundry/whats-new-azure-ai-foundry",
748768
"redirect_document_id": true
749769
},
750770
{

articles/ai-foundry/ai-services/how-to/connect-azure-openai.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,5 +145,5 @@ At some point, you want to develop apps with code. Here are some developer resou
145145

146146
## Related content
147147

148-
- [Azure OpenAI in Azure AI Foundry portal](../../azure-openai-in-ai-foundry.md)
148+
- [Azure OpenAI in Azure AI Foundry portal](../../azure-openai-in-azure-ai-foundry.md)
149149
- [Use Azure AI services resources](./connect-ai-services.md)
File renamed without changes.

articles/ai-foundry/concepts/ai-resources.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -129,5 +129,5 @@ In the Azure portal, you can find resources that correspond to your project in A
129129
## Next steps
130130

131131
- [Quickstart: Analyze images and video in the chat playground](/azure/ai-services/openai/gpt-v-quickstart)
132-
- [Learn more about Azure AI Foundry](../what-is-ai-foundry.md)
132+
- [Learn more about Azure AI Foundry](../what-is-azure-ai-foundry.md)
133133
- [Learn more about projects](../how-to/create-projects.md)

articles/ai-foundry/concepts/architecture.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ At the top level, Azure AI Foundry provides access to the following resources:
2828

2929
When you use Azure AI Foundry portal, you can directly work with Azure OpenAI without an Azure Studio project. Or you can use Azure OpenAI through a project.
3030

31-
For more information, visit [Azure OpenAI in Azure AI Foundry portal](../azure-openai-in-ai-foundry.md).
31+
For more information, visit [Azure OpenAI in Azure AI Foundry portal](../azure-openai-in-azure-ai-foundry.md).
3232

3333
- **Management center**: The management center streamlines governance and management of Azure AI Foundry resources such as hubs, projects, connected resources, and deployments.
3434

@@ -108,7 +108,7 @@ Azure AI services including Azure OpenAI provide control plane endpoints for ope
108108

109109
To reduce the complexity of Azure RBAC management, Azure AI Foundry provides a *control plane proxy* that allows you to perform operations on connected Azure AI services and Azure OpenAI resources. Performing operations on these resources through the control plane proxy only requires Azure RBAC permissions on the hub. The Azure AI Foundry service then performs the call to the Azure AI services or Azure OpenAI control plane endpoint on your behalf.
110110

111-
For more information, see [Role-based access control in Azure AI Foundry portal](rbac-ai-foundry.md).
111+
For more information, see [Role-based access control in Azure AI Foundry portal](rbac-azure-ai-foundry.md).
112112

113113
## Attribute-based access control
114114

articles/ai-foundry/concepts/concept-model-distillation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,6 @@ We used an advanced prompt during synthetic data generation. The advanced prompt
4141

4242
## Related content
4343

44-
- [What is Azure AI Foundry?](../what-is-ai-foundry.md)
44+
- [What is Azure AI Foundry?](../what-is-azure-ai-foundry.md)
4545
- [Deploy Meta Llama 3.1 models with Azure AI Foundry](../how-to/deploy-models-llama.md)
4646
- [Azure AI Foundry FAQ](../faq.yml)

articles/ai-foundry/concepts/concept-synthetic-data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,6 @@ To see how to generate synthetic data, you can use the [sample notebook](https:/
3535

3636
## Related content
3737

38-
- [What is Azure AI Foundry?](../what-is-ai-foundry.md)
38+
- [What is Azure AI Foundry?](../what-is-azure-ai-foundry.md)
3939
- [Deploy Meta Llama 3.1 models with Azure AI Foundry](../how-to/deploy-models-llama.md)
4040
- [Azure AI Foundry FAQ](../faq.yml)

articles/ai-foundry/concepts/connections.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ When you create a connection with an existing Azure storage account, you can cho
5454
- **Identity-based**: Use your Microsoft Entra ID or managed identity to authenticate data access.
5555

5656
> [!TIP]
57-
> When you use an identity-based connection, Azure role-based access control (Azure RBAC) determines who can access the connection. You must assign the correct Azure RBAC roles to your developers before they can use the connection. For more information, see [Scenario: Connections using Microsoft Entra ID](rbac-ai-foundry.md#scenario-connections-using-microsoft-entra-id-authentication).
57+
> When you use an identity-based connection, Azure role-based access control (Azure RBAC) determines who can access the connection. You must assign the correct Azure RBAC roles to your developers before they can use the connection. For more information, see [Scenario: Connections using Microsoft Entra ID](rbac-azure-ai-foundry.md#scenario-connections-using-microsoft-entra-id-authentication).
5858
5959

6060
The following table shows the supported Azure cloud-based storage services and authentication methods:
@@ -84,7 +84,7 @@ A Uniform Resource Identifier (URI) represents a storage location on your local
8484

8585
Connections allow you to securely store credentials, authenticate access, and consume data and information. Secrets associated with connections are securely persisted in the corresponding Azure Key Vault, adhering to robust security and compliance standards. As an administrator, you can audit both shared and project-scoped connections on a hub level.
8686

87-
Azure connections serve as key vault proxies, and interactions with connections are direct interactions with an Azure key vault. Azure AI Foundry connections store API keys securely, as secrets, in a key vault. The key vault [Azure role-based access control (Azure RBAC)](./rbac-ai-foundry.md) controls access to these connection resources. A connection references the credentials from the key vault storage location for further use. You won't need to directly deal with the credentials after they're stored in the hub's key vault. You have the option to store the credentials in the YAML file. A CLI command or SDK can override them. We recommend that you avoid credential storage in a YAML file, because a security breach could lead to a credential leak.
87+
Azure connections serve as key vault proxies, and interactions with connections are direct interactions with an Azure key vault. Azure AI Foundry connections store API keys securely, as secrets, in a key vault. The key vault [Azure role-based access control (Azure RBAC)](./rbac-azure-ai-foundry.md) controls access to these connection resources. A connection references the credentials from the key vault storage location for further use. You won't need to directly deal with the credentials after they're stored in the hub's key vault. You have the option to store the credentials in the YAML file. A CLI command or SDK can override them. We recommend that you avoid credential storage in a YAML file, because a security breach could lead to a credential leak.
8888

8989

9090
## Next steps

articles/ai-foundry/concepts/evaluation-metrics-built-in.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,114 @@ Generation quality metrics are used to assess the overall quality of the content
293293

294294
:::image type="content" source="../media/evaluations/quality-evaluation-diagram.png" alt-text="Diagram of generation quality metric workflow." lightbox="../media/evaluations/quality-evaluation-diagram.png":::
295295

296+
### AI-assisted: Intent Resolution
297+
298+
| Score characteristics | Score details |
299+
| ----- | --- |
300+
| Score range | 1 to 5 where 1 is the lowest quality and 5 is the highest quality. |
301+
| What is this metric? | Intent Resolution measures how well an agent identifies a user’s request, including how well it scopes the user’s intent, asks clarifying questions, and reminds end users of its scope of capabilities.|
302+
| How does it work? | The metric is calculated by instructing a language model to follow the definition (in the description) and a set of grading rubrics, evaluate the user inputs, and output a score on a 5-point scale (higher means better quality). See the following definition and grading rubric. |
303+
| When to use it? | The recommended scenario is evaluating agent’s ability to identify user intents from agent interactions. |
304+
| What does it need as input? | Query, Response, Tool Definitions (optional) |
305+
306+
Our definition and grading rubrics to be used by the Large Language Model judge to score this metric:
307+
308+
**Definition:**
309+
310+
Intent Resolution assesses the quality of the response given in relation to a query from a user, specifically focusing on the agent’s ability to understand and resolve the user intent expressed in the query. There's also a field for tool definitions describing the functions, if any, that are accessible to the agent and that the agent might invoke in the response if necessary.
311+
312+
**Ratings:**
313+
314+
| Intent Resolution | Definition |
315+
| ---|---|
316+
| Intent Resolution 1: Response completely unrelated to user intent. | The agent's response doesn't address the query at all. |
317+
| Intent Resolution 2: Response minimally relates to user intent. | The response shows a token attempt to address the query by mentioning a relevant keyword or concept, but it provides almost no useful or actionable information.|
318+
| Intent Resolution 3: Response partially addresses the user intent but lacks complete details. | The response provides a basic idea related to the query by mentioning a few relevant elements, but it omits several key details and specifics needed for fully resolving the user's query. |
319+
| Intent Resolution 4: Response addresses the user intent with moderate accuracy but has minor inaccuracies or omissions. | The response offers a moderately detailed answer that includes several specific elements relevant to the query, yet it still lacks some finer details or complete information. |
320+
| Intent Resolution 5: Response directly addresses the user intent and fully resolves it. | The response provides a complete, detailed, and accurate answer that fully resolves the user's query with all necessary information and precision. |
321+
322+
### AI-assisted: Tool Call Accuracy
323+
324+
| Score characteristics | Score details |
325+
| ----- | --- |
326+
| Score range | 1 to 5 where 1 is the lowest quality and 5 is the highest quality. |
327+
| What is this metric? | Tool Call Accuracy measures an agent’s ability to select appropriate tools, extract, and process correct parameters from previous steps of the agentic workflow. It detects whether each tool call made is accurate (binary) and reports back the average scores, which can be interpreted as a passing rate across tool calls made. |
328+
| How does it work? | The metric is calculated by instructing a language model to follow the definition (in the description) and a set of grading rubrics, evaluate the user inputs, and output a score on a 5-point scale (higher means better quality). See the following definition and grading rubric. |
329+
| When to use it? | The recommended scenario is evaluating agent’s ability to select the right tools and parameters from agentic interactions. |
330+
| What does it need as input? | Query, Response, or Tool Calls, Tool Definitions |
331+
332+
Our definition and grading rubrics to be used by the Large Language Model judge to score this metric:
333+
334+
**Definition:**
335+
336+
Tool Call Accuracy returns the correctness of a single tool call, or the passing rate of the correct tool calls among multiple ones. A correct tool call considers relevance and potential usefulness, including syntactic and semantic correctness of a proposed tool call from an intelligent system. The judgment for each tool call is based on the following provided criteria, user query, and the tool definitions available to the agent.
337+
338+
**Ratings:**
339+
340+
Criteria for an inaccurate tool call:
341+
342+
- The tool call isn't relevant and won't help resolve the user's need.
343+
- The tool call includes parameters values that aren't present or inferred from previous interaction.
344+
- The tool call has parameters not present in tool definitions.
345+
346+
Criteria for an accurate tool call:
347+
348+
- The tool call is directly relevant and very likely to help resolve the user's need.
349+
- The tool call includes parameters values that are present or inferred from previous interaction.
350+
- The tool call has parameters present in tool definitions.
351+
352+
## AI-assisted: Task Adherence
353+
354+
| Score characteristics | Score details |
355+
| ----- | --- |
356+
| Score range | 1 to 5 where 1 is the lowest quality and 5 is the highest quality. |
357+
| What is this metric? | Task Adherence measures how well an agent’s response adheres to their assigned tasks, according to their task instruction (extracted from system message and user query), and available tools. |
358+
| How does it work? | The metric is calculated by instructing a language model to follow the definition (in the description) and a set of grading rubrics, evaluate the user inputs, and output a score on a 5-point scale (higher means better quality). See the following definition and grading rubric. |
359+
| When to use it? | The recommended scenario is evaluating agent’s ability to adhere to assigned tasks. |
360+
| What does it need as input? | Query, Response, Tool Definitions (optional) |
361+
362+
Our definition and grading rubrics to be used by the Large Language Model judge to score this metric:
363+
364+
**Definition:**
365+
366+
Task Adherence assesses the quality of the response given in relation to a query from a user, specifically focusing on the agent’s ability to understand and resolve the user intent expressed in the query. There's also a field for tool definitions describing the functions, if any, that are accessible to the agent and that the agent might invoke in the response if necessary.
367+
368+
**Ratings:**
369+
370+
| Task Adherence | Definition |
371+
| ---| ---|
372+
| Task Adherence 1: Fully inadherent | The response completely ignores instructions or deviates significantly. |
373+
| Task Adherence 2: Barely adherent | The response partially aligns with instructions but has critical gaps.|
374+
| Task Adherence 3: Moderately adherent | The response meets the core requirements but lacks precision or clarity. |
375+
| Task Adherence 4: Mostly adherent | The response is clear, accurate, and aligns with instructions with minor issues. |
376+
| Task Adherence 5: Fully Adherent | The response is flawless, accurate, and follows instructions to the letter.|
377+
378+
## AI-assisted: Response Completeness
379+
380+
| Score characteristics | Score details |
381+
| ----- | --- |
382+
| Score range | 1 to 5 where 1 is the lowest quality and 5 is the highest quality. |
383+
| What is this metric? | Response Completeness measures how comprehensive an agent’s response is when compared with the ground truth provided. |
384+
| How does it work? | The metric is calculated by instructing a language model to follow the definition (in the description) and a set of grading rubrics, evaluate the user inputs, and output a score on a 5-point scale (higher means better quality). See the following definition and grading rubric. |
385+
| When to use it? | The recommended scenario is evaluating agent’s final response to be comprehensive with respect to the ground truth provided. |
386+
| What does it need as input? | Response, Ground Truth |
387+
388+
Our definition and grading rubrics to be used by the Large Language Model judge to score this metric:
389+
390+
**Definition:**
391+
392+
Response Completeness refers to how accurately and thoroughly a response represents the information provided in the ground truth. It considers both the inclusion of all relevant statements and the correctness of those statements. Each statement in the ground truth should be evaluated individually to determine if it is accurately reflected in the response.
393+
394+
**Ratings:**
395+
396+
| Response Completeness | Definition |
397+
| ---| ---|
398+
| Response Completeness 1: Fully incomplete |The response is considered fully incomplete if it doesn't contain any the necessary and relevant information with respect to the ground truth. In other words, it completely misses all the information, especially claims and statements, established in the ground truth. |
399+
| Response Completeness 2: Barely complete | The response is considered barely complete if it only contains a small percentage of all the necessary and relevant information with respect to the ground truth. In other words, it misses almost all the information, especially claims and statements, established in the ground truth. |
400+
| Response Completeness 3: Moderately complete | The response is considered moderately complete if it contains half of the necessary and relevant information with respect to the ground truth. In other words, it misses half of the information, especially claims and statements, established in the ground truth. |
401+
| Response Completeness 4: Mostly complete | The response is considered mostly complete if it contains most of the necessary and relevant information with respect to the ground truth. In other words, it misses some minor information, especially claims and statements, established in the ground truth. |
402+
| Response Completeness 5: Fully complete | The response is considered complete if it perfectly contains all the necessary and relevant information with respect to the ground truth. In other words, it doesn't miss any information from statements and claims in the ground truth. |
403+
296404
### AI-assisted: Groundedness
297405

298406
For groundedness, we provide two versions:

articles/ai-foundry/concepts/management-center.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ Assign roles, manage users, and ensure that all settings comply with organizatio
4040

4141
:::image type="content" source="../media/management-center/user-management.png" alt-text="Screenshot of the user management section of the management center." lightbox="../media/management-center/user-management.png":::
4242

43-
For more information, see [Role-based access control](rbac-ai-foundry.md#assigning-roles-in-azure-ai-foundry-portal).
43+
For more information, see [Role-based access control](rbac-azure-ai-foundry.md#assigning-roles-in-azure-ai-foundry-portal).
4444

4545
## Related content
4646

0 commit comments

Comments
 (0)