Skip to content

Commit c61a2c9

Browse files
committed
fixing merge conflicts
2 parents 00bdef5 + 760558b commit c61a2c9

File tree

57 files changed

+1304
-719
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+1304
-719
lines changed

articles/ai-foundry/concepts/authentication-options-ai-foundry.md

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ ai.usage: ai-assisted
1212
---
1313

1414
# Authentication and authorization options in Azure AI Foundry
15-
15+
<!--
1616
Azure AI Foundry supports multiple authentication approaches to balance security, operational simplicity, and speed. This article explains the control plane and data plane model, compares API key and Microsoft Entra ID (formerly Azure AD) authentication, maps identities to roles, and describes common least privilege scenarios. Use this article with:
1717
1818
- [Role-based access control for Azure AI Foundry](rbac-azure-ai-foundry.md)
@@ -22,7 +22,6 @@ Azure AI Foundry supports multiple authentication approaches to balance security
2222
2323
> [!IMPORTANT]
2424
> Use Microsoft Entra ID for production workloads to enable conditional access, managed identities, and least privilege RBAC. API keys are convenient for quick evaluation and legacy tooling but lack user level traceability.
25-
2625
## Control plane vs. data plane
2726
2827
Azure services separate management (_control plane_) from runtime operations (_data plane_).
@@ -40,7 +39,6 @@ _Source file: control-data-plane.mmd (stored alongside the image for maintenance
4039
4140
> [!NOTE]
4241
> This diagram is conceptual. Check current service documentation for the latest supported resources and operations.
43-
4442
## Authentication methods
4543
4644
### API keys
@@ -72,7 +70,6 @@ Microsoft Entra ID uses OAuth 2.0 bearer tokens. Principals get tokens for the r
7270
7371
> [!IMPORTANT]
7472
> Validate features marked [**TO VERIFY**] against current release notes if you rely on them for compliance-critical scenarios.
75-
7673
| Capability or feature | API Key | Microsoft Entra ID | Notes |
7774
|---------------------|---------|--------------------|-------|
7875
| Basic model inference (chat, embeddings) | Yes | Yes | Fully supported. |
@@ -111,7 +108,6 @@ See the authoritative list in [Azure built-in roles (AI + machine learning)](/az
111108
112109
> [!TIP]
113110
> Create a custom role when a built-in role grants more permissions than you need.
114-
115111
## Set up Microsoft Entra ID
116112
117113
High-level steps. See the detailed guide: [Configure key-less authentication](../foundry-models/how-to/configure-entra-id.md).
@@ -177,4 +173,5 @@ Some creation workflows can auto assign broad roles, such as granting the resour
177173
- [Authenticate requests to Azure AI services](/azure/ai-services/authentication)
178174
- [Configure key-less authentication with Microsoft Entra ID](../foundry-models/how-to/configure-entra-id.md)
179175
- [Azure built-in roles (AI + machine learning)](/azure/role-based-access-control/built-in-roles#ai-+-machine-learning)
180-
- [Managed identities for Azure resources](/entra/identity/managed-identities-azure-resources/overview)
176+
- [Managed identities for Azure resources](/entra/identity/managed-identities-azure-resources/overview)
177+
-->
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
title: Disable Preview Features with Role-Based Access
3+
description: Learn how to disable preview features in Azure AI Foundry using role-based access control (RBAC). Create custom roles to manage feature access effectively.
4+
#customer intent: As an IT admin, I want to disable preview features in Azure AI Foundry through role-based access control so that my organization complies with enterprise policies.
5+
author: jonburchel
6+
ms.author: jburchel
7+
ms.reviewer: meerakurup
8+
ms.date: 09/25/2025
9+
ms.topic: concept-article
10+
ms.service: azure-ai-foundry
11+
ai.usage: ai-assisted
12+
---
13+
14+
# Disable preview features in Azure AI Foundry with role-based access control
15+
16+
In Azure AI Foundry projects, some features are in preview. Administrators can bock access to them by denying specific data actions to a custom role, and granting their users role memberships to enable/disable specific features as required. This article lists the data actions for each preview feature so you can disable them on an individual basis. However, since you can't modify built-in roles in Azure AI Foundry projects, you need to create a custom role. For steps to create a custom role, see [Create or update Azure custom roles using the Azure portal - Azure RBAC](/azure/role-based-access-control/custom-roles-portal).
17+
18+
## Agents service data actions
19+
20+
Use these data actions in a custom role definition:
21+
22+
- `Microsoft.CognitiveServices/accounts/AIServices/agents/write`
23+
- `Microsoft.CognitiveServices/accounts/AIServices/agents/read`
24+
- `Microsoft.CognitiveServices/accounts/AIServices/agents/delete`
25+
26+
## Content understanding (multimodal intelligence)
27+
28+
The associated data actions to allow or disallow in your custom role
29+
definition are the following:
30+
31+
- `Microsoft.CognitiveServices/accounts/MultiModalIntelligence/analyzers/read`
32+
- `Microsoft.CognitiveServices/accounts/MultiModalIntelligence/analyzers/write`
33+
- `Microsoft.CognitiveServices/accounts/MultiModalIntelligence/analyzers/delete`
34+
- `Microsoft.CognitiveServices/accounts/MultiModalIntelligence/classifiers/read`
35+
- `Microsoft.CognitiveServices/accounts/MultiModalIntelligence/classifiers/write`
36+
- `Microsoft.CognitiveServices/accounts/MultiModalIntelligence/classifiers/delete`
37+
- `Microsoft.CognitiveServices/accounts/MultiModalIntelligence/batchAnalysisJobs/\*`
38+
- Optional: include the /labelingProjects data actions if your team labels documents in Foundry.
39+
40+
## Fine-tuning
41+
42+
The associated data actions to allow or disallow in your custom role
43+
definition are the following:
44+
45+
- `Microsoft.CognitiveServices/accounts/OpenAI/assistants/\*` (include
46+
_read_, _write_, and _delete_ and all child resources)
47+
- `Microsoft.CognitiveServices/accounts/OpenAI/assistants/files/\*`
48+
- `Microsoft.CognitiveServices/accounts/OpenAI/assistants/threads/\*`
49+
- `Microsoft.CognitiveServices/accounts/OpenAI/assistants/threads/messages/\*`
50+
- `Microsoft.CognitiveServices/accounts/OpenAI/assistants/vector_stores/\*`
51+
52+
## Tracing
53+
54+
Allow or deny the following data actions in the custom role definition.
55+
56+
Foundry’s Tracing pane uses Azure Monitor. In the custom role wizard, set the provider to Microsoft.Insights, then add or remove only the read actions you need:
57+
58+
- `Microsoft.Insights/alertRules/read`
59+
- `Microsoft.Insights/diagnosticSettings/read`
60+
- `Microsoft.Insights/logDefinitions/read`
61+
- `Microsoft.Insights/metricdefinitions/read`
62+
- `Microsoft.Insights/metrics/read`
63+
64+
## Evaluation data actions
65+
66+
The associated data actions to allow or disallow in your custom role
67+
definition are the following:
68+
69+
- `Microsoft.CognitiveServices/accounts/AIServices/evaluations/write`
70+
- `Microsoft.CognitiveServices/accounts/AIServices/evaluations/read`
71+
- `Microsoft.CognitiveServices/accounts/AIServices/evaluations/delete`
72+
73+
## Content safety risks and alerts
74+
75+
The associated data actions to allow or disallow in your custom role
76+
definition are the following
77+
78+
- `Microsoft.CognitiveServices/accounts/ContentSafety/\*`
79+
- …/`Analyze Text`
80+
- …/`Analyze Image`
81+
- …/`Analyze Protected Material`
82+
- …/`Unified Analyze`
83+
84+
## Related content
85+
86+
[Role-based access control for Azure AI Foundry](rbac-azure-ai-foundry.md)

articles/ai-foundry/concepts/evaluation-evaluators/custom-evaluators.md

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
---
2-
title: Custom evaluators
2+
title: Custom Evaluators
33
titleSuffix: Azure AI Foundry
44
description: Learn how to create custom evaluators for your AI applications using code-based or prompt-based approaches.
55
author: lgayhardt
66
ms.author: lagayhar
77
ms.reviewer: mithigpe
8-
ms.date: 07/31/2025
8+
ms.date: 10/16/2025
99
ms.service: azure-ai-foundry
1010
ms.topic: reference
1111
ms.custom:
@@ -15,11 +15,11 @@ ms.custom:
1515

1616
# Custom evaluators
1717

18-
Built-in evaluators are great out of the box to start evaluating your application's generations. However you might want to build your own code-based or prompt-based evaluator to cater to your specific evaluation needs.
18+
To start evaluating your application's generations, built-in evaluators are great out of the box. To cater to your evaluation needs, you can build your own code-based or prompt-based evaluator.
1919

2020
## Code-based evaluators
2121

22-
Sometimes a large language model isn't needed for certain evaluation metrics. This is when code-based evaluators can give you the flexibility to define metrics based on functions or callable class. You can build your own code-based evaluator, for example, by creating a simple Python class that calculates the length of an answer in `answer_length.py` under directory `answer_len/`:
22+
You don't need a large language model for certain evaluation metrics. Code-based evaluators can give you the flexibility to define metrics based on functions or callable classes. You can build your own code-based evaluator, for example, by creating a simple Python class that calculates the length of an answer in `answer_length.py` under directory `answer_len/`, as in the following example.
2323

2424
### Code-based evaluator example: Answer length
2525

@@ -32,7 +32,7 @@ class AnswerLengthEvaluator:
3232
return {"answer_length": len(answer)}
3333
```
3434

35-
Then run the evaluator on a row of data by importing a callable class:
35+
Run the evaluator on a row of data by importing a callable class:
3636

3737
```python
3838
from answer_len.answer_length import AnswerLengthEvaluator
@@ -49,13 +49,17 @@ answer_length = answer_length_evaluator(answer="What is the speed of light?")
4949

5050
## Prompt-based evaluators
5151

52-
To build your own prompt-based large language model evaluator or AI-assisted annotator, you can create a custom evaluator based on a **Prompty** file. Prompty is a file with `.prompty` extension for developing prompt template. The Prompty asset is a markdown file with a modified front matter. The front matter is in YAML format that contains many metadata fields that define model configuration and expected inputs of the Prompty. Let's create a custom evaluator `FriendlinessEvaluator` to measure friendliness of a response.
52+
To build your own prompt-based large language model evaluator or AI-assisted annotator, you can create a custom evaluator based on a *Prompty* file.
53+
54+
Prompty is a file with the `.prompty` extension for developing prompt template. The Prompty asset is a markdown file with a modified front matter. The front matter is in YAML format. It contains metadata fields that define model configuration and expected inputs of the Prompty.
55+
56+
To measure friendliness of a response, you can create a custom evaluator `FriendlinessEvaluator`:
5357

5458
### Prompt-based evaluator example: Friendliness evaluator
5559

56-
First, create a `friendliness.prompty` file that describes the definition of the friendliness metric and its grading rubric:
60+
First, create a `friendliness.prompty` file that defines the friendliness metric and its grading rubric:
5761

58-
```markdown
62+
```md
5963
---
6064
name: Friendliness Evaluator
6165
description: Friendliness Evaluator to measure warmth and approachability of answers.
@@ -108,7 +112,7 @@ generated_query: {{response}}
108112
output:
109113
```
110114

111-
Then create a class `FriendlinessEvaluator` to load the Prompty file and process the outputs with json format:
115+
Then create a class `FriendlinessEvaluator` to load the Prompty file and process the outputs with JSON format:
112116

113117
```python
114118
import os
@@ -132,7 +136,7 @@ class FriendlinessEvaluator:
132136
return response
133137
```
134138

135-
Now, you can create your own Prompty-based evaluator and run it on a row of data:
139+
Now, create your own Prompty-based evaluator and run it on a row of data:
136140

137141
```python
138142
from friendliness.friend import FriendlinessEvaluator

0 commit comments

Comments
 (0)