@@ -5,7 +5,7 @@ description: How to run evaluation in Azure DevOps which enables offline evaluat
5
5
manager : scottpolly
6
6
ms.service : azure-ai-foundry
7
7
ms.topic : how-to
8
- ms.date : 05/19 /2025
8
+ ms.date : 07/25 /2025
9
9
ms.reviewer : hanch
10
10
ms.author : lagayhar
11
11
author : lgayhardt
@@ -21,8 +21,7 @@ Similar to the [Azure AI evaluation in GitHub Actions](evaluation-github-action.
21
21
22
22
## Prerequisites
23
23
24
- [ !INCLUDE [ hub-only-prereq] ( ../includes/hub-only-prereq.md )]
25
-
24
+ - Foundry project or Hubs based project. To learn more, see [ Create a project] ( create-projects.md ) .
26
25
- Install Azure AI evaluation extension.
27
26
- Go to [ Azure DevOps Marketplace] ( https://marketplace.visualstudio.com/azuredevops ) .
28
27
- Search for Azure AI evaluation and install the extension into your Azure DevOps organization.
@@ -65,7 +64,51 @@ Similar to the [Azure AI evaluation in GitHub Actions](evaluation-github-action.
65
64
66
65
A sample YAML file:
67
66
68
- ```yml
67
+ # [Foundry project](#tab/foundry-project)
68
+
69
+ ```yaml
70
+
71
+ trigger:
72
+ - main
73
+ pool:
74
+
75
+ vmImage: 'windows-latest'
76
+
77
+ steps:
78
+
79
+ - task: AzureCLI@2
80
+ inputs:
81
+ addSpnToEnvironment: true
82
+ azureSubscription: ${{vars.Service_Connection_Name}}
83
+ scriptType: bash
84
+ scriptLocation: inlineScript
85
+
86
+ inlineScript: |
87
+ echo "##vso[task.setvariable variable=ARM_CLIENT_ID]$servicePrincipalId"
88
+ echo "##vso[task.setvariable variable=ARM_ID_TOEKN]$idToken"
89
+ echo "##vso[task.setvariable variable=ARM_TENANT_ID]$tenantId"
90
+
91
+ - bash: |
92
+
93
+ az login --service-principal -u $(ARM_CLIENT_ID) --tenant $(ARM_TENANT_ID) --allow-no-subscriptions --federated-token $(ARM_ID_TOEKN)
94
+
95
+ displayName: 'Login Azure'
96
+
97
+ - task: UsePythonVersion@0
98
+ inputs:
99
+ versionSpec: '3.11'
100
+ - task: AIAgentEvaluation@0
101
+ inputs:
102
+ azure-ai-project-endpoint: "<your-ai-project-endpoint>"
103
+ deployment-name: "gpt-4o-mini"
104
+ data-path: $(Build.SourcesDirectory)\tests\data\golden-dataset-medium.json
105
+ agent-ids: "<your-ai-agent-ids>
106
+
107
+ ```
108
+
109
+ # [ Hub based project] ( #tab/hub-project )
110
+
111
+ ``` yaml
69
112
70
113
trigger :
71
114
- main
@@ -105,6 +148,8 @@ agent-ids: "<your-ai-agent-ids>
105
148
106
149
```
107
150
151
+ ---
152
+
108
153
## Set up a new pipeline and trigger an evaluation run
109
154
110
155
Commit and run the pipeline in Azure DevOps.
0 commit comments