Skip to content

Commit 9f4735e

Browse files
authored
Merge pull request #6232 from lgayhardt/evaldevops0725
Eval Azure DevOps fdp
2 parents 5f482b3 + 122a25b commit 9f4735e

File tree

1 file changed

+49
-4
lines changed

1 file changed

+49
-4
lines changed

articles/ai-foundry/how-to/evaluation-azure-devops.md

Lines changed: 49 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: How to run evaluation in Azure DevOps which enables offline evaluat
55
manager: scottpolly
66
ms.service: azure-ai-foundry
77
ms.topic: how-to
8-
ms.date: 05/19/2025
8+
ms.date: 07/25/2025
99
ms.reviewer: hanch
1010
ms.author: lagayhar
1111
author: lgayhardt
@@ -21,8 +21,7 @@ Similar to the [Azure AI evaluation in GitHub Actions](evaluation-github-action.
2121

2222
## Prerequisites
2323

24-
[!INCLUDE [hub-only-prereq](../includes/hub-only-prereq.md)]
25-
24+
- Foundry project or Hubs based project. To learn more, see [Create a project](create-projects.md).
2625
- Install Azure AI evaluation extension.
2726
- Go to [Azure DevOps Marketplace](https://marketplace.visualstudio.com/azuredevops).
2827
- Search for Azure AI evaluation and install the extension into your Azure DevOps organization.
@@ -65,7 +64,51 @@ Similar to the [Azure AI evaluation in GitHub Actions](evaluation-github-action.
6564

6665
A sample YAML file:
6766

68-
```yml
67+
# [Foundry project](#tab/foundry-project)
68+
69+
```yaml
70+
71+
trigger:
72+
- main
73+
pool:
74+
75+
  vmImage: 'windows-latest' 
76+
77+
steps:
78+
79+
- task: AzureCLI@2
80+
  inputs:
81+
    addSpnToEnvironment: true
82+
    azureSubscription: ${{vars.Service_Connection_Name}}
83+
    scriptType: bash
84+
    scriptLocation: inlineScript    
85+
86+
    inlineScript: |
87+
      echo "##vso[task.setvariable variable=ARM_CLIENT_ID]$servicePrincipalId" 
88+
      echo "##vso[task.setvariable variable=ARM_ID_TOEKN]$idToken"
89+
      echo "##vso[task.setvariable variable=ARM_TENANT_ID]$tenantId"
90+
91+
- bash: |
92+
93+
   az login --service-principal -u $(ARM_CLIENT_ID) --tenant $(ARM_TENANT_ID) --allow-no-subscriptions --federated-token $(ARM_ID_TOEKN)
94+
95+
  displayName: 'Login Azure'
96+
97+
- task: UsePythonVersion@0
98+
  inputs:
99+
    versionSpec: '3.11'
100+
- task: AIAgentEvaluation@0
101+
  inputs:
102+
    azure-ai-project-endpoint: "<your-ai-project-endpoint>"
103+
    deployment-name: "gpt-4o-mini"
104+
    data-path: $(Build.SourcesDirectory)\tests\data\golden-dataset-medium.json
105+
agent-ids: "<your-ai-agent-ids>
106+
107+
```
108+
109+
# [Hub based project](#tab/hub-project)
110+
111+
```yaml
69112

70113
trigger:
71114
- main
@@ -105,6 +148,8 @@ agent-ids: "<your-ai-agent-ids>
105148

106149
```
107150

151+
---
152+
108153
## Set up a new pipeline and trigger an evaluation run
109154

110155
Commit and run the pipeline in Azure DevOps.

0 commit comments

Comments
 (0)