You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-foundry/how-to/evaluation-azure-devops.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,7 +32,7 @@ Similar to Azure AI evaluation in GitHub Action, an Azure DevOps extension is al
32
32
1. Create a new YAML file in your repository.
33
33
You can use the sample YAML provided in the README or clone from the [GitHub repo](https://github.com/microsoft/ai-agent-evals?tab=readme-ov-file).
34
34
2. Configure the following inputs:
35
-
- Set up [Azure CLI](/azure/devops/pipelines/tasks/reference/azure-cli-v2?view=azure-pipelines) with [service connection](/azure/devops/pipelines/library/service-endpoints?view=azure-devops) and Azure Login.
35
+
- Set up [Azure CLI](/azure/devops/pipelines/tasks/reference/azure-cli-v2) with [service connection](/azure/devops/pipelines/library/service-endpoints?view=azure-devops&preserve-view=true) and Azure Login.
36
36
- Azure AI project connection string
37
37
- Dataset and evaluators
38
38
- Specify the evaluator names you want to use for this evaluation run.
| General purpose|[QAEvaluator](../concepts/evaluation-evaluators/general-purpose-evaluators.md##question-answering-composite-evaluator)| Not Supported | Supported |
22
+
| General purpose|[QAEvaluator](../concepts/evaluation-evaluators/general-purpose-evaluators.md#question-answering-composite-evaluator)| Not Supported | Supported |
23
23
| General purpose (AI-assisted) |[CoherenceEvaluator](../concepts/evaluation-evaluators/general-purpose-evaluators.md#coherence)| Supported | Supported |
24
24
| General purpose (AI-assisted)|[FluencyEvaluator](../concepts/evaluation-evaluators/general-purpose-evaluators.md#fluency)| Supported | Supported |
25
25
| Textual similarity |[SimilarityEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators.md#similarity)| Not Supported | Supported |
26
-
| Textual similarity |[F1ScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators#f1-score)| Not Supported | Supported |
27
-
| Textual similarity |[RougeScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators)| Not Supported | Not Supported |
28
-
| Textual similarity |[GleuScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators#gleu-score)| Not Supported | Supported |
29
-
| Textual similarity |[BleuScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators#bleu-score)| Not Supported | Supported |
30
-
| Textual similarity |[MeteorScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators#meteor-score)| Not Supported | Supported |
31
-
| Retrieval-augmented Generation (RAG) (AI-assisted)|[GroundednessEvaluator](../concepts/evaluation-evaluators/rag-evaluators.md#groundedness)| Not Supported | Supported |
26
+
| Textual similarity |[F1ScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators.md#f1-score)| Not Supported | Supported |
27
+
| Textual similarity |[RougeScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators.md)| Not Supported | Not Supported |
28
+
| Textual similarity |[GleuScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators.md#gleu-score)| Not Supported | Supported |
29
+
| Textual similarity |[BleuScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators.md#bleu-score)| Not Supported | Supported |
30
+
| Textual similarity |[MeteorScoreEvaluator](../concepts/evaluation-evaluators/textual-similarity-evaluators.md#meteor-score)| Not Supported | Supported |
31
+
| Retrieval-augmented Generation (RAG) (AI-assisted)|[GroundednessEvaluator](../concepts/evaluation-evaluators/rag-evaluators.md#groundedness)| Not Supported | Supported |
| Retrieval-augmented Generation (RAG) (AI-assisted) |[DocumentRetrievalEvaluator](../concepts/evaluation-evaluators/rag-evaluators.md#document-retrieval)| Not Supported | Not Supported |
37
-
| Risk and safety (AI-assisted)|[ViolenceEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#violent-content)| Supported | Supported |
37
+
| Risk and safety (AI-assisted)|[ViolenceEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#violent-content)| Supported | Supported |
38
38
| Risk and safety (AI-assisted) |[SexualEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#sexual-content)| Supported | Supported |
39
39
| Risk and safety (AI-assisted) |[SelfHarmEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#self-harm-related-content)| Supported | Supported |
40
-
| Risk and safety (AI-assisted)|[HateUnfairnessEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#hateful-and-unfair-content)| Supported | Supported |
41
-
| Risk and safety (AI-assisted)|[IndirectAttackEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#indirect-attack-jailbreak-xpia)| Supported | Supported |
42
-
| Risk and safety (AI-assisted)|[ProtectedMaterialEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#protected-material-content)| Supported | Supported |
40
+
| Risk and safety (AI-assisted)|[HateUnfairnessEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#hateful-and-unfair-content)| Supported | Supported |
41
+
| Risk and safety (AI-assisted)|[IndirectAttackEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#indirect-attack-jailbreak-xpia)| Supported | Supported |
42
+
| Risk and safety (AI-assisted)|[ProtectedMaterialEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#protected-material-content)| Supported | Supported |
43
43
| Risk and safety (AI-assisted)|[CodeVulnerabilityEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#code-vulnerability)| Supported | Supported |
44
44
| Risk and safety (AI-assisted)|[UngroundedAttributesEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md##ungrounded-attributes)| Not Supported | Supported |
45
45
| Risk and safety (AI-assisted) |[ContentSafetyEvaluator](../concepts/evaluation-evaluators/risk-safety-evaluators.md#content-safety-composite-evaluator)| Supported | Supported |
0 commit comments