Merge pull request #7708 from MicrosoftDocs/main

Auto Publish – main to live - 2025-10-16 17:06 UTC

Author: learn-build-service-prod[bot]

articles/ai-foundry/concepts/evaluation-evaluators/custom-evaluators.md

@@ -1,11 +1,11 @@
 ---
-title: Custom evaluators
+title: Custom Evaluators
 titleSuffix: Azure AI Foundry
 description: Learn how to create custom evaluators for your AI applications using code-based or prompt-based approaches.
 author: lgayhardt
 ms.author: lagayhar
 ms.reviewer: mithigpe
-ms.date: 07/31/2025
+ms.date: 10/16/2025
 ms.service: azure-ai-foundry
 ms.topic: reference
 ms.custom:
@@ -15,11 +15,11 @@ ms.custom:
 
 # Custom evaluators
 
-Built-in evaluators are great out of the box to start evaluating your application's generations. However you might want to build your own code-based or prompt-based evaluator to cater to your specific evaluation needs.
+To start evaluating your application's generations, built-in evaluators are great out of the box. To cater to your evaluation needs, you can build your own code-based or prompt-based evaluator.
 
 ## Code-based evaluators
 
-Sometimes a large language model isn't needed for certain evaluation metrics. This is when code-based evaluators can give you the flexibility to define metrics based on functions or callable class. You can build your own code-based evaluator, for example, by creating a simple Python class that calculates the length of an answer in `answer_length.py` under directory `answer_len/`:
+You don't need a large language model for certain evaluation metrics. Code-based evaluators can give you the flexibility to define metrics based on functions or callable classes. You can build your own code-based evaluator, for example, by creating a simple Python class that calculates the length of an answer in `answer_length.py` under directory `answer_len/`, as in the following example.
 
 ### Code-based evaluator example: Answer length
 
@@ -32,7 +32,7 @@ class AnswerLengthEvaluator:
         return {"answer_length": len(answer)}
 ```
 
-Then run the evaluator on a row of data by importing a callable class:
+Run the evaluator on a row of data by importing a callable class:
 
 ```python
 from answer_len.answer_length import AnswerLengthEvaluator
@@ -49,13 +49,17 @@ answer_length = answer_length_evaluator(answer="What is the speed of light?")
 
 ## Prompt-based evaluators
 
-To build your own prompt-based large language model evaluator or AI-assisted annotator, you can create a custom evaluator based on a **Prompty** file. Prompty is a file with `.prompty` extension for developing prompt template. The Prompty asset is a markdown file with a modified front matter. The front matter is in YAML format that contains many metadata fields that define model configuration and expected inputs of the Prompty. Let's create a custom evaluator `FriendlinessEvaluator` to measure friendliness of a response.
+To build your own prompt-based large language model evaluator or AI-assisted annotator, you can create a custom evaluator based on a *Prompty* file.
+
+Prompty is a file with the `.prompty` extension for developing prompt template. The Prompty asset is a markdown file with a modified front matter. The front matter is in YAML format. It contains metadata fields that define model configuration and expected inputs of the Prompty.
+
+To measure friendliness of a response, you can create a custom evaluator `FriendlinessEvaluator`:
 
 ### Prompt-based evaluator example: Friendliness evaluator
 
-First, create a `friendliness.prompty` file that describes the definition of the friendliness metric and its grading rubric:
+First, create a `friendliness.prompty` file that defines the friendliness metric and its grading rubric:
 
-```markdown
+```md
 ---
 name: Friendliness Evaluator
 description: Friendliness Evaluator to measure warmth and approachability of answers.
@@ -108,7 +112,7 @@ generated_query: {{response}}
 output:
 ```
 
-Then create a class `FriendlinessEvaluator` to load the Prompty file and process the outputs with json format:
+Then create a class `FriendlinessEvaluator` to load the Prompty file and process the outputs with JSON format:
 
 ```python
 import os
@@ -132,7 +136,7 @@ class FriendlinessEvaluator:
         return response
 ```
 
-Now, you can create your own Prompty-based evaluator and run it on a row of data:
+Now, create your own Prompty-based evaluator and run it on a row of data:
 
 ```python
 from friendliness.friend import FriendlinessEvaluator

articles/ai-foundry/concepts/evaluation-evaluators/textual-similarity-evaluators.md

@@ -1,11 +1,11 @@
 ---
-title: Textual similarity evaluators for generative AI
+title: Textual Similarity Evaluators for Generative AI
 titleSuffix: Azure AI Foundry
 description: Learn about textual similarity evaluators for generative AI, including semantic similarity, F1 score, BLEU, GLEU, ROUGE, and METEOR metrics.
 author: lgayhardt
 ms.author: lagayhar
 ms.reviewer: changliu2
-ms.date: 07/31/2025
+ms.date: 10/16/2025
 ms.service: azure-ai-foundry
 ms.topic: reference
 ms.custom:
@@ -15,7 +15,9 @@ ms.custom:
 
 # Textual similarity evaluators
 
-It's important to compare how closely the textual response generated by your AI system matches the response you would expect, typically called the "ground truth". Use LLM-judge metric like [`SimilarityEvaluator`](#similarity) with a focus on the semantic similarity between the generated response and the ground truth, or use metrics from the field of natural language processing (NLP) including [F1 Score](#f1-score), [BLEU](#bleu-score), [GLEU](#gleu-score), [ROUGE](#rouge-score), and [METEOR](#meteor-score) with a focus on the overlaps of tokens or n-grams between the two.
+It's important to compare how closely the textual response generated by your AI system matches the response you would expect. The expected response is called the *ground truth*.
+
+Use a LLM-judge metric like [`SimilarityEvaluator`](#similarity) with a focus on the semantic similarity between the generated response and the ground truth. Or, use metrics from the field of natural language processing (NLP), including [F1 score](#f1-score), [BLEU](#bleu-score), [GLEU](#gleu-score), [ROUGE](#rouge-score), and [METEOR](#meteor-score) with a focus on the overlaps of tokens or n-grams between the two.
 
 ## Model configuration for AI-assisted evaluators
 
@@ -36,11 +38,11 @@ model_config = AzureOpenAIModelConfiguration(
 ```
 
 > [!TIP]
-> We recommend using `o3-mini` for a balance of reasoning capability and cost efficiency.
+> We recommend that you use `o3-mini` to balance reasoning capability and cost efficiency.
 
 ## Similarity
 
-`SimilarityEvaluator` measures the degrees of semantic similarity between the generated text and its ground truth with respect to a query. Compared to other text-similarity metrics that require ground truths, this metric focuses on semantics of a response (instead of simple overlap in tokens or n-grams) and also considers the broader context of a query.
+`SimilarityEvaluator` measures the degrees of semantic similarity between the generated text and its ground truth with respect to a query. Compared to other text-similarity metrics that require ground truths, this metric focuses on semantics of a response, instead of simple overlap in tokens or n-grams. It also considers the broader context of a query.
 
 ### Similarity example
 
@@ -57,7 +59,7 @@ similarity(
 
 ### Similarity output
 
-The numerical score on a likert scale (integer 1 to 5) and a higher score means a higher degree of similarity. Given a numerical threshold (default to 3), we also output "pass" if the score >= threshold, or "fail" otherwise. Using the reason field can help you understand why the score is high or low.
+The output is a numerical score on a likert scale, integer 1 to 5. A higher score means a higher degree of similarity. Given a numerical threshold (default to 3), this example also outputs *pass* if the score >= threshold, or *fail* otherwise. Use the reason field to understand why the score is high or low.
 
 ```python
 {
@@ -70,7 +72,10 @@ The numerical score on a likert scale (integer 1 to 5) and a higher score means
 
 ## F1 score
 
-`F1ScoreEvaluator` measures the similarity by shared tokens between the generated text and the ground truth, focusing on both precision and recall. The F1-score computes the ratio of the number of shared words between the model generation and the ground truth. Ratio is computed over the individual words in the generated response against those in the ground truth answer. The number of shared words between the generation and the truth is the basis of the F1 score. Precision is the ratio of the number of shared words to the total number of words in the generation. Recall is the ratio of the number of shared words to the total number of words in the ground truth.
+`F1ScoreEvaluator` measures the similarity by shared tokens between the generated text and the ground truth. It focuses on both precision and recall. The F1-score computes the ratio of the number of shared words between the model generation and the ground truth. The ratio is computed over the individual words in the generated response against those words in the ground truth answer. The number of shared words between the generation and the truth is the basis of the F1 score.
+
+- *Precision* is the ratio of the number of shared words to the total number of words in the generation. 
+- *Recall* is the ratio of the number of shared words to the total number of words in the ground truth.
 
 ### F1 score example
 
@@ -86,7 +91,7 @@ f1_score(
 
 ### F1 score output
 
-The numerical score is a 0-1 float and a higher score is better. Given a numerical threshold (default to 0.5), we also output "pass" if the score >= threshold, or "fail" otherwise.
+The numerical score is a 0-1 float. A higher score is better. Given a numerical threshold (default to 0.5), it also outputs *pass* if the score >= threshold, or *fail* otherwise.
 
 ```python
 {
@@ -98,7 +103,7 @@ The numerical score is a 0-1 float and a higher score is better. Given a numeric
 
 ## BLEU score
 
-`BleuScoreEvaluator` computes the BLEU (Bilingual Evaluation Understudy) score commonly used in natural language processing (NLP) and machine translation. It measures how closely the generated text matches the reference text.
+`BleuScoreEvaluator` computes the Bilingual Evaluation Understudy (BLEU) score commonly used in natural language processing and machine translation. It measures how closely the generated text matches the reference text.
 
 ### BLEU example
 
@@ -114,7 +119,7 @@ bleu_score(
 
 ### BLEU output
 
-The numerical score is a 0-1 float and a higher score is better. Given a numerical threshold (default to 0.5), we also output "pass" if the score >= threshold, or "fail" otherwise.
+The numerical score is a 0-1 float. A higher score is better. Given a numerical threshold (default to 0.5), it also outputs *pass* if the score >= threshold, or *fail* otherwise.
 
 ```python
 {
@@ -126,7 +131,7 @@ The numerical score is a 0-1 float and a higher score is better. Given a numeric
 
 ## GLEU score
 
-`GleuScoreEvaluator` computes the GLEU (Google-BLEU) score. It measures the similarity by shared n-grams between the generated text and ground truth, similar to the BLEU score, focusing on both precision and recall. But it addresses the drawbacks of the BLEU score using a per-sentence reward objective.
+`GleuScoreEvaluator` computes the Google-BLEU (GLEU) score. It measures the similarity by shared n-grams between the generated text and ground truth. Similar to the BLEU score, it focuses on both precision and recall. It addresses the drawbacks of the BLEU score using a per-sentence reward objective.
 
 ### GLEU score example
 
@@ -142,7 +147,7 @@ gleu_score(
 
 ### GLEU score output
 
-The numerical score is a 0-1 float and a higher score is better. Given a numerical threshold (default to 0.5), we also output "pass" if the score >= threshold, or "fail" otherwise.
+The numerical score is a 0-1 float. A higher score is better. Given a numerical threshold (default to 0.5), it also outputs *pass* if the score >= threshold, or *fail* otherwise.
 
 ```python
 {
@@ -154,7 +159,7 @@ The numerical score is a 0-1 float and a higher score is better. Given a numeric
 
 ## ROUGE score
 
-`RougeScoreEvaluator` computes the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) scores, a set of metrics used to evaluate automatic summarization and machine translation. It measures the overlap between generated text and reference summaries. ROUGE focuses on recall-oriented measures to assess how well the generated text covers the reference text. The ROUGE score is composed of precision, recall, and F1 score.
+`RougeScoreEvaluator` computes the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores, a set of metrics used to evaluate automatic summarization and machine translation. It measures the overlap between generated text and reference summaries. ROUGE focuses on recall-oriented measures to assess how well the generated text covers the reference text. The ROUGE score is composed of precision, recall, and F1 score.
 
 ### ROUGE score example
 
@@ -170,7 +175,7 @@ rouge(
 
 ### ROUGE score output
 
-The numerical score is a 0-1 float and a higher score is better. Given a numerical threshold (default to 0.5), we also output "pass" if the score >= threshold, or "fail" otherwise.
+The numerical score is a 0-1 float. A higher score is better. Given a numerical threshold (default to 0.5), it also outputs *pass* if the score >= threshold, or *fail* otherwise.
 
 ```python
 {
@@ -188,7 +193,7 @@ The numerical score is a 0-1 float and a higher score is better. Given a numeric
 
 ## METEOR score
 
-`MeteorScoreEvaluator` measures the similarity by shared n-grams between the generated text and the ground truth, similar to the BLEU score, focusing on precision and recall. But it addresses limitations of other metrics like the BLEU score by considering synonyms, stemming, and paraphrasing for content alignment.
+`MeteorScoreEvaluator` measures the similarity by shared n-grams between the generated text and the ground truth. Similar to the BLEU score, it focuses on precision and recall. It addresses limitations of other metrics like the BLEU score by considering synonyms, stemming, and paraphrasing for content alignment.
 
 ### METEOR score example
 
@@ -204,7 +209,7 @@ meteor_score(
 
 ### METEOR score output
 
-The numerical score is a 0-1 float and a higher score is better. Given a numerical threshold (default to 0.5), we also output "pass" if the score >= threshold, or "fail" otherwise.
+The numerical score is a 0-1 float. A higher score is better. Given a numerical threshold (default to 0.5), it also outputs *pass* if the score >= threshold, or *fail* otherwise.
 
 ```python
 {