You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Built-in quality and safety metrics take in query and response pairs, along with additional information for specific evaluators.
47
47
@@ -161,11 +161,11 @@ You can do this with functionality and attack datasets generated with the [direc
161
161
162
162
### Composite evaluators
163
163
Composite evaluators are built in evaluators that combine the individual quality or safety metrics to easily provide a wide range of metrics right out of the box for both query response pairs or chat messages.
|`QAEvaluator`|`GroundednessEvaluator`, `RelevanceEvaluator`, `CoherenceEvaluator`, `FluencyEvaluator`, `SimilarityEvaluator`, `F1ScoreEvaluator`| Combines all the quality evaluators for a single output of combined metrics for query and response pairs |
167
-
|`ContentSafetyEvaluator`|`ViolenceEvaluator`, `SexualEvaluator`, `SelfHarmEvaluator`, `HateUnfairnessEvaluator`| Combines all the safety evaluators for a single output of combined metrics for query and response pairs |
168
164
165
+
| Composite evaluator | Contains | Description |
166
+
|--|--|--|
167
+
|`QAEvaluator`|`GroundednessEvaluator`, `RelevanceEvaluator`, `CoherenceEvaluator`, `FluencyEvaluator`, `SimilarityEvaluator`, `F1ScoreEvaluator`| Combines all the quality evaluators for a single output of combined metrics for query and response pairs |
168
+
|`ContentSafetyEvaluator`|`ViolenceEvaluator`, `SexualEvaluator`, `SelfHarmEvaluator`, `HateUnfairnessEvaluator`| Combines all the safety evaluators for a single output of combined metrics for query and response pairs |
169
169
170
170
## Custom evaluators
171
171
@@ -386,7 +386,7 @@ The `evaluate()` API has a few requirements for the data format that it accepts
386
386
387
387
#### Data format
388
388
389
-
The `evaluate()` API only accepts data in the JSONLines format. For all built-in evaluators, `evaluate()` requires data in the following format with required input fields. See the [previous section on required data input for built-in evaluators](###data-requirements-for-built-in-evaluators).
389
+
The `evaluate()` API only accepts data in the JSONLines format. For all built-in evaluators, `evaluate()` requires data in the following format with required input fields. See the [previous section on required data input for built-in evaluators](#data-requirements-for-built-in-evaluators).
Copy file name to clipboardExpand all lines: articles/ai-studio/how-to/evaluate-generative-ai-app.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -163,7 +163,7 @@ From the flow page: From the collapsible left menu, select **Prompt flow** > **E
163
163
The evaluator library is a centralized place that allows you to see the details and status of your evaluators. You can view and manage Microsoft curated evaluators.
164
164
165
165
> [!TIP]
166
-
> You can use custom evaluators via the prompt flow SDK. For more information, see [Evaluate with the prompt flow SDK](../how-to/develop/flow-evaluate-sdk.md#custom-evaluators).
166
+
> You can use custom evaluators via the prompt flow SDK. For more information, see [Evaluate with the prompt flow SDK](../how-to/develop/evaluate-sdk.md#custom-evaluators).
167
167
168
168
The evaluator library also enables version management. You can compare different versions of your work, restore previous versions if needed, and collaborate with others more easily.
169
169
@@ -172,7 +172,7 @@ To use the evaluator library in AI Studio, go to your project's **Evaluation** p
172
172
:::image type="content" source="../media/evaluations/evaluate/evaluator-library-list.png" alt-text="Screenshot of the page to select evaluators from the evaluator library." lightbox="../media/evaluations/evaluate/evaluator-library-list.png":::
173
173
174
174
You can select the evaluator name to see more details. You can see the name, description, and parameters, and check any files associated with the evaluator. Here are some examples of Microsoft curated evaluators:
175
-
- For performance and quality evaluators curated by Microsoft, you can view the annotation prompt on the details page. You can adapt these prompts to your own use case by changing the parameters or criteria according to your data and objectives [with the prompt flow SDK](../how-to/develop/flow-evaluate-sdk.md#custom-evaluators). For example, you can select *Groundedness-Evaluator* and check the Prompty file showing how we calculate the metric.
175
+
- For performance and quality evaluators curated by Microsoft, you can view the annotation prompt on the details page. You can adapt these prompts to your own use case by changing the parameters or criteria according to your data and objectives [with the prompt flow SDK](../how-to/develop/evaluate-sdk.md#custom-evaluators). For example, you can select *Groundedness-Evaluator* and check the Prompty file showing how we calculate the metric.
176
176
- For risk and safety evaluators curated by Microsoft, you can see the definition of the metrics. For example, you can select the *Self-Harm-Related-Content-Evaluator* and learn what it means and how Microsoft determines the various severity levels for this safety metric
Copy file name to clipboardExpand all lines: articles/ai-studio/quickstarts/get-started-code.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -271,7 +271,7 @@ You should see an output that looks like this:
271
271
272
272
Looks like we scored 5 for coherence and fluency of the LLM responses on this conversation!
273
273
274
-
For more information on how to use prompt flow evaluators, including how to make your own custom evaluators and log evaluation results to AI Studio, be sure to check out [Evaluate your app using the prompt flow SDK](../how-to/develop/flow-evaluate-sdk.md).
274
+
For more information on how to use prompt flow evaluators, including how to make your own custom evaluators and log evaluation results to AI Studio, be sure to check out [Evaluate your app using the prompt flow SDK](../how-to/develop/evaluate-sdk.md).
0 commit comments