Skip to content

Commit 2589885

Browse files
committed
update groundedness support
1 parent e6ac06a commit 2589885

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

articles/ai-foundry/how-to/develop/agent-evaluate-sdk.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -328,13 +328,14 @@ If you're using agents outside Azure AI Foundry Agent Service, you can still eva
328328

329329
Agents typically emit messages to interact with a user or other agents. Our built-in evaluators can accept simple data types such as strings in `query`, `response`, and `ground_truth` according to the [single-turn data input requirements](./evaluate-sdk.md#data-requirements-for-built-in-evaluators). However, it can be a challenge to extract these simple data types from agent messages, due to the complex interaction patterns of agents and framework differences. For example, a single user query can trigger a long list of agent messages, typically with multiple tool calls invoked.
330330

331-
As illustrated in the following example, we enable agent message support specifically for the built-in evaluators `IntentResolutionEvaluator`, `ToolCallAccuracyEvaluator`, and `TaskAdherenceEvaluator` to evaluate these aspects of agentic workflow. These evaluators take `tool_calls` or `tool_definitions` as parameters unique to agents.
331+
As illustrated in the following example, we enable agent message support for the following built-in evaluators to evaluate these aspects of agentic workflow. These evaluators may take `tool_calls` or `tool_definitions` as parameters unique to agents when evaluating agents.
332332

333333
| Evaluator | `query` | `response` | `tool_calls` | `tool_definitions` |
334334
|----------------|---------------|---------------|---------------|---------------|
335335
| `IntentResolutionEvaluator` | Required: `Union[str, list[Message]]` | Required: `Union[str, list[Message]]` | Doesn't apply | Optional: `list[ToolCall]` |
336336
| `ToolCallAccuracyEvaluator` | Required: `Union[str, list[Message]]` | Optional: `Union[str, list[Message]]` | Optional: `Union[dict, list[ToolCall]]` | Required: `list[ToolDefinition]` |
337337
| `TaskAdherenceEvaluator` | Required: `Union[str, list[Message]]` | Required: `Union[str, list[Message]]` | Doesn't apply | Optional: `list[ToolCall]` |
338+
| `GroundednessEvaluator` | Required: `Union[str, list[Message]]` | Required: `Union[str, list[Message]]` | Doesn't apply | Required: `list[ToolCall]` |
338339

339340
- `Message`: `dict` OpenAI-style message that describes agent interactions with a user, where the `query` must include a system message as the first message.
340341
- `ToolCall`: `dict` that specifies tool calls invoked during agent interactions with a user.

0 commit comments

Comments
 (0)