Span for Output Validation

[alibeyram]

Description

We would love to see an individual span for output validation. Basically when pydantic validation fails, we can see it in the message history but not separately as a span.

If you tell me where the output validation is happening, I can add the span and create a PR as well.

My suggestion for span name is "Output Validation"

References

No response

[DouweM]

@alibeyram Thanks for filing the issue, a new span makes sense to me.

When using Tool Output mode, output (tool args) are validated here:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_tool_manager.py

Lines 166 to 169 in 53962e5

	if isinstance(call.args, str):
	args_dict = validator.validate_json(call.args or '{}', allow_partial=pyd_allow_partial)
	else:
	args_dict = validator.validate_python(call.args or {}, allow_partial=pyd_allow_partial)

That's inside ToolManager._call_tool, which is called from ToolManager.handle_call. That methods calls _call_function_tool for function tools, which adds a span and then calls _call_tool. For output tools, _call_tool is called directly, but I suggest we add a new _call_output_tool based on _call_function_tool that wraps the call in a "validate output" span.

---

When using Native or Prompted Output, the output text is validated here:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_output.py

Line 657 in 53962e5

output = self.validate(data, allow_partial)

In the call method in that same class, we use execute_traced_output_function to add spans for output function execution, so that can be used for inspiration.

That validate method is called from these two places:

pydantic-ai/pydantic_ai_slim/pydantic_ai/_agent_graph.py

Line 720 in 53962e5

result_data = await text_processor.process(text, run_context)

pydantic-ai/pydantic_ai_slim/pydantic_ai/result.py

Lines 194 to 196 in 53962e5

	result_data = await text_processor.process(
	text, self._run_ctx, allow_partial=allow_partial, wrap_validation_errors=False
	)

---

I think we should also have a span for output validators, which happen after basic Pydantic validation. You can find those calls by searching for validator.validate( in the codebase. That span could be similar to the one for output function I mentioned above.

In any case, a PR is welcome with some or all of that implemented :)