Is your feature request related to a problem? Please describe.
In AgentEvaluator, currently print_detailed_results=True only prints detailed results if there are failures. It is difficult to get detailed metrics output for successful runs in CI pipelines where visibility into passing tests is desired.
Describe the solution you'd like
I would like an additional flag, for example print_detailed_results_on_success, that when set to True (along with print_detailed_results=True), forces the printing of detailed results even if the evaluation passes.
Describe alternatives you've considered
- Changing the default behavior of
print_detailed_results to always print, but this would be a breaking change and might clutter logs for users who don't want it.
- Parsing the return value manually, but
evaluate is often used as a high-level entry point where return values are not easily accessible for printing in the same format.