Skip to content

feat: use collections metrics for AG-UI evaluation #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: feature/ag-ui
Choose a base branch
from
Open

feat: use collections metrics for AG-UI evaluation #1

wants to merge 1 commit into from

Conversation

@contextablemark
Copy link
Owner

@contextablemark contextablemark commented Nov 12, 2025 •
edited
Loading

Summary

  • add an adapter so AG-UI evaluations can consume collections metrics through the existing ragas.evaluate pipeline
  • update the AG-UI example script to build the evaluator with llm_factory, run the new collections metrics, and log richer summaries
  • refresh the AG-UI documentation to highlight the modern LLM setup and bundled metrics

Testing

  • pytest tests/unit/integrations/test_ag_ui.py -k evaluate_ag_ui_agent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

codex

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@contextablemark