Skip to content

Conversation

@jp-agenta
Copy link
Member

release/v0.61.0

Copilot AI review requested due to automatic review settings November 10, 2025 13:58
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Nov 10, 2025
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


GitHub CI seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@jp-agenta jp-agenta enabled auto-merge November 10, 2025 14:01
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR bumps the version from 0.60.2 to 0.61.0 and introduces significant enhancements to support custom evaluations, improve metric handling, and refactor workflow utilities. Key changes include adding evaluator revision support, enhancing metric resolution with better slug tracking, implementing CSV export for custom evaluations, and refactoring SDK workflow utilities.

Key Changes

  • Added evaluator revision support with data merging and fallback mechanisms
  • Enhanced metric column factory with nested object support, improved slug resolution, and dynamic type inference
  • Implemented custom evaluation type ("custom") throughout the evaluation flow
  • Added CSV export functionality for custom evaluations with robust column flattening
  • Refactored SDK workflow utilities (renamed PARAMETERS_REGISTRY to CONFIGURATION_REGISTRY, made slug optional in builtin workflows)

Reviewed Changes

Copilot reviewed 113 out of 118 changed files in this pull request and generated 55 comments.

Show a summary per file
File Description
web/package.json, web/oss/package.json, web/ee/package.json, sdk/pyproject.toml Version bump to 0.61.0
web/oss/src/state/evaluators/atoms.ts Added evaluator revision fetching and merging logic
web/oss/src/state/app/hooks.ts, atoms/fetcher.ts Enhanced app filtering and validation logic
web/ee/.../evaluation pages Added "custom" eval type support in routing
web/ee/src/lib/metricColumnFactory.tsx Major refactoring with nested metrics and improved resolution
web/ee/.../CustomEvaluation.tsx Added CSV export with column flattening
web/ee/.../utils.tsx Enhanced evaluator metric resolution with run indexes
sdk/agenta/sdk/workflows/utils.py Renamed registry and refactored URI handling
sdk/agenta/sdk/workflows/builtin.py Made slug parameter optional
sdk/agenta/sdk/models/* Added new model files for git, evaluations, testsets, blobs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Nov 10, 2025
@jp-agenta jp-agenta merged commit 1c1b446 into main Nov 10, 2025
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backend Evaluation lgtm This PR has been approved by a maintainer SDK size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants