Skip to content

Conversation

@roomote
Copy link
Contributor

@roomote roomote bot commented Jul 22, 2025

This PR adds support for predefined named groups of eval exercises that can be quickly selected in the web-evals app.

Changes

  • Added exercise-groups.json file to store predefined exercise groups
  • Created a "Hard" group containing 26 challenging exercises across Go, Java, JavaScript, Python, and Rust
  • Added server action getExerciseGroups() to load exercise groups
  • Updated the NewRun component to display exercise group buttons when "Some" exercises are selected
  • Clicking a group button automatically selects all exercises in that group

UI Enhancement

When users select "Some" exercises instead of "All", they now see predefined group buttons above the exercise selector. Clicking "Hard (26)" will instantly select all 26 exercises in the Hard group.

This feature makes it easier to run specific sets of exercises without manually selecting each one.


Important

Adds predefined exercise groups to web-evals, allowing quick selection of exercise sets in the NewRun component.

  • Behavior:
    • Adds exercise-groups.json to define predefined exercise groups.
    • Implements getExerciseGroups() in exercise-groups.ts to fetch exercise groups.
    • Updates NewRun component to show group buttons when "Some" exercises are selected.
    • Clicking a group button selects all exercises in that group.
  • UI Enhancement:
    • Displays predefined group buttons in NewRun when "Some" exercises are selected.
    • "Hard (26)" button selects 26 challenging exercises across multiple languages.

This description was created by Ellipsis for 9861ca6. You can customize this summary. It will automatically update as commits are pushed.

- Added exercise-groups.json with "Hard" group containing 26 exercises
- Created server action to load exercise groups
- Updated NewRun component to display predefined exercise groups
- Added UI buttons to quickly select exercise groups
@roomote roomote bot requested review from cte, jr and mrubens as code owners July 22, 2025 22:09
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Jul 22, 2025
if (group) {
setValue("suite", "partial")
setValue("exercises", group.exercises)
toast.success(`Selected "${groupName}" exercise group with ${group.exercises.length} exercises`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the i18n function (e.g., t('...')) for the success toast message instead of inline English strings.

Suggested change
toast.success(`Selected "${groupName}" exercise group with ${group.exercises.length} exercises`)
toast.success(t('Selected "{{groupName}}" exercise group with {{count}} exercises', { groupName, count: group.exercises.length }))

This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Jul 22, 2025
@daniel-lxs
Copy link
Member

For visibility @cte

@daniel-lxs daniel-lxs moved this from Triage to PR [Draft / In Progress] in Roo Code Roadmap Jul 23, 2025
@hannesrudolph hannesrudolph added PR - Draft / In Progress and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Jul 23, 2025
@github-project-automation github-project-automation bot moved this from PR [Draft / In Progress] to Done in Roo Code Roadmap Sep 23, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Sep 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request PR - Draft / In Progress size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants