-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Aider-inspired polyglot benchmarks #689
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🦋 Changeset detectedLatest commit: e2bdce0 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
8a36522 to
f8a2828
Compare
|
This pull request is quite large, with 27 files changed and over 3000 lines added. It includes a variety of changes such as new features, documentation updates, and refactoring across different areas like Dockerfiles, environment variables, CLI scripts, and coding exercise instructions. To improve the review process and maintainability, it might be beneficial to split this pull request into smaller, more focused ones. Here are some suggestions on how the changes could be grouped:
Please consider splitting the pull request accordingly to facilitate a more efficient review process. Thank you! |
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Description
Explainer video: https://www.youtube.com/watch?v=iidPH5KsNog
Exercises + web-ui: https://github.com/cte/Roo-Code-Benchmark
Inspiration: https://aider.chat/2024/12/21/polyglot.html#the-polyglot-benchmark
Additional context
Related Issues
Reviewers
Important
Introduces Aider-inspired polyglot benchmarks with Docker setup, CLI, and environment configuration, supporting multiple programming languages.
benchmarkdirectory with setup scripts, prompts, and configuration files.benchmark/src/cli.tsfor running benchmarks.Dockerfileand.dockerignorefor setting up a Docker environment..env.local.samplefor environment variable configuration.package.jsoninbenchmarkfor managing dependencies and scripts.package.jsonto include benchmark-related scripts.knip.jsonand.vscodeignoreto include benchmark paths.src/core/Cline.tsandsrc/exports/api.tsto handle new benchmark tasks.src/exports/roo-code.d.tsandsrc/shared/getApiMetrics.ts.This description was created by
for f108dfa. It will automatically update as commits are pushed.