Evals

[cte]

Context

Implementation

Screenshots

before	after

How to Test

Get in Touch

---

Important

This pull request introduces a comprehensive evaluation framework with setup scripts, database schema, IPC communication, and configuration files for ESLint, TypeScript, and Turbo, supporting multiple programming languages and tasks.

• Setup and Configuration:
  • Adds setup.sh script for environment setup, supporting macOS only.
  • Introduces turbo.json for task management with Turbo.
  • Adds ESLint and TypeScript configurations in evals/config.
• Database:
  • Implements database schema using drizzle-orm in drizzle.config.ts and schema.ts.
  • Adds migration files in drizzle/ directory.
  • Provides database query functions in src/queries/.
• Packages:
  • Adds @evals/db, @evals/lib, and @evals/types packages with respective configurations and scripts.
  • Implements IPC client and server in @evals/ipc package.
• Utilities and Tests:
  • Adds utility functions and tests in @evals/lib.
  • Defines types and schemas in @evals/types.
• Miscellaneous:
  • Updates package.json with new scripts and dependencies.
  • Adds knip.json for project entry and ignore paths.

^{This description was created by for 99d8ba1. It will automatically update as commits are pushed.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 99d8ba1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[ellipsis-dev]

This pull request is quite large, with 140 files changed, 13,998 lines added, and 3,152 lines removed. It seems to involve two main areas:

1. Removal of benchmark-related files: This includes deleting Dockerfiles, environment samples, and various scripts and prompts related to benchmarks.
2. Addition of new configurations and components for the evals directory: This includes new ESLint configurations, package configurations, CLI utilities, and various UI components.

To make the review process more manageable and ensure each set of changes can be thoroughly reviewed, it might be beneficial to split this pull request into two separate ones:

• One pull request focused on the removal of benchmark-related files.
• Another pull request focused on the addition of new evals configurations and components.

Please consider splitting the changes if they are not directly related to one another. This will help reviewers focus on each aspect more effectively. Thank you!