GHA evals

[cte]

Description

Run evals in CI.

Triggered by adding the evals label to your PR:

Example run: https://github.com/RooCodeInc/Roo-Code/actions/runs/15563240184/job/43820736051?pr=4472

---

Important

Adds CI workflow for running evaluations with Docker support and refactors task processing scripts for improved logging and execution.

• CI Workflow:
  • Adds .github/workflows/evals.yml to run evaluations in CI when the 'evals' label is added to a PR or triggered manually.
  • Sets up Docker environment and runs evaluations using docker compose.
• Docker:
  • Updates Dockerfile.runner and Dockerfile.web to include necessary packages and configurations for running evaluations.
  • Modifies docker-compose.yml to expose necessary ports and configure services for db, redis, web, and runner.
• Scripts and Utilities:
  • Refactors processTask and runTask in runTask.ts to handle task execution and logging.
  • Introduces runCi in runCi.ts to manage CI-specific evaluation runs.
  • Updates runEvals.ts to handle task processing in a queue with logging.
  • Adds Logger class in utils.ts for improved logging capabilities.
• Miscellaneous:
  • Adds health check endpoint in route.ts to report service status.
  • Updates package.json scripts to include new Docker and evaluation commands.

^{This description was created by for 41df128. You can customize this summary. It will automatically update as commits are pushed.}

[cte]

Turns out that we still need this if you want to run the evals natively instead of inside Docker.

[jr]

Right, forgot about that case.

[cte]

I'll expand this once I've switched to a more powerful GHA runner via Blacksmith.