Improve Docker setup for evals

[cte]

Description

Update the Dockerfile to reflect the new monorepo structure, and use a docker-compose.yml to manage multiple images.

---

Important

Enhance Docker setup and task management for evals with improved isolation, real-time monitoring, and streamlined configuration.

• Docker Setup:
  • Updated Dockerfile.runner and Dockerfile.web to install necessary packages and configure environments for evals.
  • Added docker-compose.yml to manage services like db, redis, web, and runner.
  • Introduced entrypoint scripts runner.sh and web.sh for container initialization.
• Task Management:
  • Implemented processTask and runTask functions in processTask.ts and runTask.ts for task execution and management.
  • Utilized Redis for task event publishing and runner registration in redis.ts.
  • Added runEvals.ts to orchestrate task execution using p-queue for concurrency control.
• Real-time Monitoring:
  • Integrated Server-Sent Events (SSE) using SSEStream in sse-stream.ts for real-time task updates.
  • Added use-event-source.ts and use-runners.ts hooks for client-side event handling and runner status tracking.
• Configuration and Scripts:
  • Updated .dockerignore and package.json files to reflect new dependencies and scripts.
  • Added check-services.sh to verify required services are running before starting the application.
  • Modified setup.sh to automate environment setup and Docker service checks.
• Testing and Documentation:
  • Added sse-stream.spec.ts for testing SSE stream functionality.
  • Updated README.md and added ARCHITECTURE.md to document system architecture and setup instructions.

^{This description was created by for 6d8dba3. You can customize this summary. It will automatically update as commits are pushed.}

[ellipsis-dev]

Remove large blocks of commented-out code if no longer needed for clarity.