🏃‍♂️ Add auto-generated small benchmarks on merge

[codegen-sh]

Summary

This PR implements auto-generated small benchmarks that run on every merge into the grainchain repo, as requested. The system now supports a dual benchmark architecture:

🏃‍♂️ Small Benchmarks (New - Auto on Merge)

• Trigger: Automatically run on every PR merge to main
• Iterations: 10 per provider (fast execution ~2-3 minutes)
• Providers: local, e2b
• Purpose: Quick regression detection and merge validation
• Configuration: benchmarks/configs/merge-small.json

🔬 Comprehensive Benchmarks (Existing - Manual/Scheduled)

• Trigger: Manual execution or daily scheduled runs
• Iterations: 3+ per provider (thorough analysis)
• Purpose: Detailed performance analysis and provider comparison
• Providers: local, e2b, daytona, morph

Changes Made

New Files

• .github/workflows/merge-benchmark.yml - New workflow that triggers on PR merges
• benchmarks/configs/merge-small.json - Lightweight config with 10 iterations

Modified Files

• .github/workflows/benchmark.yml - Renamed to "Comprehensive Benchmarks" for clarity
• README.md - Updated documentation to explain the dual benchmark system

Key Features

✅ Merge-triggered: Runs automatically when PRs are merged to main
✅ Fast execution: 10 iterations per provider for quick feedback
✅ PR comments: Automatically posts benchmark results as PR comments
✅ Artifact storage: Results saved as GitHub Actions artifacts
✅ Git integration: Results committed back to repo for tracking
✅ Manual testing: Supports workflow_dispatch for manual testing

Testing

The workflow can be tested manually using the "Run workflow" button in GitHub Actions, or will automatically trigger on the next PR merge.

Compliance

• ✅ Follows repository rules (no direct push to main)
• ✅ Uses existing benchmark infrastructure
• ✅ Maintains backward compatibility
• ✅ Respects pre-commit hooks and formatting

---

💻 View my work • About Codegen