Add comprehensive evaluation infrastructure for spec and plan templates

[kfinkels]

Summary

• Implemented comprehensive evaluation infrastructure using PromptFoo to test spec and plan template quality
• Added custom annotation tool with FastHTML-based UI for manual spec evaluation
• Integrated automated error analysis for iterative prompt refinement
• Achieved 90% evaluation pass rate through systematic prompt improvements
• Added GitHub Actions workflow for continuous evaluation on pull requests
• Created dataset of 17 real specs and 2 real plans for testing
• Made LLM model configurable via environment variables

Key Features

• PromptFoo Integration: Automated evaluation framework with custom graders for assessing spec/plan quality
• Annotation Tool: Interactive web UI for reviewing and annotating evaluation results
• Error Analysis: Automated analysis of failures with actionable improvement recommendations
• GitHub Actions: CI/CD integration for running evaluations on PRs
• Comprehensive Documentation: Setup guides, workflows, and quick reference materials
• Model Flexibility: Configurable LLM provider and model selection

Test plan

• Run evaluation locally using evals/scripts/run-promptfoo-eval.sh
• Verify GitHub Actions workflow executes successfully on PR
• Test annotation tool with evals/scripts/run-annotation-tool.sh
• Review automated error analysis output
• Confirm evaluation scores meet minimum thresholds (60% for spec, 75% for plan)