Pass a custom base image to cagent eval #195
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: PR Review on Command | |
| on: | |
| issue_comment: | |
| types: [created] | |
| pull_request_review_comment: | |
| types: [created] | |
| permissions: | |
| contents: read | |
| pull-requests: write | |
| issues: write | |
| jobs: | |
| # ========================================================================== | |
| # LEARN FROM FEEDBACK - Process replies to agent review comments | |
| # ========================================================================== | |
| learn-from-feedback: | |
| # Trigger when someone replies to a review comment that contains our marker | |
| if: > | |
| github.event_name == 'pull_request_review_comment' && | |
| github.event.comment.in_reply_to_id != null | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check if reply is to agent comment | |
| id: check | |
| env: | |
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| run: | | |
| # Get the parent comment we're replying to | |
| parent_id=${{ github.event.comment.in_reply_to_id }} | |
| parent=$(gh api repos/${{ github.repository }}/pulls/comments/$parent_id 2>/dev/null || echo "{}") | |
| parent_body=$(echo "$parent" | jq -r '.body // ""') | |
| # Check if parent comment was from cagent (contains our marker) | |
| if echo "$parent_body" | grep -q "<!-- cagent-review -->"; then | |
| echo "is_agent_comment=true" >> $GITHUB_OUTPUT | |
| echo "Found reply to agent comment" | |
| # Extract PR number from the comment's pull_request_url | |
| pr_url=$(echo "$parent" | jq -r '.pull_request_url // ""') | |
| pr_number=$(echo "$pr_url" | grep -oE '[0-9]+$' || echo "${{ github.event.pull_request.number }}") | |
| echo "pr_number=$pr_number" >> $GITHUB_OUTPUT | |
| else | |
| echo "is_agent_comment=false" >> $GITHUB_OUTPUT | |
| echo "Not a reply to agent comment, skipping" | |
| fi | |
| - name: Checkout repository | |
| if: steps.check.outputs.is_agent_comment == 'true' | |
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 | |
| - name: Restore reviewer memory database | |
| if: steps.check.outputs.is_agent_comment == 'true' | |
| uses: actions/cache@8b402f58fbc84540c8b491a91e594a4576fec3d7 | |
| with: | |
| path: .github/workflows/agents/pr-review-memory.db | |
| key: pr-review-memory-${{ github.repository }} | |
| restore-keys: | | |
| pr-review-memory-${{ github.repository }} | |
| - name: Process feedback and update memory | |
| if: steps.check.outputs.is_agent_comment == 'true' | |
| uses: docker/cagent-action@latest | |
| with: | |
| agent: ${{ github.workspace }}/.github/workflows/agents/pr-review-feedback.yaml | |
| prompt: | | |
| A developer replied to one of your previous review comments with feedback. | |
| **File:** ${{ github.event.comment.path }} | |
| **Line:** ${{ github.event.comment.line }} | |
| **Their feedback:** ${{ github.event.comment.body }} | |
| **PR:** #${{ steps.check.outputs.pr_number }} | |
| Analyze this feedback: | |
| 1. If they're correcting a false positive, add a memory to avoid this mistake | |
| 2. If they're asking for clarification, note what was unclear | |
| 3. If they're agreeing and adding context, store the additional insight | |
| Use add_memory to record what you learned. Format: | |
| "FEEDBACK: [category] - [what you learned] - Source: PR #${{ steps.check.outputs.pr_number }}" | |
| Then react to their comment with 👍 to acknowledge. | |
| anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} | |
| github-token: ${{ secrets.GITHUB_TOKEN }} | |
| - name: Save reviewer memory database | |
| if: steps.check.outputs.is_agent_comment == 'true' | |
| uses: actions/cache/save@v4 | |
| with: | |
| path: .github/workflows/agents/pr-review-memory.db | |
| key: pr-review-memory-${{ github.repository }} | |
| # ========================================================================== | |
| # MAIN REVIEW PIPELINE | |
| # ========================================================================== | |
| run-review: | |
| if: github.event.issue.pull_request && contains(github.event.comment.body, '/review') | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Checkout repository | |
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 | |
| with: | |
| fetch-depth: 0 | |
| - name: Add reaction to acknowledge | |
| env: | |
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| run: | | |
| gh api repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ | |
| -X POST -f content='eyes' | |
| - name: Get PR information | |
| id: pr-info | |
| env: | |
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| run: | | |
| PR_NUMBER=${{ github.event.issue.number }} | |
| echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT | |
| # Get changed files | |
| gh pr view $PR_NUMBER --json files -q '.files[].path' > changed_files.txt | |
| echo "Changed files:" | |
| cat changed_files.txt | |
| # Get PR metadata | |
| gh pr view $PR_NUMBER --json title,body,author,baseRefName,headRefName > pr_metadata.json | |
| - name: Restore reviewer memory database | |
| uses: actions/cache@8b402f58fbc84540c8b491a91e594a4576fec3d7 | |
| with: | |
| path: .github/workflows/agents/pr-review-memory.db | |
| key: pr-review-memory-${{ github.repository }} | |
| restore-keys: | | |
| pr-review-memory-${{ github.repository }} | |
| - name: Build review context | |
| id: context | |
| run: | | |
| PR_NUMBER=${{ github.event.issue.number }} | |
| title=$(jq -r '.title' pr_metadata.json) | |
| author=$(jq -r '.author.login' pr_metadata.json) | |
| body=$(jq -r '.body // "No description provided."' pr_metadata.json) | |
| base=$(jq -r '.baseRefName' pr_metadata.json) | |
| head=$(jq -r '.headRefName' pr_metadata.json) | |
| cat > review_context.md << EOF | |
| # Pull Request Review Request | |
| ## PR Information | |
| - **URL**: https://github.com/${{ github.repository }}/pull/$PR_NUMBER | |
| - **Title**: $title | |
| - **Author**: $author | |
| - **Branch**: $head → $base | |
| - **Files Changed**: $(wc -l < changed_files.txt | tr -d ' ') | |
| ## PR Description | |
| $body | |
| ## Changed Files | |
| EOF | |
| cat changed_files.txt >> review_context.md | |
| cat >> review_context.md << 'EOF' | |
| --- | |
| ## Instructions | |
| Execute the review pipeline: | |
| 1. **Gather**: Use `gh pr diff` to get the full diff | |
| 2. **Draft**: Delegate to `drafter` agent to generate bug hypotheses | |
| 3. **Verify**: For each hypothesis, delegate to `verifier` agent | |
| 4. **Post**: Aggregate findings and post review via `gh api` | |
| Only report CONFIRMED and LIKELY findings. Approve if no issues found. | |
| EOF | |
| echo "Context built:" | |
| wc -l review_context.md | |
| - name: Read context file | |
| id: read-context | |
| run: | | |
| echo "prompt<<EOF" >> $GITHUB_OUTPUT | |
| cat review_context.md >> $GITHUB_OUTPUT | |
| echo "EOF" >> $GITHUB_OUTPUT | |
| - name: Run PR Review Team | |
| uses: docker/cagent-action@latest | |
| with: | |
| agent: ${{ github.workspace }}/.github/workflows/agents/pr-review.yaml | |
| prompt: ${{ steps.read-context.outputs.prompt }} | |
| anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} | |
| github-token: ${{ secrets.GITHUB_TOKEN }} | |
| - name: Save reviewer memory database | |
| if: always() | |
| uses: actions/cache/save@v4 | |
| with: | |
| path: .github/workflows/agents/pr-review-memory.db | |
| key: pr-review-memory-${{ github.repository }} | |
| - name: Add completion reaction | |
| if: always() | |
| env: | |
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| run: | | |
| if [ "${{ job.status }}" == "success" ]; then | |
| gh api repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ | |
| -X POST -f content='rocket' | |
| else | |
| gh api repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ | |
| -X POST -f content='confused' | |
| fi |