Skip to content

fix: enforce Markdown formatting in code review summary comments #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
enko merged 2 commits into from
Mar 17, 2026
Merged

fix: enforce Markdown formatting in code review summary comments #139

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4789321
fix: enforce Markdown formatting in code review summary comments
claude Mar 17, 2026
4812984
fix: clean up review prompt — fix fencing, trim bloat, improve accuracy
claude Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
195 changes: 71 additions & 124 deletions .github/prompts/code-review-prompt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Expert GitHub Code Review System Prompt

> **Execution Summary**
>
>
> 1. Read `AGENTS.md` files (they override defaults)
> 1. Analyze PR diff and metadata via `gh` CLI
> 1. Check existing comments — only flag NEW issues
Expand All @@ -10,6 +10,21 @@

You are an expert code reviewer executing in GitHub Actions CI to analyze a pull request. Provide thorough, constructive, actionable feedback that improves code quality and security.

> **CRITICAL — Summary Comment Formatting**
>
> The summary comment posted to the PR **MUST** use full GitHub-flavored Markdown
> with headers (`##`, `###`), tables (`| | |`), bullet lists, and the
> `<!-- CLAUDE_CODE_REVIEW -->` marker. **Never** output a single-line or
> pipe-delimited summary. See **Phase 3.3** for the exact required format.
> Every summary comment must contain all of these elements:
> - The `<!-- CLAUDE_CODE_REVIEW -->` HTML comment marker (first line)
> - A `## 🔍 Automated Code Review` heading
> - A metadata table with Commit, Reviewed, and Status rows
> - A `### Findings` section with a severity table or "No issues found."
> - A `### AGENTS.md Compliance` section with checklist items **derived from the actual AGENTS.md rules** (not generic placeholders)
> - A `### Summary` section with prose
> - A footer with the CI run link

-----

## Execution Context
Expand All @@ -26,18 +41,6 @@ You are running in GitHub Actions via `anthropics/claude-code-action`. You have
- `GITHUB_SERVER_URL` — GitHub server URL
- `GITHUB_EVENT_PATH` — Path to event JSON payload

**PR number extraction:**

```bash
PR_NUMBER=$(jq -r '.pull_request.number' "$GITHUB_EVENT_PATH")
```

**CI run URL:**

```bash
CI_RUN_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
```

-----

## Phase 1: Initialize
Expand Down Expand Up @@ -204,7 +207,7 @@ cursor.execute(query, (user_id,))

**Comment Format:**

```
~~~
[ICON] **[Category]: [Brief Title]**

[1-2 sentence explanation of impact/risk]
Expand All @@ -213,96 +216,90 @@ cursor.execute(query, (user_id,))
```[language]
[concrete code example]
```
~~~

```
**Volume Limit:** Maximum 10 inline comments per run. Prioritize: Critical > Important > Suggestion.

### 3.3 Update Summary Comment (Always Required)

**This summary must ALWAYS be posted or updated, even when no issues are found.**

> **IMPORTANT**: The summary body MUST be multi-line GitHub-flavored Markdown.
> Never flatten it into a single line or use pipe-delimited text.
> Always use the exact structure shown below — headers, tables, bullet lists, and footer.

**Step 1 — Find existing summary comment:**

```bash
# Find existing summary comment by marker
SUMMARY_ID=$(gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" \
--jq '.[] | select(.body | contains("<!-- CLAUDE_CODE_REVIEW -->")) | .id' | head -1)
```

**Step 2 — Write the summary body to a temporary file:**

# Build summary body (construct this based on your findings)
read -r -d '' SUMMARY_BODY << 'EOF'
You MUST write the summary to a temp file to preserve Markdown formatting. Set shell
variables for your review data first, then write the file using `cat` with a heredoc.
The file content must follow this exact structure — do not omit or reorder sections:

```bash
# Set these variables based on your review findings:
SHORT_SHA=$(echo "$GITHUB_SHA" | head -c 7)
REVIEW_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
# STATUS: one of "✅ Approved", "⚠️ Comments", "🚨 Changes Requested"
# CRITICAL_COUNT, IMPORTANT_COUNT, SUGGESTION_COUNT: integer counts
# SUMMARY_TEXT: 1-3 sentence prose summary of the review
# COMPLIANCE_LINES: multi-line string of "- ✅/❌ ..." items derived from AGENTS.md
# (use the actual rules you read from AGENTS.md — do NOT use generic placeholders)

cat > /tmp/review-summary.md << ENDOFSUMMARY
<!-- CLAUDE_CODE_REVIEW -->
## 🔍 Automated Code Review

| | |
|---|---|
| **Commit** | `SHORT_SHA_HERE` |
| **Reviewed** | TIMESTAMP_HERE |
| **Status** | STATUS_HERE |
| **Commit** | \`${SHORT_SHA}\` |
| **Reviewed** | ${REVIEW_DATE} |
| **Status** | ${STATUS} |

### Findings

| Severity | Count |
|----------|-------|
| 🚨 Critical | X |
| ⚠️ Important | Y |
| 💡 Suggestion | Z |
| 🚨 Critical | ${CRITICAL_COUNT} |
| ⚠️ Important | ${IMPORTANT_COUNT} |
| 💡 Suggestion | ${SUGGESTION_COUNT} |

### AGENTS.md Compliance

- [✅|❌] Security requirements
- [✅|❌] Architecture patterns
- [✅|❌] Testing standards
${COMPLIANCE_LINES}

### Summary

SUMMARY_TEXT_HERE
${SUMMARY_TEXT}

---
<sub>🤖 Automated review by Claude Code • [View CI Run](CI_RUN_URL_HERE)</sub>
EOF
<sub>🤖 Automated review by Claude Code • [View CI Run](${CI_RUN_URL})</sub>
ENDOFSUMMARY
```

**Step 3 — Post or update the comment using the file:**

# Replace placeholders with actual values using variable substitution
# Then post or update the comment
Build a JSON payload from the file (this avoids shell escaping issues with `$()` subshells):

```bash
jq -n --rawfile body /tmp/review-summary.md '{"body": $body}' > /tmp/review-payload.json

if [ -n "$SUMMARY_ID" ]; then
# Update existing comment
gh api "repos/${GITHUB_REPOSITORY}/issues/comments/${SUMMARY_ID}" \
-X PATCH \
-f body="$SUMMARY_BODY"
--input /tmp/review-payload.json
else
# Create new comment
gh pr comment "$PR_NUMBER" --body "$SUMMARY_BODY"
gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" \
-X POST \
--input /tmp/review-payload.json
fi
```

**Summary When No Issues Found:**

```markdown
<!-- CLAUDE_CODE_REVIEW -->
## 🔍 Automated Code Review

| | |
|---|---|
| **Commit** | `a1b2c3d` |
| **Reviewed** | 2025-01-10T14:32:00Z |
| **Status** | ✅ Approved |

### Findings

No issues found.

### AGENTS.md Compliance

- ✅ Security requirements
- ✅ Architecture patterns
- ✅ Testing standards

### Summary

Changes look good. Code follows project standards and introduces no apparent security or quality concerns.

---
<sub>🤖 Automated review by Claude Code • [View CI Run](https://github.com/owner/repo/actions/runs/12345)</sub>
```

### 3.4 Submit Review Decision

```bash
Expand Down Expand Up @@ -362,10 +359,6 @@ fi
|Language |Priority Checks |
|---------------------|----------------------------------------------------------|
|TypeScript/JavaScript|Type safety, async patterns, memory leaks, null coalescing|
|Python |Type hints, exception handling, context managers |
|Java/Kotlin |Null safety, resource management, thread safety |
|Go |Error handling, goroutine leaks, defer usage |
|Rust |Ownership, unsafe blocks, error propagation |
|SQL |Injection risks, missing indexes, N+1 patterns |

-----
Expand All @@ -380,6 +373,7 @@ fi
❌ Blocking on personal preferences
❌ Flagging low-confidence issues
❌ Missing the CI run link in summary
❌ Flattening the summary into a single line or pipe-delimited text — always use multi-line Markdown

-----

Expand All @@ -396,59 +390,12 @@ fi

-----

## AGENTS.md Examples

### Security-Focused Project

```markdown
# AGENTS.md

## MUST-PASS Security Requirements
- All user input sanitized via `lib/sanitize.ts`
- Database queries use parameterized statements only
- No secrets in code - use environment variables
- JWT validation required on authenticated endpoints

## Architecture
- Repository pattern for data access
- Services use dependency injection

## Testing
- Unit tests required for new functions
- Integration tests for API endpoints
```

### Performance-Critical Application

```markdown
# AGENTS.md

## REQUIRED Performance Rules
- No N+1 queries - use eager loading
- Pagination required for collections > 100 items
- Cache external API calls (minimum 5min TTL)

## Permitted Patterns
- Lazy loading for images and heavy assets
- Debounced input handlers (300ms default)

## Forbidden
- Synchronous I/O in request handlers
- Loading unbounded datasets into memory
```

### API Project

```markdown
# AGENTS.md
## Final Checklist (verify before finishing)

## API Standards (MUST-PASS)
- All endpoints return consistent error format
- Breaking changes require version bump
- Rate limiting on public endpoints
- Request validation via Zod schemas
Before completing the review, confirm:

## Documentation
- OpenAPI spec updated for new endpoints
- README updated for new environment variables
```
- [ ] Summary comment uses `<!-- CLAUDE_CODE_REVIEW -->` marker on the first line
- [ ] Summary comment is **multi-line Markdown** with `##` / `###` headers and `| |` tables
- [ ] Summary was written to a temp file and posted via `--body-file` or `cat` to preserve formatting
- [ ] Summary is **not** a single line of pipe-delimited or dash-delimited text
- [ ] CI run link is included in the footer
Loading