Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions marketplaces/default.json
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,19 @@
"cloud-native"
]
},
{
"name": "learn-from-code-review",
"source": "./learn-from-code-review",
"description": "Distill code review feedback from GitHub PRs into reusable skills and guidelines. Use when users ask to learn from code reviews, extract review patterns, or generate coding standards from historical PR feedback.",
"category": "code-quality",
"keywords": [
"code-review",
"learning",
"skills",
"guidelines",
"pr-feedback"
]
},
{
"name": "notion",
"source": "./notion",
Expand Down
64 changes: 64 additions & 0 deletions skills/learn-from-code-review/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Learn from Code Review

Distill code review feedback from GitHub pull requests into reusable skills and repository guidelines.

## What It Does

This skill analyzes PR review comments from a repository and extracts recurring patterns of feedback. These patterns are then transformed into:

- **Repository-specific skills** (`.openhands/skills/`) for domain-specific patterns
- **AGENTS.md updates** for general coding conventions and best practices

## When to Use

- "Learn from our code reviews"
- "Distill PR feedback into guidelines"
- "What patterns do reviewers keep pointing out?"
- "Generate coding standards from review history"
- Running `/learn-from-reviews`

## Requirements

- `GITHUB_TOKEN` environment variable
- GitHub CLI (`gh`) available

## Example

```
User: Learn from our code reviews over the past month

Agent: I'll analyze recent PR review comments and distill them into actionable guidelines.

[Analyzes 25 merged PRs with 150 review comments]
[Filters out bot comments and low-signal responses]
[Identifies 4 recurring patterns]

Found the following patterns from code review feedback:

1. Error Handling (12 comments)
- Always include context when logging errors
- Use structured error responses in APIs

2. Testing (8 comments)
- Add edge case tests for validation logic
- Mock external services consistently

3. Database Queries (6 comments)
- Use parameterized queries exclusively
- Add appropriate indexes for new queries

I'll create a draft PR with:
- New skill: `.openhands/skills/error-handling/SKILL.md`
- Updates to `AGENTS.md` with testing and database guidelines
```

## Output

The skill generates a draft PR containing proposed changes based on the analysis. Human review is expected before merging.

## Related Skills

- `github-pr-review` - Post structured code reviews
- `code-review` - Perform code reviews
- `skill-creator` - Create new skills manually
- `agent-memory` - Persist repository knowledge
229 changes: 229 additions & 0 deletions skills/learn-from-code-review/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
---
name: learn-from-code-review
description: Distill code review feedback from GitHub PRs into reusable skills and guidelines. This skill should be used when users ask to "learn from code reviews", "distill PR feedback", "improve coding standards", "extract learnings from reviews", or want to generate skills/guidelines from historical review comments.
triggers:
- /learn-from-reviews
- learn from code review
- distill reviews
---

# Learn from Code Review

Analyze code review comments from GitHub pull requests and distill them into reusable skills or repository guidelines that improve future code quality.

## Overview

Code review feedback contains valuable institutional knowledge that often gets buried across hundreds of PRs. This skill extracts meaningful patterns from review comments and transforms them into:

1. **Repository-specific skills** - Placed in `.openhands/skills/` for domain-specific patterns
2. **AGENTS.md guidelines** - Overall repository conventions and best practices

## Prerequisites

- `GITHUB_TOKEN` environment variable must be set
- GitHub CLI (`gh`) should be available

## Workflow

### Step 1: Identify Target Repository

Determine the repository to analyze:

```bash
# Get current repo info
gh repo view --json nameWithOwner -q '.nameWithOwner'
```

If not in a repository, ask the user which repository to analyze.

### Step 2: Fetch Review Comments

Retrieve PR review comments from the repository:

```bash
# Fetch merged PRs from the last 30 days (adjustable)
gh pr list --repo {owner}/{repo} \
--state merged \
--limit 50 \
--json number,title,mergedAt

# For each PR, fetch review comments
gh api repos/{owner}/{repo}/pulls/{pr_number}/comments \
--jq '.[] | {body: .body, path: .path, user: .user.login, created_at: .created_at}'

# Also fetch review-level comments (not tied to specific lines)
gh api repos/{owner}/{repo}/pulls/{pr_number}/reviews \
--jq '.[] | select(.body != "") | {body: .body, user: .user.login, state: .state}'
```

### Step 3: Filter and Categorize Comments

Apply noise filtering to keep only meaningful feedback:

**Exclude:**
- Bot comments (dependabot, copilot, github-actions, etc.)
- Low-signal responses ("LGTM", "+1", "looks good", "thanks", "nice")
- Comments shorter than 30 characters
- Auto-generated comments (CI status, coverage reports)

**Categorize remaining comments by:**
- Security concerns
- Performance patterns
- Code style/conventions
- Architecture/design patterns
- Error handling
- Testing requirements
- Documentation standards

### Step 4: Distill Patterns

For each category with sufficient examples (3+ similar comments), identify:

1. **The recurring issue** - What mistake or oversight keeps appearing
2. **The desired pattern** - What reviewers consistently ask for
3. **Example context** - Concrete before/after code snippets when available

### Step 5: Generate Output

Based on the patterns found, generate appropriate outputs:

#### For Specific Domain Patterns → Create Skills

When patterns are specific to a domain (e.g., database queries, API design, testing), create a skill file:

```
.openhands/skills/{domain-name}/SKILL.md
```

Example structure:
```yaml
---
name: database-queries
description: Database query patterns and best practices derived from code review feedback.
---

# Database Query Guidelines

## Patterns from Code Reviews

### Always Use Parameterized Queries
[Pattern description with examples]

### Connection Pool Management
[Pattern description with examples]
```

#### For General Guidelines → Update AGENTS.md

When patterns are general repository conventions, propose additions to AGENTS.md:

```markdown
## Code Review Learnings

### Error Handling
- Always wrap external API calls in try-catch blocks
- Log errors with context before re-throwing

### Testing
- Include edge case tests for user input validation
- Mock external services in unit tests
```

### Step 6: Create Draft PR

Open a draft PR with the proposed changes:

```bash
# Create branch
git checkout -b openhands/learn-from-reviews-$(date +%Y%m%d)

# Add generated files
git add .openhands/skills/ AGENTS.md

# Commit
git commit -m "Add learnings from code review analysis

Distilled patterns from recent PR reviews into:
- [List of new skills if any]
- AGENTS.md guideline updates"

# Push and create draft PR
git push -u origin HEAD
```

Use the `create_pr` tool to open a draft PR with a summary of:
- Number of PRs analyzed
- Number of comments processed
- Categories of patterns found
- List of proposed changes

## Example Output

### Sample Skill: API Error Handling

```yaml
---
name: api-error-handling
description: API error handling patterns learned from code review feedback.
---

# API Error Handling Patterns

Based on recurring code review feedback, follow these patterns:

## Always Return Structured Errors

❌ Avoid:
```python
return {"error": str(e)}
```

✅ Prefer:
```python
return {
"error": {
"code": "VALIDATION_ERROR",
"message": "Invalid input",
"details": {"field": "email", "reason": "Invalid format"}
}
}
```

## Log Before Returning Errors

```python
logger.error(f"API error in {endpoint}: {e}", exc_info=True)
return error_response(e)
```
```

## Configuration Options

When running this workflow, consider:

| Option | Default | Description |
|--------|---------|-------------|
| Time range | 30 days | How far back to look for PRs |
| Min comments | 3 | Minimum similar comments to form a pattern |
| Include bots | No | Whether to include bot review comments |
| Output format | both | Generate skills, AGENTS.md, or both |

## Best Practices

1. **Run periodically** - Schedule monthly or quarterly to capture evolving patterns
2. **Review before merging** - Generated content is a draft; human review is essential
3. **Iterate** - Refine patterns based on team feedback
4. **Avoid duplication** - Check existing AGENTS.md and skills before adding
5. **Cite sources** - Reference PR numbers when documenting patterns

## Limitations

- Only analyzes accessible repositories (requires appropriate permissions)
- Cannot capture verbal feedback from pair programming or meetings
- Patterns may reflect individual reviewer preferences vs. team consensus
- Historical comments may reference outdated code patterns

## Additional Resources

For posting structured code reviews, see the `github-pr-review` skill.
For creating new skills, see the `skill-creator` skill.
Loading