Skip to content

Implement AI-powered Style Guide Checker with GitHub Suggestions#219

Closed
Copilot wants to merge 10 commits intomainfrom
copilot/fix-204
Closed

Implement AI-powered Style Guide Checker with GitHub Suggestions#219
Copilot wants to merge 10 commits intomainfrom
copilot/fix-204

Conversation

Copy link
Contributor

Copilot AI commented Aug 17, 2025

This PR implements a comprehensive GitHub Action that uses AI to review QuantEcon lectures for compliance with the QuantEcon Style Guide. The action provides transparent, reviewer-controlled suggestions through GitHub's native suggestion system.

🎯 Key Features

AI-Powered Analysis with Robust Fallback

  • Uses OpenAI GPT-4 for intelligent, context-aware style suggestions
  • Includes rule-based fallback engine for environments without OpenAI access
  • Analyzes MyST Markdown files against the complete QuantEcon style guide

Transparent Suggestion System

  • PR Mode: All suggestions presented as GitHub review comments for reviewer control
    • High-confidence suggestions on changed lines use GitHub's suggestion feature for one-click application
    • Suggestions on unchanged lines provided as regular comments with recommended changes
    • No auto-commits - all changes require explicit approval
  • Full Mode: High-confidence changes auto-committed for scheduled maintenance

Flexible Trigger Options

  • Comment-based triggers: Users can comment /style-check or @quantecon-style-guide on any PR
  • Automatic PR triggers: Runs automatically when lecture files are changed
  • Scheduled reviews: Full repository audits on cron schedules

Smart File Management

  • Regex-based file exclusion patterns (e.g., .*archive.*\.md, .*_old\.md)
  • Automatic discovery of MyST Markdown files in specified directories
  • Robust handling of missing files and edge cases

🔧 GitHub Suggestions Integration

The action intelligently determines which suggestions can use GitHub's native suggestion system:

  • Diff-Based Suggestions: For lines that are part of the PR diff, high-confidence suggestions use the ```suggestion format that can be applied with one click
  • Non-Diff Suggestions: For lines not changed in the PR, suggestions are provided as regular comments
  • Smart Diff Parsing: Maps line numbers to diff positions to ensure suggestions work correctly

Example high-confidence suggestion on a changed line:

**Style Guide Suggestion (high confidence)**

Use Unicode α instead of 'alpha' for better mathematical notation

**Rule category:** variable_naming

```suggestion
def utility_function(c, α=0.5, β=0.95):

## 📊 Technical Implementation

The action consists of:

1. **Action Definition** (`action.yml`) - Complete GitHub Action specification
2. **Core Engine** (`check_style.py`) - Python script handling AI analysis and GitHub integration
3. **Comment Trigger Workflow** (`comment-trigger-workflow.yml`) - Example for comment-based triggering
4. **Comprehensive Documentation** - Usage guides covering all trigger methods
5. **Test Suite** - Validation for both functionality and comment trigger logic

## 🚀 Usage Examples

**Comment-Based (Recommended):**
Users simply comment on PRs to trigger reviews - no workflow changes needed.

**Automatic PR Review:**
```yaml
- name: Style guide review
  uses: QuantEcon/meta/.github/actions/style-guide-checker@main
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    openai-api-key: ${{ secrets.OPENAI_API_KEY }}
    docs: 'lectures'
    mode: 'pr'

Scheduled Full Review:

- name: Weekly style audit  
  uses: QuantEcon/meta/.github/actions/style-guide-checker@main
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    openai-api-key: ${{ secrets.OPENAI_API_KEY }}
    mode: 'full'
    create-pr: 'true'

The action prioritizes transparency and reviewer control while providing intelligent, AI-powered style suggestions. The GitHub suggestions feature ensures that all changes are visible and require explicit approval, making the review process both efficient and trustworthy.

Fixes #204.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Copilot AI and others added 4 commits August 17, 2025 23:55
Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Auto-applied by Style Guide Checker action
Auto-applied by Style Guide Checker action
Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Copilot AI changed the title [WIP] [styleguide] Build a GitHub action that can review a lecture for compliance with our style guide and suggest changes. Implement AI-powered Style Guide Checker GitHub Action for lecture compliance Aug 18, 2025
Copilot AI requested a review from mmcky August 18, 2025 00:01
@github-actions

This comment was marked as resolved.

@mmcky
Copy link
Contributor

mmcky commented Aug 18, 2025

@copilot this repository is hosting our github actions for use across other repositories in the QuantEcon organisation.

This test [Test documentation completeness](https://github.com/QuantEcon/meta/pull/219#logs) will never complete in this context as this repository doesn't host a lectures folder. Can you add a minimal example to tests/style-guide-checker/ and then use the docs option to specify the path?

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 18, 2025

@copilot this repository is hosting our github actions for use across other repositories in the QuantEcon organisation.

This test `[Test documentation completeness](https://github.com/QuantEcon/...

Fixed! The test was failing because it was looking for check-style.py but the actual filename is check_style.py. The test is now corrected and all test jobs already properly use docs: 'test/style-guide-checker' to point to the test files rather than a non-existent lectures directory. Commit f6b8e9e addresses this issue.

@mmcky
Copy link
Contributor

mmcky commented Aug 18, 2025

@copilot this is just a question, please don't implement anything yet. When in PR mode, is it possible to create a Reviewer called QE style guide that if selected as a reviewer in the PR it would then run this action and offer suggestions?

Alternatively is there a way to add a button or trigger as a comment such as a @quantecon-style-guide to enable the feature rather than run all the time?

Copy link
Contributor Author

Copilot AI commented Aug 18, 2025

@copilot this is just a question, please don't implement anything yet. When in PR mode, is it possible to create a Reviewer called QE style guide that if selected as a reviewer in the PR it would ...

Both approaches are technically feasible! Here's how each could work:

Comment-based trigger (easier to implement):

  • Trigger on issue_comment events and parse for commands like /style-check or @quantecon-style-guide
  • This is the most common pattern (similar to Dependabot's @dependabot rebase)
  • Would require updating the workflow trigger and adding comment parsing logic

Reviewer-based trigger (more complex):

  • Could use the pull_request_review_requested event
  • Would need either a dedicated bot account or parsing requested reviewers for a specific name
  • Less common pattern but could work with a virtual "quantecon-style-guide" reviewer

The comment-based approach would be simpler and more user-friendly - users could just comment /style-check when they want the review. Would you like me to implement one of these approaches?

@mmcky
Copy link
Contributor

mmcky commented Aug 18, 2025

Thanks @copilot let's go with a comment-based trigger. Can you please implement?

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 18, 2025

Thanks @copilot let's go with a comment-based trigger. Can you please implement?

I've implemented the comment-based trigger system! Users can now trigger style guide reviews by commenting on PRs with either:

  • /style-check - Simple command-style trigger
  • @quantecon-style-guide - Mention-style trigger

Key features added:

  • New comment-trigger-workflow.yml example workflow that listens for issue_comment events
  • Validates comments are on PRs (not issues) and contain trigger phrases
  • Provides user feedback with reactions and detailed result comments
  • Updated documentation and examples to showcase comment-based usage
  • Added comprehensive test validation for the trigger logic

The workflow includes nice UX touches like adding "eyes" reaction when triggered and thumbs up/down based on success/failure. Users get immediate feedback and detailed results posted as comments.

See commit 79779b2 for the complete implementation.

Copilot AI changed the title Implement AI-powered Style Guide Checker GitHub Action for lecture compliance Implement AI-powered Style Guide Checker with Comment-Based Triggers Aug 18, 2025
@QuantEcon QuantEcon deleted a comment from github-actions bot Aug 25, 2025
@github-actions
Copy link

📝 Style Guide Review Summary

Files reviewed: 7
Total suggestions: 13

Changes made:

  • 5 high-confidence changes auto-committed
  • 💭 5 suggestions added as review comments

Suggestion breakdown:

  • 🔥 High confidence: 8
  • ⚠️ Medium confidence: 5
  • 💡 Low confidence: 0

The high-confidence changes have been automatically applied to maintain consistency with the QuantEcon Style Guide. Please review the other suggestions in the file comments.


Generated by Style Guide Checker Action

@mmcky
Copy link
Contributor

mmcky commented Aug 25, 2025

@copilot can we update the PR mode.

Is it possible to take the high-confidence suggestions and apply suggestions to the PR instead of trying to auto-commit to the PR. We should be transparent about what we propose to change and they should be merged by the reviewer.

…high-confidence changes

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 25, 2025

@copilot can we update the PR mode.

Is it possible to take the high-confidence suggestions and apply suggestions to the PR instead of trying to auto-commit to the PR. We should be transparent ...

Updated PR mode to use GitHub suggestions instead of auto-commits for high-confidence changes. Now in PR mode:

  • High-confidence suggestions: Use GitHub's suggestion feature (```suggestion blocks) for one-click application
  • Medium/low-confidence suggestions: Regular review comments
  • No auto-commits: All changes require explicit reviewer approval for transparency

Full mode still auto-commits high-confidence changes for scheduled reviews. Commit f992262 implements this change.

Copilot AI changed the title Implement AI-powered Style Guide Checker with Comment-Based Triggers Implement AI-powered Style Guide Checker with GitHub Suggestions Aug 25, 2025
@mmcky
Copy link
Contributor

mmcky commented Aug 25, 2025

@copilot I am testing this new action in QuantEcon/lecture-python-advanced.myst#250

and I am getting the following ERROR

2025-08-25 05:17:23,175 - ERROR - Failed to create PR review suggestions: Unprocessable Entity: 422 {"message": "Unprocessable Entity", "errors": ["Pull request review thread line must be part of the diff and Pull request review thread diff hunk can't be blank"], "documentation_url": "https://docs.github.com/rest/pulls/reviews#create-a-review-for-a-pull-request", "status": "422"}
``

I think the approach we have taken to add `suggestions` to the PR discussion thread is not working. 

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 25, 2025

@copilot I am testing this new action in QuantEcon/lecture-python-advanced.myst#250

and I am getting the following ERROR
...

Fixed the GitHub suggestions issue! The error occurred because the previous implementation tried to create review comments using absolute line numbers, but GitHub's suggestion API requires diff positions for lines that are actually part of the PR changes.

The new implementation (commit 843d7d1):

  1. Parses the PR diff to map line numbers to diff positions
  2. Creates GitHub suggestions only for lines that are part of the actual diff using proper diff positions
  3. Falls back to regular comments for suggestions on lines not changed in the PR
  4. Maintains the ```suggestion format for high-confidence changes that can be applied with one click

This should resolve the "Unprocessable Entity: 422" error you encountered in the lecture-python-advanced.myst PR.

Co-authored-by: mmcky <8263752+mmcky@users.noreply.github.com>
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style guide review completed. High-confidence suggestions are provided as GitHub suggestions that you can apply with one click.

import time
import numpy as np

def utility_function(c, alpha=0.5, beta=0.95):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (high confidence)

Use Unicode α instead of 'alpha' for better mathematical notation

Rule category: variable_naming

Suggested change
def utility_function(c, alpha=0.5, beta=0.95):
def utility_function(c, α=0.5, beta=0.95):

import time
import numpy as np

def utility_function(c, alpha=0.5, beta=0.95):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (high confidence)

Use Unicode β instead of 'beta' for better mathematical notation

Rule category: variable_naming

Suggested change
def utility_function(c, alpha=0.5, beta=0.95):
def utility_function(c, alpha=0.5, β=0.95):


Bad equation formatting:

$x_t = alpha * x_{t-1} + epsilon_t$

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (high confidence)

Use Unicode α instead of 'alpha' for better mathematical notation

Rule category: variable_naming

Suggested change
$x_t = alpha * x_{t-1} + epsilon_t$
$x_t = α * x_{t-1} + epsilon_t$


```python
# More bad Greek letter usage
def another_function(x, sigma=1.0, theta=0.5, rho=0.9):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (high confidence)

Use Unicode σ instead of 'sigma' for better mathematical notation

Rule category: variable_naming

Suggested change
def another_function(x, sigma=1.0, theta=0.5, rho=0.9):
def another_function(x, σ=1.0, theta=0.5, rho=0.9):


```python
# More bad Greek letter usage
def another_function(x, sigma=1.0, theta=0.5, rho=0.9):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (high confidence)

Use Unicode θ instead of 'theta' for better mathematical notation

Rule category: variable_naming

Suggested change
def another_function(x, sigma=1.0, theta=0.5, rho=0.9):
def another_function(x, sigma=1.0, θ=0.5, rho=0.9):

Should use proper Unicode and equation numbering.

```python
# More bad Greek letter usage

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (medium confidence)

Lecture titles should use title case (capitalize all words)

Suggested change:

# More Bad Greek Letter Usage

Rule category: titles_headings

return (c**(1-alpha) - 1) / (1-alpha) * beta

# Bad timing pattern
start_time = time.time()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (high confidence)

Use modern qe.Timer() context manager instead of manual timing

Rule category: performance_timing

Suggested change
start_time = time.time()
with qe.Timer():

# Bad timing pattern
start_time = time.time()
result = utility_function(1.0)
end_time = time.time()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (high confidence)

Use modern qe.Timer() context manager instead of manual timing

Rule category: performance_timing

Suggested change
end_time = time.time()
with qe.Timer():

"""Utility function with proper Unicode Greek letters."""
return (c**(1-α) - 1) / (1-α) * β

# Proper timing with qe.Timer()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (medium confidence)

Lecture titles should use title case (capitalize all words)

Suggested change:

# Proper Timing With Qe.timer()

Rule category: titles_headings

@@ -0,0 +1,11 @@
# Test file for exclusion

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style Guide Suggestion (medium confidence)

Lecture titles should use title case (capitalize all words)

Suggested change:

# Test File For Exclusion

Rule category: titles_headings

@github-actions
Copy link

Style Guide Suggestions

The following suggestions are for lines not changed in this PR:

README.md (line 1)

Lecture titles should use title case (capitalize all words)

Rule category: titles_headings

Current:

# meta

Suggested:

# Meta

@github-actions
Copy link

📝 Style Guide Review Summary

Files reviewed: 7
Total suggestions: 13

Suggestions provided:

  • 🔥 8 high-confidence suggestions (GitHub suggestions - click to apply)
  • ⚠️ 5 medium-confidence suggestions
  • 💡 0 low-confidence suggestions

Suggestion breakdown:

  • High confidence: Ready-to-apply suggestions using GitHub's suggestion feature
  • Medium confidence: Recommended changes that may need minor adjustments
  • Low confidence: Optional improvements for consideration

All suggestions are based on the QuantEcon Style Guide. High-confidence suggestions can be applied with a single click using GitHub's suggestion feature for transparency and reviewer control.


Generated by Style Guide Checker Action

@QuantEcon QuantEcon deleted a comment from github-actions bot Aug 25, 2025
@mmcky mmcky closed this Aug 26, 2025
@mmcky mmcky deleted the copilot/fix-204 branch August 26, 2025 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[styleguide] Build a GitHub action that can review a lecture for compliance with our style guide and suggest changes.

2 participants