Skip to content

Conversation

@lossyrob
Copy link

@lossyrob lossyrob commented Nov 20, 2025

Add reply_to_review_comment Tool

Summary

This PR adds a new MCP tool reply_to_review_comment that enables AI agents to reply directly to individual pull request review comment threads. This maintains conversation context at specific code locations, allowing agents to participate in threaded code review discussions just as human developers do.

Problem: Previously, AI agents could only post general PR comments, which separated responses from the code they referenced and made review conversations fragmented.

Solution: The new tool provides direct access to GitHub's review comment reply API, allowing agents to respond within existing comment threads at specific code locations.

Related Issue

Closes #1323

Implementation Details

Core Changes

New Tool: reply_to_review_comment in pkg/github/pullrequests.go

  • Uses GitHub REST API: POST /repos/{owner}/{repo}/pulls/{pull_number}/comments
  • Calls client.PullRequests.CreateCommentInReplyTo() from go-github v79
  • Parameters: owner, repo, pull_number, comment_id, body
  • Returns MinimalResponse with reply ID and URL

Toolset Registration: Added to repository_management toolset as a write tool

Testing: Comprehensive unit tests with 10 test cases covering:

  • Successful reply creation (HTTP 201)
  • Error scenarios (404, 403, 422)
  • Parameter validation
  • Mock HTTP client for REST API testing

Technical Decisions

  • REST API over GraphQL: go-github provides a clean CreateCommentInReplyTo method
  • Required pull_number: GitHub API requires PR number in the endpoint path
  • int64 comment IDs: Uses RequiredBigInt validation for proper type handling
  • Single-reply operations: Agents orchestrate batch operations by calling the tool multiple times
  • Minimal response format: Returns only ID and URL, consistent with other create/update tools

Testing

Manual Testing Performed

Tested the tool in VS Code with the GitHub MCP Server:

  1. Created a test PR with review comments
  2. Retrieved comment IDs using pull_request_read with get_review_comments method
  3. Used reply_to_review_comment to respond to review comments
  4. Verified replies appeared as threaded responses in GitHub UI at correct code locations
  5. Confirmed notifications were sent to reviewers

Result: Tool works as expected - replies appear correctly threaded in the GitHub UI.

Automated Tests

  • All unit tests pass (script/test)
  • Linting passes with 0 issues (script/lint)
  • Toolsnap validation confirms schema stability
  • Documentation generated successfully

Documentation

  • README.md updated with tool documentation (auto-generated via script/generate-docs)
  • Tool parameters clearly documented with types and descriptions
  • Error handling documented for common failure scenarios

Project Artifacts

This implementation followed a structured development process. Supporting documentation is available in my fork:

Example Usage

{
  "owner": "myorg",
  "repo": "myproject",
  "pull_number": 42,
  "comment_id": 12345,
  "body": "Good point! I've renamed the variable to `userSessionManager` in commit abc123."
}

Response:

{
  "id": "67890",
  "url": "https://github.com/myorg/myproject/pull/42#discussion_r67890"
}

Breaking Changes

None - this is a new tool addition that doesn't affect existing tools or APIs.


🐾 Generated with PAW

- Specification with user stories and requirements
- Spec research on GitHub API behavior and constraints
- Code research documenting implementation patterns
- Implementation plan with 4 phases
- Workflow context and prompt templates
…plan

[Reply To Review Comments] Planning: Add reply_to_review_comment tool
Implements the core MCP tool function that enables AI agents to reply
directly to pull request review comment threads. The tool uses the
GitHub REST API's CreateCommentInReplyTo method to create threaded
replies that maintain conversation context at specific code locations.

Key features:
- Required parameters: owner, repo, pull_number, comment_id, body
- Uses RequiredBigInt for comment_id (int64 type)
- Returns MinimalResponse with reply ID and URL
- Follows established error handling patterns
- Marked as write tool (ReadOnlyHint: false)

Part of Phase 1: Core Tool Implementation
…phase1

[Reply To Review Comments] Phase 1: Core Tool Implementation
- Add ReplyToReviewComment to AddWriteTools section in pkg/github/tools.go
- Position after AddCommentToPendingReview to group review-related write tools
- Tool is now discoverable through the MCP server's tool listing
- Uses REST client (getClient parameter) consistent with tool implementation

Phase 2 complete: All automated verification passed (build, lint).
Add comment explaining that the 'body' variable is intentionally shadowed
when reading the HTTP response body in the error path. This follows the
established pattern used throughout pullrequests.go (e.g., CreatePullRequest).
Replace shadowing pattern with clearer variable name 'responseBody' in
error handling path. While shadowing is used elsewhere in this file,
using distinct names improves code clarity for readers.
…phase2

[Reply To Review Comments] Phase 2: Toolset Integration
- Add unit tests with toolsnap validation in pkg/github/pullrequests_test.go
- Implement table-driven behavioral tests covering:
  * Successful reply creation (201 status)
  * Error scenarios: 404 Not Found, 403 Forbidden, 422 Unprocessable Entity
  * Parameter validation for all required fields (owner, repo, pull_number, comment_id, body)
  * Invalid comment_id type handling
- Generate toolsnap schema snapshot in pkg/github/__toolsnaps__/reply_to_review_comment.snap
- Add end-to-end test in e2e/e2e_test.go following established patterns:
  * Creates test repository, branch, commit, and pull request
  * Adds review comment via pending review workflow
  * Tests reply_to_review_comment tool with actual GitHub API
  * Verifies reply appears in review comments list with correct body text
  * Validates MinimalResponse structure (id and url fields)

All tests pass. Linting clean.
E2E tests have high maintenance costs and require PAT tokens with write access.
The comprehensive unit tests with mocked HTTP responses already provide
sufficient coverage of success and error scenarios.
…phase3

[Reply To Review Comments] Implementation Phase 3: Testing
- Generated README.md documentation for reply_to_review_comment tool
- Tool documented with all parameters (owner, repo, pull_number, comment_id, body)
- Verified all validation checks pass:
  * Linting: 0 issues
  * Full test suite: All tests passed
  * License compliance: No issues (no new dependencies)
- Updated ImplementationPlan.md with Phase 4 completion summary
- Ready for Implementation Review Agent to review and open Phase 4 PR
…phase4

[Reply To Review Comments] Implementation Phase 4: Documentation & Validation
- Created detailed Docs.md with architecture, design decisions, and usage
- Includes user guide with basic and advanced usage examples
- Documents error handling, edge cases, and limitations
- Provides comprehensive testing guide for manual verification
- Covers migration path and compatibility notes
- README.md already updated from Phase 4 auto-generation
…docs

[Reply To Review Comments] Documentation
Copilot AI review requested due to automatic review settings November 20, 2025 03:48
@lossyrob lossyrob requested a review from a team as a code owner November 20, 2025 03:48
Copilot finished reviewing on behalf of lossyrob November 20, 2025 03:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a new MCP tool reply_to_review_comment that enables AI agents to reply directly to individual pull request review comment threads, maintaining conversation context at specific code locations. This addresses a significant gap where agents previously could only post general PR comments, losing the threaded context of code review discussions.

Key changes:

  • New ReplyToReviewComment function in pkg/github/pullrequests.go using GitHub's REST API
  • Tool registration in the repository_management toolset as a write operation
  • Comprehensive unit tests with 10 test cases covering success and error scenarios
  • Toolsnap schema validation for API stability
  • Auto-generated README.md documentation

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pkg/github/pullrequests.go Implements the ReplyToReviewComment tool with proper parameter validation, error handling, and MinimalResponse return format following established patterns
pkg/github/tools.go Registers the new tool in the repository_management toolset's write tools section, positioned logically with other review-related tools
pkg/github/pullrequests_test.go Adds comprehensive table-driven tests with toolsnap validation, success case, and 9 error scenarios including parameter validation
pkg/github/__toolsnaps__/reply_to_review_comment.snap Generated JSON schema snapshot for tool validation and API stability tracking
README.md Auto-generated documentation for the new tool with clear parameter descriptions
.paw/work/reply-to-review-comments/* Development artifacts (specifications, implementation plans, research, manual testing guides) documenting the development process

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ability to reply to individual pull request review comments

1 participant