Skip to content

fix(agents): Add PR number extraction from natural language and URLs #829

@rjmurillo

Description

@rjmurillo

Agent Context Inference Gap: PR Numbers Not Extracted from Initial Prompts

Problem

AI agents prompt users for information already provided in the initial request, violating autonomy principles.

Example:

User: "Review PR 806 comments... https://github.com/rjmurillo/ai-agents/pull/806"
Agent: "What PR number would you like me to review?"

Impact

  • User Friction: Redundant clarification questions
  • Autonomy Violation: Agents should infer discoverable context
  • Wasted Tokens: Unnecessary back-and-forth
  • Poor UX: "The whole point of this project is to have agents be independent and self-sufficient"

Root Cause

Three-Layer Problem:

  1. Agent Definitions: No context inference guidance in pr-comment-responder.md (1,551 lines, zero extraction logic)
  2. Missing Utilities: No shared GitHub context extraction patterns (URLs, text patterns)
  3. Global Guidance: AGENTS.md lacks "extract before prompt" principle

Acceptance Criteria

Recommended Solution

Phase 1: Quick Fix (P0, 2 hours)

Add "Phase -1: Context Inference" to pr-comment-responder skill:

# Extract from text patterns
if ($userPrompt -match 'PR #?(\d+)|pull request #?(\d+)|#(\d+)') {
    $prNumber = $matches[1] -or $matches[2] -or $matches[3]
}

# Extract from GitHub URL
if ($userPrompt -match 'github\.com/([^/]+)/([^/]+)/pull/(\d+)') {
    $owner = $matches[1]
    $repo = $matches[2]
    $prNumber = $matches[3]
}

# Error if missing during autonomous execution
if (-not $prNumber -and $autonomous) {
    throw "Cannot extract PR number from prompt. Provide explicit number."
}

Phase 2: Shared Utility (P1, 4 hours)

Create .claude/skills/github/scripts/utils/Extract-GitHubContext.ps1:

  • Reusable across all agents
  • Handles PR/issue text and URL patterns
  • Repository extraction from URLs
  • Includes Pester test coverage

Phase 3: Global Guidance (P1, 1 hour)

Update AGENTS.md with "Context Inference Requirements":

  • Pattern matching table for GitHub URLs
  • "Inference Before Clarification" rule
  • Utility integration examples

Related

  • Autonomous Execution Guardrails memory: Stricter protocols during unattended execution
  • Agent Workflow Scope Discipline memory: Infer discoverable context

Evidence

Total Effort

  • P0: 2 hours
  • P1: 5 hours
  • Total: ~7 hours

Metadata

Metadata

Labels

agent-memoryContext persistence agentagent-orchestratorTask coordination agentagent-qaTesting and verification agentarea-promptsAgent prompts and templatesarea-skillsSkills documentation and patternsarea-workflowsGitHub Actions workflowsautomationAutomated workflows and processesbugSomething isn't workingenhancementNew feature or requestpriority:P0Critical: Blocks core functionality, security vulnerability, or data losspriority:P1Important: Affects user experience significantly, high business value

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions