feat(skills): Establish skill prompt size limits with validation

[rjmurillo-bot]

Summary

Define maximum size limits for skill prompts and add automated validation.

Problem Statement

The pr-review skill prompt has grown to 500+ lines. Large prompts:

• Consume more tokens per invocation
• Are harder for agents to process
• Become difficult to maintain
• Increase likelihood of inconsistencies

Evidence: Session 109 export analysis (commit 9b693a5)

Requirements

Size Limits

• Define maximum prompt size (proposed: 200 lines)
• Define maximum config file size (proposed: 500 lines)
• Document exceptions process for justified overages

Validation

• Add lint check for skill prompt size
• Add pre-commit hook to flag oversized prompts
• Provide actionable guidance when limit exceeded

Acceptance Criteria

1. Size limits documented in skill development guide
2. Lint check validates prompt size
3. At least 1 oversized skill refactored to meet limit
4. Pre-commit provides actionable error message

Priority

P2 - Enhancement (prevents future bloat)

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Similar Issues

Possible Duplicates

• refactor(skill): Simplify pr-review skill prompt (500+ lines to structured config) #672

Related Issues

• refactor: decompose skills-github-cli memory into focused modules #239
• feat: enforce PR size limits to prevent tooling failures and improve review quality #468

🔗 Related PRs

#245 - docs(memory): add decomposition analysis for Issue #239 [merged]
#300 - docs(retrospective): autonomous PR monitoring session analysis [merged]
#301 - docs: autonomous PR monitoring prompt and retrospective [merged]
#315 - feat(skill): add ADR review skill [merged]
#401 - docs(retrospective): PR #395 Copilot SWE failure analysis [merged]

👤 Suggested Assignees

• rjmurillo-bot
• rjmurillo

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[github-actions]

Product Requirements Document

Important

This PRD was automatically generated based on issue triage escalation.
Complexity Score: 7/12 (Detailed)
Escalation Criteria: feature_request,multi_phase_work

About this PRD

This PRD was generated by the Explainer Agent after the issue was flagged
for detailed analysis. The document is designed to be self-contained - it can
be passed to a separate AI agent instance for implementation without additional
context.

PRD Depth Levels:

• Standard (4-6): Brief analysis, clear requirements
• Detailed (7-9): Research section, blocking questions
• Comprehensive (10-12): Phased implementation, risk analysis

---

Let me gather more context on skill files and existing validation scripts.

Now let me check the pr-review skill mentioned as the problematic 500+ line prompt.

Now I have enough context. Let me create the PRD for this issue.

I cannot create the file directly. Let me output the PRD content for the user.

---

PRD: Skill Prompt Size Limits with Validation

Executive Summary

The pr-comment-responder skill prompt has grown to 1551 lines (500+ referenced in issue). Large prompts consume more tokens per invocation, are harder for agents to process, and become difficult to maintain. This PRD defines maximum size limits for skill prompts and adds automated validation to prevent future bloat.

Scope	Status	Blocker
Define size limits	🟢 READY	None
Add lint check for prompt size	🟢 READY	None
Add pre-commit hook integration	🟢 READY	None
Refactor oversized skill	🟡 PARTIAL	Requires modular decomposition strategy
Document exceptions process	🟢 READY	None

Verdict: READY (core validation can proceed; refactoring follows)

Problem Statement

Current State

Asset	Lines	Tokens (est.)	Status
src/claude/pr-comment-responder.md	1551	~6,000+	OVERSIZED
src/claude/skillbook.md	457	~1,800	AT LIMIT
src/claude/implementer.md	~400	~1,600	OK
Average agent prompt	~200	~800	TARGET

Gap Analysis

1. No size limits defined: No documented maximum for skill prompts
2. No validation: Prompts can grow unbounded without warning
3. Token waste: Large prompts consume context budget on every invocation
4. Maintenance burden: 1500+ line files are difficult to review and update
5. Agent processing: Larger prompts increase likelihood of instruction conflicts

User Impact

• Agents: Consume more tokens per invocation (cost)
• Maintainers: Harder to review and update large prompts
• Contributors: Unclear guidance on prompt size expectations

Research Findings

Primary Sources

Session 109 Export Analysis (commit 9b693a5):

• Identified pr-review skill at 500+ lines
• Confirmed: Actual file is 1551 lines (verified via view tool)
• Confidence: CONFIRMED

Existing Validation Pattern (Validate-TokenBudget.ps1):

• Uses heuristic token estimation (~4 chars/token base)
• Applies safety margin (5% buffer)
• Provides actionable guidance on failure
• Confidence: CONFIRMED (reviewed source)

Existing Pre-commit Infrastructure:

• .githooks/pre-commit handles multiple validation types
• Uses PowerShell for validation scripts
• Supports BLOCKING and non-blocking checks
• Confidence: CONFIRMED (reviewed 900-line pre-commit hook)

Secondary Sources

ADR-005 (PowerShell-Only Scripting):

• All validation scripts must be PowerShell
• No bash scripts in .github/scripts/
• Confidence: CONFIRMED

Empirical Data

Prompt File	Line Count	Est. Tokens	Complexity
pr-comment-responder.md	1551	~6,200	HIGH
skillbook.md	457	~1,800	MEDIUM
implementer.md	~400	~1,600	MEDIUM
orchestrator.md	~350	~1,400	MEDIUM
analyst.md	~250	~1,000	LOW

Proposed Solution

Phase 1: Define Size Limits (Status: READY)

Estimated Effort: S
Blockers: None

Task	Description
1.1	Define maximum prompt size: 200 lines (proposed), 500 lines (hard limit)
1.2	Define maximum config file size: 500 lines
1.3	Document limits in .agents/steering/agent-prompts.md
1.4	Create exception request process via ADR

Phase 2: Add Validation Script (Status: READY)

Estimated Effort: M
Blockers: None

Task	Description
2.1	Create scripts/Validate-PromptSize.ps1
2.2	Implement line counting with threshold checks
2.3	Implement token estimation (reuse from Validate-TokenBudget.ps1)
2.4	Support -CI flag for blocking behavior
2.5	Support -StagedOnly flag for pre-commit
2.6	Provide actionable error messages

Phase 3: Pre-commit Integration (Status: READY)

Estimated Effort: S
Blockers: None

Task	Description
3.1	Add prompt size check section to .githooks/pre-commit
3.2	Configure as WARNING (non-blocking) initially
3.3	Provide path to exception documentation

Phase 4: Refactor Oversized Skill (Status: PARTIAL)

Estimated Effort: L
Blockers: Requires decomposition strategy decision

Task	Description
4.1	Analyze pr-comment-responder.md for modular decomposition
4.2	Identify extractable sections (e.g., GitHub API patterns, bot handling)
4.3	Create supporting skill files in .agents/skills/
4.4	Update main prompt to reference extracted skills
4.5	Verify prompt size under limit after refactoring

Functional Requirements

FR-1: Size Limit Thresholds

Priority: P0

ID	Requirement	Acceptance Criteria
FR-1.1	Soft limit of 200 lines for agent prompts	Validation warns at 200+ lines
FR-1.2	Hard limit of 500 lines for agent prompts	Validation blocks at 500+ lines in CI
FR-1.3	Hard limit of 500 lines for config files	Validation blocks config files at 500+ lines
FR-1.4	Token estimation with 2000 token soft limit	Warns when estimated tokens exceed 2000

FR-2: Validation Script

Priority: P0

ID	Requirement	Acceptance Criteria
FR-2.1	Script validates files matching src/claude/*.md	Glob pattern correctly targets agent prompts
FR-2.2	Script validates files matching src/vs-code-agents/*.md	Both platform directories covered
FR-2.3	Script validates files matching src/copilot-cli/*.md	All three platforms validated
FR-2.4	Script outputs line count, estimated tokens, and status	Output shows all three metrics
FR-2.5	Script exits 0 on pass, 1 on CI failure	Exit codes match pre-commit convention
FR-2.6	Script provides actionable guidance on failure	Error message includes remediation steps

FR-3: Pre-commit Hook

Priority: P1

ID	Requirement	Acceptance Criteria
FR-3.1	Hook runs validation on staged prompt files	Only staged files validated (efficiency)
FR-3.2	Hook displays WARNING for soft limit violations	Yellow output, non-blocking
FR-3.3	Hook provides link to exception documentation	Error message includes doc path

FR-4: Documentation

Priority: P1

ID	Requirement	Acceptance Criteria
FR-4.1	Size limits documented in agent-prompts.md	Steering file updated with limits
FR-4.2	Exception process documented	Clear process for requesting exceptions
FR-4.3	Refactoring guidance provided	Patterns for decomposing large prompts

Non-Functional Requirements

Consistency

• Validation script follows existing patterns from Validate-TokenBudget.ps1
• Pre-commit integration follows existing hook structure
• Error messages follow repository style guide (no emojis, text indicators)

Testability

• Validation script has Pester tests in scripts/tests/
• Tests cover: under limit, at limit, over soft limit, over hard limit
• Tests verify exit codes and output format

Documentation

• Update .github/instructions/agent-prompts.instructions.md
• Update .agents/steering/agent-prompts.md (currently placeholder)
• Reference in .agents/skills/README.md

Technical Design

Validate-PromptSize.ps1 Configuration

# Default thresholds (overridable via parameters)
$SoftLimitLines = 200
$HardLimitLines = 500
$SoftLimitTokens = 2000
$HardLimitTokens = 4000

# File patterns to validate
$PromptPatterns = @(
    "src/claude/*.md",
    "src/vs-code-agents/*.md",
    "src/copilot-cli/*.md"
)

# Exclusions (files that have approved exceptions)
$Exclusions = @(
    # Add files with approved ADRs here
)

Script Output Format

Prompt Size Validation
  Checking: src/claude/pr-comment-responder.md
    Lines: 1551 (OVER hard limit: 500)
    Tokens: ~6200 (OVER hard limit: 4000)
    Status: [FAIL]
  
  Checking: src/claude/skillbook.md
    Lines: 457 (OK, under limit: 500)
    Tokens: ~1800 (OK, under limit: 4000)
    Status: [PASS]

Summary:
  Checked: 21 files
  Passed: 20
  Soft limit warnings: 0
  Hard limit failures: 1

[BLOCKED] 1 file(s) exceed hard limit.
  Action: Refactor or request exception via ADR.
  See: .agents/steering/agent-prompts.md#size-limits

Pre-commit Hook Addition

#
# Prompt Size Validation (Non-blocking warning, BLOCKING in CI)
#
# Validates agent prompt size limits (ADR-TBD).
# Warns at 200 lines, blocks at 500 lines in CI.
#
# Security: Checks for symlinks to prevent TOCTOU attacks (MEDIUM-002)
# Related: Issue #TBD
#
PROMPT_SIZE_SCRIPT="$REPO_ROOT/scripts/Validate-PromptSize.ps1"

# Check if any agent prompt files are staged
STAGED_PROMPT_FILES=$(echo "$STAGED_FILES" | grep -E '^src/(claude|vs-code-agents|copilot-cli)/.*\.md$' || true)

if [ -n "$STAGED_PROMPT_FILES" ]; then
    echo_info "Checking prompt size limits..."
    
    # MEDIUM-002: Reject symlinks for security
    if [ -L "$PROMPT_SIZE_SCRIPT" ]; then
        echo_warning "Skipping prompt size validation: script path is a symlink"
    elif [ -f "$PROMPT_SIZE_SCRIPT" ]; then
        if command -v pwsh &> /dev/null; then
            # Run validation on staged files only (non-blocking for now)
            if ! pwsh -NoProfile -ExecutionPolicy Bypass -File "$PROMPT_SIZE_SCRIPT" -StagedOnly 2>&1; then
                echo_warning "Prompt size validation found issues (see above)."
                echo_info "  Consider refactoring large prompts or requesting exception."
                echo_info "  See: .agents/steering/agent-prompts.md#size-limits"
                # Non-blocking: just warn, don't fail the commit
            else
                echo_success "Prompt size validation: PASS"
            fi
        else
            echo_warning "PowerShell not available. Skipping prompt size validation."
        fi
    else
        echo_info "Prompt size validation script not found. Skipping."
    fi
fi

Implementation Plan

Prerequisites

• PowerShell 7+ available
• Access to modify .githooks/pre-commit
• Access to create/modify scripts in scripts/

Phase 1: Create Validation Script (Day 1)

1. Create scripts/Validate-PromptSize.ps1 with:

   • Line counting logic
   • Token estimation (borrowed from Validate-TokenBudget.ps1)
   • Threshold configuration
   • -StagedOnly support for pre-commit
   • -CI support for blocking behavior
   • Actionable error messages

2. Create scripts/tests/Validate-PromptSize.Tests.ps1 with:

   • Test cases for under/at/over limits
   • Test cases for exit codes
   • Mock file generation for testing

Phase 2: Pre-commit Integration (Day 1)

1. Add prompt size check section to .githooks/pre-commit
2. Configure as non-blocking (WARNING only)
3. Verify hook triggers on staged prompt files

Phase 3: Documentation (Day 2)

1. Update .agents/steering/agent-prompts.md:

   • Add Size Limits section with thresholds
   • Add Exception Process section
   • Add Refactoring Guidance section

2. Update .github/instructions/agent-prompts.instructions.md:

   • Reference new size limits

Phase 4: Refactor pr-comment-responder.md (Day 3-5)

1. Analyze sections for extraction:

   • GitHub API patterns -> .agents/skills/github-api-patterns.md
   • Bot-specific handling -> .agents/skills/bot-handling.md
   • Verification gates -> .agents/skills/pr-verification-gates.md

2. Create extracted skill files

3. Update main prompt to reference skills

4. Verify final size under 500 lines

Decision Tree

Is validation script created?
├── NO → Phase 1: Create script
└── YES → Is pre-commit integrated?
          ├── NO → Phase 2: Pre-commit integration
          └── YES → Is documentation complete?
                    ├── NO → Phase 3: Documentation
                    └── YES → Is any file over limit?
                              ├── NO → [COMPLETE]
                              └── YES → Phase 4: Refactor

Risks and Mitigations

Risk	Likelihood	Impact	Mitigation
False positives from token estimation	Medium	Low	Use conservative thresholds; token limit is secondary to line limit
Refactoring breaks agent functionality	Medium	High	Incremental extraction with testing after each change
Exception process abused	Low	Medium	Require ADR with justification; periodic review
Pre-commit adds latency	Low	Low	Only check staged files; PowerShell startup is minimal

Success Metrics

Metric	Target
Validation script created and passing tests	100%
Pre-commit hook integrated	Complete
Documentation updated	Complete
At least 1 oversized file refactored under limit	pr-comment-responder.md < 500 lines
All new prompts pass validation	100% (enforced by pre-commit)

Blocking Questions

None. This PRD can proceed to implementation.

Open Questions (Non-Blocking)

1. Q: Should token estimation use tiktoken library instead of heuristics?

   • Hypothesis: Heuristics are sufficient for line-based limits; token limit is secondary
   • Status: Validated by existing Validate-TokenBudget.ps1 approach

2. Q: What is the right soft limit for lines?

   • Hypothesis: 200 lines balances readability with functionality
   • Status: Can be adjusted after initial deployment based on feedback

3. Q: Should extracted skills be in .agents/skills/ or .serena/memories/?

   • Hypothesis: .agents/skills/ for human-maintained content; .serena/memories/ for learned patterns
   • Status: Follow existing pattern from linting.md, documentation.md

Appendices

Appendix A: Research Sources

1. Session 109 export analysis (commit 9b693a5)
2. scripts/Validate-TokenBudget.ps1 - existing token validation pattern
3. .githooks/pre-commit - existing hook infrastructure
4. src/claude/pr-comment-responder.md - example of oversized prompt
5. .agents/steering/agent-prompts.md - placeholder steering file

Appendix B: File Size Analysis

File	Lines	Est. Tokens	Notes
pr-comment-responder.md	1551	~6,200	Needs refactoring
skillbook.md	457	~1,800	At edge of limit
orchestrator.md	~350	~1,400	OK
implementer.md	~400	~1,600	OK
analyst.md	~250	~1,000	OK
architect.md	~200	~800	Good example

Appendix C: Related Issues/PRs

• This issue: feat(skills): Establish skill prompt size limits with validation
• Session 109: Original analysis identifying the problem
• ADR-005: PowerShell-only scripting (validation script constraint)

---

_{Generated by AI PRD Generation | View Workflow}

[github-actions]

AI Triage Summary

Note

This issue has been automatically triaged by AI agents

What is AI Triage?

This issue was analyzed by AI agents:

• Analyst Agent: Categorizes the issue and suggests appropriate labels
• Roadmap Agent: Aligns the issue with project milestones and priorities
• Explainer Agent (if escalated): Generates comprehensive PRD

Triage Results

Property	Value
Category	enhancement
Labels	["enhancement","area-prompts","area-infrastructure"]
Priority	P2
Milestone	v1.1
PRD Escalation	Generated (see below)

Categorization Analysis

```json
{
  "labels": ["enhancement", "area-prompts", "area-infrastructure"],
  "category": "enhancement",
  "confidence": 0.95,
  "reasoning": "Issue proposes new validation features and size limits for skill prompts, not a bug fix"
}

</details>

<details>
<summary>Roadmap Alignment</summary>

```json
```json
{
  "milestone": "v1.1",
  "priority": "P2",
  "epic_alignment": "",
  "confidence": 0.70,
  "reasoning": "Skill prompt size limits is a maintenance enhancement that aligns with v1.1 maintainability focus but does not match existing epics; P2 per issue labels and problem statement.",
  "escalate_to_prd": true,
  "escalation_criteria": ["feature_request", "multi_phase_work"],
  "complexity_score": 7
}

</details>

---

<sub>Powered by [AI Issue Triage](https://github.com/rjmurillo/ai-agents) workflow</sub>