[Feature] Implement tool documentation validation

[dougborg]

Overview

Implement build-time validation to ensure tool documentation is complete and accurate.

This is Phase 1 of ADR 003.

Context

See ADR 003 for full context: docs/architecture/decisions/003-automated-tool-documentation.md

Implementation guide available: docs/contributing/tool-documentation-guide.md

Scope

Build validation tooling to catch documentation errors at development time.

Components to Implement

1. Docstring Parser (utils/docstring_parser.py)

Parse Google-style docstrings to extract:

• Summary line
• Args section → parameter descriptions
• Returns section
• Example JSON
• Workflow sections (How It Works, See Also)

Reference: Implementation guide Section 1 (lines 1-125)

2. Tool Metadata Extractor (utils/tool_metadata.py)

Extract tool information from function signatures:

• Function name and parameters
• Pydantic request/response models
• Risk level detection (checks for context.elicit() usage)
• Request model field information

Reference: Implementation guide Section 2 (lines 142-280)

3. Validation Script (scripts/validate_tool_docs.py)

Validate tools against documentation standards:

Validation Rules:

• ✅ All parameters (except context) have Args entries
• ✅ All Pydantic Field() have description parameter
• ✅ Tools using context.elicit() have 🔴 HIGH-RISK indicator
• ✅ Example JSON (if present) is valid and matches request schema
• ✅ Response model documented in Returns section

Exit behavior:

• Exit 0 if all validations pass
• Exit 1 with clear error messages if any fail

4. CLI Integration

Tasks

• Implement utils/docstring_parser.py
• Implement utils/tool_metadata.py
• Implement scripts/validate_tool_docs.py
• Add tests for validation logic
• Add poe task: uv run poe validate-docs
• Document usage in contributing guide
• Fix any existing tools that fail validation

Acceptance Criteria

• All 40+ tools pass validation
• Validation catches missing Args entries
• Validation detects missing Field descriptions
• Validation enforces 🔴 indicator on elicitation tools
• Clear, actionable error messages
• Fast execution (< 2 seconds for all tools)

Testing Plan

1. Create test tools with intentional documentation errors
2. Verify validation catches each error type
3. Verify error messages are clear and actionable
4. Test against actual production tools
5. Ensure no false positives

Dependencies

• Requires: [ADR] Document automated tool documentation strategy #85 (ADR 003) to be merged for context
• Blocks: [Feature] Implement tool documentation generation CLI #88 (pre-commit integration)

Related

• ADR 002: Documents Pydantic Field pattern we're validating
• ADR 003: Documents overall validation strategy (Phase 1)

Labels

enhancement