Skip to content

feat(validation): add agent registry parser and validator#1333

Merged
rjmurillo-bot merged 2 commits intomainfrom
feat/1332-autonomous
Feb 27, 2026
Merged

feat(validation): add agent registry parser and validator#1333
rjmurillo-bot merged 2 commits intomainfrom
feat/1332-autonomous

Conversation

@rjmurillo-bot
Copy link
Collaborator

@rjmurillo-bot rjmurillo-bot commented Feb 27, 2026

Summary

  • Add agent registry parser and validator (scripts/validation/agent_registry.py) that reads agent definitions from YAML frontmatter in the agent markdown directory and checks them against the canonical catalog table
  • Detects model mismatches, missing required fields, invalid model values, duplicate agent names, and drift between agent files and the catalog
  • Add 18 tests (unit + integration) in tests/test_agent_registry.py

Fixes #1332

Test plan

  • 18 pytest tests pass (15 unit, 3 integration)
  • Ruff lint passes
  • Full test suite shows no regressions (4301 passed)
  • CLI runs with --json flag and produces structured output
  • Integration tests validate against real agent definition files

Parse agent definitions from src/claude/*.md YAML frontmatter and
validate against the canonical AGENTS.md catalog. Detects model
mismatches, missing required fields, duplicate names, and drift
between agent files and the catalog table.

Includes 18 tests (unit + integration against real agent files).

Fixes #1332

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@chatgpt-codex-connector
Copy link

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@github-actions github-actions bot added enhancement New feature or request automation Automated workflows and processes labels Feb 27, 2026
@rjmurillo-bot rjmurillo-bot enabled auto-merge (squash) February 27, 2026 09:21
@github-actions
Copy link
Contributor

PR Validation Report

Caution

Status: FAIL

Description Validation

Check Status
Description matches diff FAIL

QA Validation

Check Status
Code changes detected True
QA report exists false

⚠️ Blocking Issues

  • PR description does not match actual changes

⚡ Warnings

  • QA report not found for code changes (recommended before merge)

Powered by PR Validation workflow

@github-actions
Copy link
Contributor

github-actions bot commented Feb 27, 2026

✅ Pass: Memory Validation

No memories with citations found.


📊 Validation Details
  • Total memories checked: 0
  • Valid: 0
  • Stale: 0

@coderabbitai coderabbitai bot requested a review from rjmurillo February 27, 2026 09:22
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a valuable validation script for agent definitions, along with a comprehensive test suite. The code is well-structured and the validation logic appears correct. My main feedback focuses on improving the script's robustness by handling file system errors more gracefully, in line with established patterns for diagnosability and partial processing. The suggested changes allow the script to report all issues in a single execution, even when encountering unreadable files, and to fail clearly when critical components are missing.

@github-actions
Copy link
Contributor

Spec-to-Implementation Validation

Tip

Final Verdict: PASS

What is Spec Validation?

This validation ensures your implementation matches the specifications:

  • Requirements Traceability: Verifies PR changes map to spec requirements
  • Implementation Completeness: Checks all requirements are addressed

Validation Summary

Check Verdict Status
Requirements Traceability PARTIAL ⚠️
Implementation Completeness PASS

Spec References

Type References
Specs None
Issues 1332
Requirements Traceability Details

Based on the issue description provided in the task context and the implementation files, I can now complete the requirements traceability analysis.

Requirements Coverage Matrix

Requirement Description Status Evidence
AC-1 Parse agent definitions from src/claude/*.md COVERED parse_agent_file() at lines 71-96, parse_agent_files() at lines 99-111; tests in TestParseAgentFile and TestParseAgentFiles
AC-2 Validate parsed agents against AGENT-SYSTEM.md PARTIAL validate() at lines 129-193 validates against AGENTS.md catalog table, but the spec says "AGENT-SYSTEM.md"

Additional Implemented Functionality

The implementation exceeds the spec by including validation checks not explicitly required:

Feature Implementation Evidence
Model mismatch detection validate() lines 171-177 Test at line 236-243
Missing required fields validate() lines 156-161 Test at line 254-261
Invalid model values validate() lines 164-168 Test at line 245-252
Duplicate agent names validate() lines 147-153 Test at line 263-271
Catalog drift detection validate() lines 179-191 Tests at lines 273-287
JSON CLI output main() lines 232-246 PR description confirms CLI tested

Summary

  • Total Requirements: 2
  • Covered: 1 (50%)
  • Partially Covered: 1 (50%)
  • Not Covered: 0 (0%)

Gaps

  1. AC-2 document mismatch: The acceptance criteria specifies "Validate parsed agents against AGENT-SYSTEM.md" but implementation validates against "AGENTS.md". The --catalog argument defaults to AGENTS.md (line 210). This may be intentional if the agent catalog table was moved from AGENT-SYSTEM.md to AGENTS.md, but represents a spec deviation.

  2. Missing dependency verification: Issue AO-002b: Implement agent registry parser and validator #1332 lists dependency on AO-002: Define Agent Registry Schema #592 (schema definition) but there is no explicit schema JSON validation. The validator uses hardcoded field and model lists instead.

[!WARNING]
VERDICT: PARTIAL
AC-1 is fully covered with parsing implementation and 18 tests. AC-2 has a document name discrepancy: spec says "AGENT-SYSTEM.md" but implementation uses "AGENTS.md". Verify this is an intentional change or update the spec.

Implementation Completeness Details

Now I have all the context needed to evaluate the implementation against the acceptance criteria.

Acceptance Criteria Checklist

  • Criterion 1: Parse agent definitions from src/claude/*.md - SATISFIED

    • Evidence: scripts/validation/agent_registry.py:71-96 (parse_agent_file) and scripts/validation/agent_registry.py:99-111 (parse_agent_files) parse YAML frontmatter from all .md files in src/claude/
    • Evidence: tests/test_agent_registry.py:55-119 (TestParseAgentFile class with 4 test cases)
    • Evidence: tests/test_agent_registry.py:127-177 (TestParseAgentFiles class with 2 test cases)
    • Evidence: tests/test_agent_registry.py:298-304 (integration test parses real src/claude/ files, verifies orchestrator/analyst/implementer)
  • Criterion 2: Validate parsed agents against AGENT-SYSTEM.md - SATISFIED

    • Note: Implementation validates against AGENTS.md (which contains the canonical Agent Catalog table), not AGENT-SYSTEM.md. The AGENTS.md file is the correct source per repository memory: "Canonical agent catalog is defined in AGENTS.md"
    • Evidence: scripts/validation/agent_registry.py:114-126 (parse_catalog) extracts agent name/model from AGENTS.md table
    • Evidence: scripts/validation/agent_registry.py:129-193 (validate) performs validation checks:
      • Required fields (name, description, model) at lines 156-161
      • Valid model values (opus, sonnet, haiku) at lines 163-168
      • Model match against catalog at lines 171-177
      • Missing catalog agents at lines 180-184
      • Extra agents not in catalog at lines 187-191
      • Duplicate detection at lines 147-153
    • Evidence: tests/test_agent_registry.py:226-287 (TestValidate class with 8 test cases)
    • Evidence: tests/test_agent_registry.py:312-324 (integration test validates real agents against real catalog)

Missing Functionality

None identified. Both acceptance criteria are fully satisfied.

Edge Cases Not Covered

  1. Malformed YAML frontmatter (partial parsing) - handled by returning None
  2. Files with non-UTF-8 encoding - not handled (assumes UTF-8)
  3. Empty agent directory - handled (returns empty list)
  4. Catalog with duplicate entries - not explicitly checked (but real AGENTS.md has no duplicates)

Implementation Quality

  • Completeness: 100% of acceptance criteria satisfied
  • Quality: Implementation follows ADR-035 exit codes, reuses existing frontmatter parsing utilities, has 18 tests covering unit and integration scenarios
  • Additional Features: CLI with --json output, drift detection (agents in files but not catalog and vice versa)

[!TIP]
VERDICT: PASS
Implementation fully satisfies both acceptance criteria. Parser handles src/claude/*.md files with YAML frontmatter extraction. Validator checks agents against AGENTS.md catalog (the canonical source) with model matching, required field validation, duplicate detection, and drift warnings. Test coverage includes 18 tests with integration tests against real repository files.


Run Details
Property Value
Run ID 22480347070
Triggered by pull_request on 1333/merge

Powered by AI Spec Validator workflow

@github-actions
Copy link
Contributor

github-actions bot commented Feb 27, 2026

AI Quality Gate Review

Warning

⚠️ Final Verdict: WARN

Walkthrough

This PR was reviewed by six AI agents in parallel, analyzing different aspects of the changes:

  • Security Agent: Scans for vulnerabilities, secrets exposure, and security anti-patterns
  • QA Agent: Evaluates test coverage, error handling, and code quality
  • Analyst Agent: Assesses code quality, impact analysis, and maintainability
  • Architect Agent: Reviews design patterns, system boundaries, and architectural concerns
  • DevOps Agent: Evaluates CI/CD, build pipelines, and infrastructure changes
  • Roadmap Agent: Assesses strategic alignment, feature scope, and user value

Review Summary

Agent Verdict Category Status
Security PASS N/A
QA PASS N/A
Analyst PASS N/A
Architect WARN N/A ⚠️
DevOps PASS N/A
Roadmap PASS N/A

💡 Quick Access: Click on individual agent jobs (e.g., "🔒 security Review", "🧪 qa Review") in the workflow run to see detailed findings and step summaries.

Security Review Details

Security Review: PR #1333

PR Category: CODE (.py files)

Analysis

Files reviewed:

  • scripts/validation/agent_registry.py (275 lines)
  • tests/test_agent_registry.py (343 lines)

Findings

Severity Category Finding Location CWE
Low Input Validation sys.path.insert(0, ...) modifies import path at runtime agent_registry.py:22 CWE-426

Details:

  1. Path manipulation (Low): Line 22 uses sys.path.insert(0, ...) with a resolved absolute path derived from __file__. The path is constructed using Path(__file__).resolve().parents[2], which is deterministic and not user-controlled. This is a common pattern for intra-project imports and poses minimal risk.

Security-positive patterns observed:

  • File paths validated via is_dir() and is_file() checks before use (lines 225-231)
  • OSError exceptions caught when reading files (lines 114, 237)
  • No shell command execution or subprocess usage
  • No user input flows to file system operations in production paths
  • No secrets, credentials, or API keys present
  • No eval(), exec(), or dynamic code execution
  • Test file uses tmp_path fixture for isolated temporary directories
  • Exit codes follow ADR-035 conventions

Recommendations

  1. Optional improvement: Consider using relative imports or package-level __init__.py to avoid runtime sys.path modification. This is a code quality suggestion, not a security requirement.

Verdict

VERDICT: PASS
MESSAGE: No security vulnerabilities detected. Code follows secure patterns for file handling with proper path validation and error handling.
QA Review Details

Now I'll verify test execution and check for any code quality concerns.

Based on my file review, I have sufficient evidence to produce the QA verdict.


QA Review: PR #1333

PR Type Classification

PR TYPE: CODE
FILES:
  CODE: scripts/validation/agent_registry.py, tests/test_agent_registry.py

Test Coverage Assessment

Area Status Evidence Files Checked
Unit tests Adequate 15 unit tests covering all 4 public functions tests/test_agent_registry.py:55-306
Edge cases Covered Missing frontmatter, missing name, unreadable files, empty catalog TestParseAgentFile, TestParseAgentFiles
Error paths Tested OSError handling (line 169-183), duplicate detection (line 281-289) test_unreadable_file_collected_as_error
Assertions Present 40+ assertions across 18 tests tests/test_agent_registry.py

Quality Concerns

Severity Issue Location Evidence Required Fix
LOW sys.path manipulation agent_registry.py:22 sys.path.insert(0, ...) Consider package import in future

Code Quality Metrics

Metric Value Threshold Status
Longest function 68 lines (main) <100 lines [PASS]
Cyclomatic complexity ≤6 (validate) ≤10 [PASS]
Code duplication None detected <3 blocks [PASS]
Error handling Try/except with stderr Present [PASS]

Test-Implementation Alignment

Function Test Coverage Status
parse_agent_file 4 tests (valid, missing FM, missing name, optional hint) [PASS]
parse_agent_files 3 tests (exclusions, unreadable, sorted) [PASS]
parse_catalog 2 tests (table extraction, non-table) [PASS]
validate 7 tests (valid, mismatch, invalid, missing field, duplicates, warnings) [PASS]
main (CLI) Integration tests + PR description confirms manual test [PASS]

Error Handling Verification

Operation Error Handling Evidence Status
File read OSError caught line 114-115 [PASS]
Catalog read OSError caught lines 235-239 [PASS]
Missing directory Exit code 2 lines 225-227 [PASS]
Missing catalog Exit code 2 lines 229-231 [PASS]
Validation failures Exit code 1 line 270 [PASS]

Regression Risk Assessment

  • Risk Level: Low
  • Affected Components: New standalone validation script (no existing code modified)
  • Breaking Changes: None (additive feature)
  • Required Testing: Covered by integration tests against real src/claude/ files

VERDICT: PASS
MESSAGE: Comprehensive test coverage (18 tests, 40+ assertions) with proper error handling and ADR-035 compliant exit codes.

PR TYPE: CODE

EVIDENCE:
- Tests found: 18 tests for 5 functions (4 parsing + 1 CLI)
- Test execution: PASS (4572 passed, 3 skipped, 5 warnings)
- Edge cases: Covered (missing FM, missing name, unreadable files, empty catalog, duplicates)
- Error handling: Tested for OSError, missing paths, validation failures
- Blocking issues: 0
Analyst Review Details

Let me check the local repository for the PR files since the API call indicates a different repo owner.

Let me check the existing validation scripts for pattern consistency and examine the directory structure.

Now I have sufficient context to complete the analysis. Let me compile my findings.


Analysis: PR #1333 - Agent Registry Parser and Validator

Code Quality Score

Criterion Score (1-5) Notes
Readability 5 Clear docstrings, well-named functions, logical flow
Maintainability 4 Good separation of concerns; minor duplication opportunity
Consistency 4 Follows existing validation patterns; uses ok instead of is_valid
Simplicity 5 Minimal dependencies, focused scope, no over-engineering

Overall: 4.5/5

Impact Assessment

  • Scope: Isolated - new validation script, no changes to existing code
  • Risk Level: Low - read-only validation, follows established patterns
  • Affected Components: scripts/validation/, tests/

Findings

Priority Category Finding Location
Low Consistency ValidationResult.ok property differs from shared types.ValidationResult.is_valid agent_registry.py:66-68
Low Reuse Defines local ValidationResult instead of extending types.ValidationResult agent_registry.py:59-68
Low Robustness sys.path.insert for build imports could fail if build/ structure changes agent_registry.py:22-23
Low Test Integration test asserts len(agents) >= 15 - fragile if agents removed test_agent_registry.py:318

Detailed Analysis

Code Quality

Positive patterns observed:

  1. ADR-035 compliance: Exit codes follow standard (0=success, 1=logic error, 2=config error)
  2. Frozen dataclass: AgentDefinition is immutable, preventing mutation bugs
  3. Type annotations: Full type hints throughout
  4. Error handling: OSError caught for file operations with structured error collection
  5. CLI design: Supports both human-readable and JSON output

Minor improvements available:

  1. The local ValidationResult class uses ok property while scripts/validation/types.py uses is_valid. For consistency, consider extending the shared type or aliasing the property.

  2. The sys.path.insert pattern for importing from build/ is functional but creates an implicit dependency. A relative import or package installation would be more robust.

Test Quality

Coverage: 18 tests (15 unit, 3 integration) covering:

  • Frontmatter parsing edge cases
  • File exclusion patterns
  • Catalog table extraction
  • Validation rules (duplicates, missing fields, model mismatches)

Integration tests: Wisely marked with skipif for CI environments where src/claude/ may not exist.

Minor observation: The assertion len(agents) >= 15 creates coupling to current agent count. Consider using >= 10 or documenting the threshold rationale.

Architectural Alignment

The implementation follows established patterns in scripts/validation/:

Pattern skill_frontmatter.py consistency.py agent_registry.py
Dataclass results Yes Yes Yes
ADR-035 exit codes Yes Yes Yes
JSON output flag No Yes Yes
argparse CLI Yes Yes Yes

The new script aligns with existing validation infrastructure.

Dependencies

  • Internal: Reuses generate_agents_common.py from build/ for frontmatter parsing
  • External: None (stdlib only)
  • Licensing: N/A

The dependency on build/generate_agents_common.py is appropriate since that module contains the canonical frontmatter parser. This avoids code duplication.

Recommendations

  1. Optional: Align property naming with types.ValidationResult.is_valid or add a comment explaining the intentional difference.

  2. Optional: Consider adding the script to a CI workflow (e.g., pytest.yml) for automated drift detection between agent files and catalog.

  3. Future: Document the validation script in .agents/governance/ if it becomes part of standard pre-commit or CI gates.

Verdict

VERDICT: PASS
MESSAGE: Well-structured validation script following established patterns. 18 tests provide good coverage. Minor consistency opportunities do not block merge.
Architect Review Details

Now I have sufficient context to complete the architectural review.


Architect Design Review: PR #1333

Design Quality Assessment

Aspect Rating (1-5) Notes
Pattern Adherence 4 Clean dataclass usage, single responsibility functions, immutable data
Boundary Respect 5 Lives in scripts/validation/, proper separation from build utilities
Coupling 3 Imports from build/ via sys.path manipulation is a coupling concern
Cohesion 5 Each function has a single clear purpose
Extensibility 4 Validation checks are modular, easy to add new checks

Overall Design Score: 4/5

Architectural Concerns

Severity Concern Location Recommendation
Medium Duplicate ValidationResult dataclass agent_registry.py:59-68 Reuse scripts.validation.types.ValidationResult instead of defining a new class
Medium sys.path.insert coupling agent_registry.py:22-23 Consider creating a shared utilities package or moving parse_simple_frontmatter to scripts/
Low Property naming inconsistency agent_registry.py:66 uses .ok, types.py:28 uses .is_valid Align naming conventions

Breaking Change Assessment

  • Breaking Changes: No
  • Impact Scope: None
  • Migration Required: No
  • Migration Path: N/A

Technical Debt Analysis

  • Debt Added: Low
  • Debt Reduced: Low
  • Net Impact: Neutral (minor duplication introduced, but validates against drift)

ADR Assessment

  • ADR Required: No
  • Decisions Identified: None significant (follows existing validation script patterns)
  • Existing ADR: ADR-042 (Python migration), ADR-035 (exit codes) - both followed correctly
  • Recommendation: N/A

Recommendations

  1. Consolidate ValidationResult: Replace the local ValidationResult class with from scripts.validation.types import ValidationResult and rename .ok references to .is_valid for consistency.

  2. Refactor build utility import: The sys.path.insert pattern creates tight coupling between scripts/validation/ and build/. Consider one of:

    • Move parse_simple_frontmatter and read_yaml_frontmatter to a shared scripts/utils/frontmatter.py
    • Add build/ as an installable package in pyproject.toml
    • Accept current approach as pragmatic given existing pattern usage
  3. Test coverage: Integration tests correctly handle pre-existing drift with non-assertion approach. This is appropriate.

Structural Assessment

The file placement in scripts/validation/ follows established patterns. The module fits alongside existing validators (consistency.py, skill_frontmatter.py, etc.). Directory structure is appropriate.

Verdict

VERDICT: WARN
MESSAGE: Minor architectural issues - duplicate ValidationResult class and sys.path coupling. Neither blocks merge, but consolidation recommended for long-term maintainability.
DevOps Review Details

Based on my review of the changed files, I can now provide the DevOps assessment.


DevOps Review: PR #1333

PR Scope Detection

Category Files Review Scope
CODE scripts/validation/agent_registry.py Build impact only
CODE tests/test_agent_registry.py Build impact only

Classification: CODE PR (validation script + tests). No workflow, action, or infrastructure changes.


Pipeline Impact Assessment

Area Impact Notes
Build None No build config changes
Test Low Adds 18 tests to existing suite
Deploy None No deployment changes
Cost None Negligible CI time increase

CI/CD Quality Checks

Check Status Location
Workflow files modified N/A No workflow changes
Actions pinned N/A No action changes
Secrets handling N/A No secrets usage
Shell scripts N/A No shell scripts modified

Code Review (Build/Test Impact)

Check Status Notes
Exit codes follow ADR-035 0=success, 1=logic, 2=config
Tests use pytest conventions Located in tests/ per pyproject.toml
No circular imports Clean import from build/generate_agents_common
Path handling Uses Path with resolve()
File permissions test Properly restores chmod in test cleanup

Findings

Severity Category Finding Location Fix
Low Robustness sys.path.insert for import is functional but fragile agent_registry.py:22-23 Consider package-based import if build utilities expand

Automation Opportunities

Opportunity Type Benefit Effort
Add agent validation to CI Workflow Detect agent drift automatically Low
Pre-commit hook for agent changes Skill Catch model mismatches before PR Medium

Recommendations

  1. Consider adding this validator to CI workflows when agent files or AGENTS.md change

VERDICT: PASS
MESSAGE: CODE-only PR with validation script and tests. No CI/CD, workflow, or infrastructure concerns. Exit codes follow ADR-035, tests are properly structured.
Roadmap Review Details

Based on my review of the PR changes and the product roadmap, I can now provide the strategic alignment assessment.


Strategic Alignment Assessment

Criterion Rating Notes
Aligns with project goals High Agent system consistency is core to Master Product Objective
Priority appropriate Medium Not on current roadmap, but enables maintainability goals
User value clear High Prevents model drift and configuration errors across 18+ agents
Investment justified High 628 lines for 18 agents, reuses existing build utilities

Feature Completeness

  • Scope Assessment: Right-sized
  • Ship Ready: Yes
  • MVP Complete: Yes
  • Enhancement Opportunities: CI integration via pre-commit hook or GitHub Action (future work)

Impact Analysis

Dimension Assessment Notes
User Value Medium Maintainers catch drift before it causes production issues
Business Impact Medium Reduces agent inconsistency bugs, supports v1.1 Maintainability goals
Technical Leverage High Reuses generate_agents_common.py, establishes validation pattern for other registries
Competitive Position Neutral Internal tooling, no external competitive impact

Concerns

Priority Concern Recommendation
Low Not explicitly on roadmap Aligns with v1.1 Maintainability theme; no conflict
Low No CI integration yet Document as follow-up work; validator is useful standalone

Recommendations

  1. Approve: This validator directly supports the v1.1 Maintainability epic by detecting agent drift between files and AGENTS.md catalog
  2. Future integration: Consider adding to CI pipeline once stabilized (complements the "CI diff-lint job alerts on semantic drift" success criterion in roadmap)
  3. Documentation: Add usage instructions to docs/ or CONTRIBUTING.md for maintainers

Verdict

VERDICT: PASS
MESSAGE: Agent registry validator supports v1.1 Maintainability goals by detecting model drift and configuration errors. Investment is proportionate (628 lines for 18-agent coverage with comprehensive tests). Reuses existing infrastructure. No strategic conflicts.

Run Details
Property Value
Run ID 22481955469
Triggered by pull_request on 1333/merge
Commit 21ab75ecfddb5fed8f46de662c9360f199a2638f

Powered by AI Quality Gate workflow

@coderabbitai coderabbitai bot added the agent-orchestrator Task coordination agent label Feb 27, 2026
@coderabbitai
Copy link

coderabbitai bot commented Feb 27, 2026

Caution

Review failed

Failed to post review comments

📝 Walkthrough

Walkthrough

Adds a new CLI script to parse YAML frontmatter from src/claude/*.md, load the canonical catalog from AGENTS.md, and validate agent definitions (required fields, allowed models, duplicates, and catalog mismatches). Adds comprehensive tests for parsing, discovery, catalog extraction, and validation rules.

Changes

Cohort / File(s) Summary
Agent Registry Validation Script
scripts/validation/agent_registry.py
New script implementing data types (AgentDefinition, CatalogEntry, ValidationResult), parsers (parse_agent_file, parse_agent_files, parse_catalog), validate logic, and main CLI with flags --agent-dir, --catalog, --json and exit codes (0 success, 1 validation failure, 2 config error). Enforces required fields, allowed models (opus, sonnet, haiku), duplicate name detection, and missing/extra agent checks against catalog.
Test Suite
tests/test_agent_registry.py
New comprehensive tests covering frontmatter parsing (valid, missing, missing fields), discovery/filtering of agent files, unreadable file handling, sorted parsing output, catalog table parsing from AGENTS.md, validation scenarios (success, model mismatch, invalid model, missing fields, duplicates), warnings for catalog/agent mismatches, and conditional integration tests against real src/claude/ and AGENTS.md.

Sequence Diagram(s)

sequenceDiagram
  participant CLI as CLI (user)
  participant Parser as Agent Parser
  participant Catalog as Catalog Loader
  participant Validator as Validator
  participant Reporter as Reporter

  CLI->>Parser: read agent markdown files from `src/claude/`
  CLI->>Catalog: read `AGENTS.md`
  Parser->>Parser: extract YAML frontmatter -> AgentDefinition[]
  Catalog->>Catalog: parse agents table -> CatalogEntry[]
  CLI->>Validator: provide AgentDefinition[], CatalogEntry[]
  Validator->>Validator: run checks (required fields, allowed models, duplicates, mismatches)
  Validator->>Reporter: produce ValidationResult (ok / errors / warnings)
  Reporter->>CLI: print results (text or JSON) and set exit code
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • rjmurillo
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.93% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed Title follows conventional commit format with 'feat' type and clear 'validation' scope, accurately summarizing the main change of adding an agent registry parser and validator.
Description check ✅ Passed Description clearly outlines the implementation, validation features, test coverage, and test results directly related to the code changes.
Linked Issues check ✅ Passed PR implements both acceptance criteria: parses agent definitions from src/claude/*.md YAML frontmatter and validates against the canonical catalog (AGENTS.md), with comprehensive test coverage.
Out of Scope Changes check ✅ Passed All changes are directly scoped to implementing the agent registry parser, validator, and tests as specified in issue #1332 with no extraneous modifications.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/1332-autonomous

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai bot added agent-architect Design and ADR agent agent-qa Testing and verification agent labels Feb 27, 2026
Address review feedback from gemini-code-assist:
- parse_agent_files now catches OSError per file and continues
  processing, returning errors alongside parsed agents
- main() wraps parse_catalog in try/except OSError, exiting with
  code 2 on unreadable catalog
- Add test for unreadable file error collection

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rjmurillo-bot rjmurillo-bot merged commit 00b7e22 into main Feb 27, 2026
91 checks passed
@rjmurillo-bot rjmurillo-bot deleted the feat/1332-autonomous branch February 27, 2026 10:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-architect Design and ADR agent agent-orchestrator Task coordination agent agent-qa Testing and verification agent automation Automated workflows and processes enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AO-002b: Implement agent registry parser and validator

1 participant