feat: Add SWE-bench experiment system for validating AgentReady impact

[jeremyeder]

Summary

Implements MVP for quantifying AgentReady attribute effectiveness using SWE-bench benchmarks with both SWE-agent and Claude Code.

This enables data-driven validation of which AgentReady attributes provide the best ROI for AI-assisted development workflows.

New Features

Services (src/agentready/services/)

• sweagent_runner.py: SWE-agent batch execution wrapper
• claudecode_runner.py: Claude Code headless mode integration
• swebench_evaluator.py: Evaluation harness wrapper
• experiment_comparer.py: Multi-experiment result comparison
• attribute_analyzer.py: Correlation analysis + Plotly heatmap generation

CLI Commands (agentready experiment)

• run-agent: Execute SWE-bench tasks with specified agent
• evaluate: Score predictions using evaluation harness
• compare: Compare multiple experiment results
• analyze: Generate correlation analysis and interactive heatmap

Pre-configured Experiments (experiments/configs/)

1. baseline.yaml - Control (no AgentReady changes)
2. claude-md.yaml - CLAUDE.md only (Tier 1 essential)
3. types-docs.yaml - Type annotations + inline documentation
4. tier1.yaml - All 5 Tier 1 attributes
5. full-bootstrap.yaml - All AgentReady best practices

Interactive Visualization

• Plotly Express heatmaps with hover tooltips
• Shows config, agent, score, delta from baseline
• Zoom/pan capability, RdYlGn colormap (seaborn-style)
• Standalone HTML export (shareable without Python)

Usage

# 1. Run agent on repository
agentready experiment run-agent sweagent \
  --repo-path /path/to/repo \
  --dataset lite \
  --output predictions.jsonl

# 2. Evaluate predictions
agentready experiment evaluate \
  --predictions predictions.jsonl \
  --output results.json

# 3. Analyze and generate heatmap
agentready experiment analyze \
  --results-dir results/ \
  --heatmap heatmap.html

# 4. View interactive results
open heatmap.html

Expected Results

Based on sample data:

• Baseline: ~38-39% SWE-bench pass rate
• CLAUDE.md only: +7-8pp improvement
• Full bootstrap: +14pp improvement
• Correlation: r ≈ 0.87 between AgentReady score and SWE-bench performance

Dependencies Added

pandas>=2.0.0
plotly>=5.0.0
scipy>=1.10.0

Plus optional external tools:

• swebench - Evaluation harness
• sweagent - Agent execution

Documentation

• experiments/README.md - Complete workflow guide
• CLAUDE.md - Updated with experiment section
• .plans/swe-bench-experiment-mvp.md - Cold-start prompt for future implementation

Test Plan

• CLI commands accessible (agentready experiment --help)
• Sample heatmap generation works
• All code formatted with black
• Manual validation: Run 1-2 SWE-bench tasks with SWE-agent
• Manual validation: Verify predictions.jsonl format
• Manual validation: Generate analysis and heatmap

Checklist

• Code follows project conventions
• CLI commands registered
• Dependencies added to pyproject.toml
• Documentation updated (CLAUDE.md, experiments/README.md)
• Pre-configured experiment templates created
• Code formatted with black
• Integration tests (manual validation required)

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Include experiment metadata in evaluate outputs

The analysis pipeline assumes each results JSON contains agentready_score, swebench_score, config_name, and agent, e.g. here correlation is computed from agentready_score values. However, agentready experiment evaluate writes only dataset/total/solved/pass_rate (see experiment.py lines 41‑58), so feeding those files into agentready experiment analyze or compare immediately raises a KeyError before any correlation or heatmap can run. The quickstart workflow in experiments/README.md therefore fails as soon as you call analyze on the outputs produced by evaluate.

Useful? React with 👍 / 👎.

[github-actions]

🤖 AgentReady Assessment Report

Repository: agentready
Path: /home/runner/work/agentready/agentready
Branch: HEAD | Commit: 5563470d
Assessed: November 24, 2025 at 5:07 AM
AgentReady Version: 1.29.0
Run by: runner@runnervmg1sw1

---

📊 Summary

Metric	Value
Overall Score	70.0/100
Certification Level	Silver
Attributes Assessed	19/31
Attributes Not Assessed	12
Assessment Duration	1.1s

Languages Detected

• Python: 123 files
• Markdown: 96 files
• YAML: 19 files
• JSON: 9 files
• Shell: 6 files

Repository Stats

• Total Files: 287
• Total Lines: 170,282

🎖️ Certification Ladder

• 💎 Platinum (90-100)
• 🥇 Gold (75-89)
• 🥈 Silver (60-74) → YOUR LEVEL ←
• 🥉 Bronze (40-59)
• ⚠️ Needs Improvement (0-39)

📋 Detailed Findings

API Documentation

Attribute	Tier	Status	Score
OpenAPI/Swagger Specifications	T3	⊘ not_applicable	—

Build & Development

Attribute	Tier	Status	Score
One-Command Build/Setup	T2	✅ pass	100
One-Command Build/Setup	T2	⊘ not_applicable	—
Container/Virtualization Setup	T4	⊘ not_applicable	—

Code Organization

Attribute	Tier	Status	Score
Separation of Concerns	T2	✅ pass	97

Code Quality

Attribute	Tier	Status	Score
Type Annotations	T1	❌ fail	40
Cyclomatic Complexity Thresholds	T3	✅ pass	100
Semantic Naming	T3	✅ pass	100
Structured Logging	T3	❌ fail	0
Code Smell Elimination	T4	⊘ not_applicable	—

❌ Type Annotations

Measured: 32.3% (Threshold: ≥80%)

Evidence:

• Typed functions: 392/1212
• Coverage: 32.3%

📝 Remediation Steps

Add type annotations to function signatures

1. For Python: Add type hints to function parameters and return types
2. For TypeScript: Enable strict mode in tsconfig.json
3. Use mypy or pyright for Python type checking
4. Use tsc --strict for TypeScript
5. Add type annotations gradually to existing code

Commands:

# Python
pip install mypy
mypy --strict src/

# TypeScript
npm install --save-dev typescript
echo '{"compilerOptions": {"strict": true}}' > tsconfig.json

Examples:

# Python - Before
def calculate(x, y):
    return x + y

# Python - After
def calculate(x: float, y: float) -> float:
    return x + y

// TypeScript - tsconfig.json
{
  "compilerOptions": {
    "strict": true,
    "noImplicitAny": true,
    "strictNullChecks": true
  }
}

❌ Structured Logging

Measured: not configured (Threshold: structured logging library)

Evidence:

• No structured logging library found
• Checked files: pyproject.toml
• Using built-in logging module (unstructured)

📝 Remediation Steps

Add structured logging library for machine-parseable logs

1. Choose structured logging library (structlog for Python, winston for Node.js)
2. Install library and configure JSON formatter
3. Add standard fields: timestamp, level, message, context
4. Include request context: request_id, user_id, session_id
5. Use consistent field naming (snake_case for Python)
6. Never log sensitive data (passwords, tokens, PII)
7. Configure different formats for dev (pretty) and prod (JSON)

Commands:

# Install structlog
pip install structlog

# Configure structlog
# See examples for configuration

Examples:

# Python with structlog
import structlog

# Configure structlog
structlog.configure(
    processors=[
        structlog.stdlib.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer()
    ]
)

logger = structlog.get_logger()

# Good: Structured logging
logger.info(
    "user_login",
    user_id="123",
    email="user@example.com",
    ip_address="192.168.1.1"
)

# Bad: Unstructured logging
logger.info(f"User {user_id} logged in from {ip}")

Context Window Optimization

Attribute	Tier	Status	Score
CLAUDE.md Configuration Files	T1	✅ pass	100
File Size Limits	T2	⊘ not_applicable	—

Dependency Management

Attribute	Tier	Status	Score
Lock Files for Reproducibility	T1	❌ fail	0
Dependency Freshness & Security	T2	⊘ not_applicable	—

❌ Lock Files for Reproducibility

Measured: none (Threshold: at least one lock file)

Evidence:

• No lock files found

📝 Remediation Steps

Add lock file for dependency reproducibility

1. Use npm install, poetry lock, or equivalent to generate lock file

Commands:

npm install  # generates package-lock.json

Documentation

Attribute	Tier	Status	Score
Concise Documentation	T2	❌ fail	70
Inline Documentation	T2	✅ pass	100

❌ Concise Documentation

Measured: 276 lines, 40 headings, 38 bullets (Threshold: <500 lines, structured format)

Evidence:

• README length: 276 lines (excellent)
• Heading density: 14.5 per 100 lines (target: 3-5)
• 1 paragraphs exceed 10 lines (walls of text)

📝 Remediation Steps

Make documentation more concise and structured

1. Break long README into multiple documents (docs/ directory)
2. Add clear Markdown headings (##, ###) for structure
3. Convert prose paragraphs to bullet points where possible
4. Add table of contents for documents >100 lines
5. Use code blocks instead of describing commands in prose
6. Move detailed content to wiki or docs/, keep README focused

Commands:

# Check README length
wc -l README.md

# Count headings
grep -c '^#' README.md

Examples:

# Good: Concise with structure

## Quick Start
```bash
pip install -e .
agentready assess .

Features

• Fast repository scanning
• HTML and Markdown reports
• 25 agent-ready attributes

Documentation

See docs/ for detailed guides.

Bad: Verbose prose

This project is a tool that helps you assess your repository
against best practices for AI-assisted development. It works by
scanning your codebase and checking for various attributes that
make repositories more effective when working with AI coding
assistants like Claude Code...

[Many more paragraphs of prose...]

</details>

### Documentation Standards

| Attribute | Tier | Status | Score |
|-----------|------|--------|-------|
| README Structure | T1 | ✅ pass | 100 |
| Architecture Decision Records (ADRs) | T3 | ❌ fail | 0 |
| Architecture Decision Records | T3 | ⊘ not_applicable | — |

#### ❌ Architecture Decision Records (ADRs)

**Measured**: no ADR directory (Threshold: ADR directory with decisions)

**Evidence**:
- No ADR directory found (checked docs/adr/, .adr/, adr/, docs/decisions/)

<details><summary><strong>📝 Remediation Steps</strong></summary>


Create Architecture Decision Records (ADRs) directory and document key decisions

1. Create docs/adr/ directory in repository root
2. Use Michael Nygard ADR template or MADR format
3. Document each significant architectural decision
4. Number ADRs sequentially (0001-*.md, 0002-*.md)
5. Include Status, Context, Decision, and Consequences sections
6. Update ADR status when decisions are revised (Superseded, Deprecated)

**Commands**:

```bash
# Create ADR directory
mkdir -p docs/adr

# Create first ADR using template
cat > docs/adr/0001-use-architecture-decision-records.md << 'EOF'
# 1. Use Architecture Decision Records

Date: 2025-11-22

## Status
Accepted

## Context
We need to record architectural decisions made in this project.

## Decision
We will use Architecture Decision Records (ADRs) as described by Michael Nygard.

## Consequences
- Decisions are documented with context
- Future contributors understand rationale
- ADRs are lightweight and version-controlled
EOF

Examples:

# Example ADR Structure

```markdown
# 2. Use PostgreSQL for Database

Date: 2025-11-22

## Status
Accepted

## Context
We need a relational database for complex queries and ACID transactions.
Team has PostgreSQL experience. Need full-text search capabilities.

## Decision
Use PostgreSQL 15+ as primary database.

## Consequences
- Positive: Robust ACID, full-text search, team familiarity
- Negative: Higher resource usage than SQLite
- Neutral: Need to manage migrations, backups

</details>

### Git & Version Control

| Attribute | Tier | Status | Score |
|-----------|------|--------|-------|
| Conventional Commit Messages | T2 | ❌ fail | 0 |
| .gitignore Completeness | T2 | ✅ pass | 100 |
| Branch Protection Rules | T4 | ⊘ not_applicable | — |
| Issue & Pull Request Templates | T4 | ⊘ not_applicable | — |

#### ❌ Conventional Commit Messages

**Measured**: not configured (Threshold: configured)

**Evidence**:
- No commitlint or husky configuration

<details><summary><strong>📝 Remediation Steps</strong></summary>


Configure conventional commits with commitlint

1. Install commitlint
2. Configure husky for commit-msg hook

**Commands**:

```bash
npm install --save-dev @commitlint/cli @commitlint/config-conventional husky

Performance

Attribute	Tier	Status	Score
Performance Benchmarks	T4	⊘ not_applicable	—

Repository Structure

Attribute	Tier	Status	Score
Standard Project Layouts	T1	✅ pass	100
Issue & Pull Request Templates	T3	✅ pass	100
Separation of Concerns	T2	⊘ not_applicable	—

Security

Attribute	Tier	Status	Score
Security Scanning Automation	T4	⊘ not_applicable	—

Testing & CI/CD

Attribute	Tier	Status	Score
Test Coverage Requirements	T2	✅ pass	100
Pre-commit Hooks & CI/CD Linting	T2	✅ pass	100
CI/CD Pipeline Visibility	T3	❌ fail	60

❌ CI/CD Pipeline Visibility

Measured: basic config (Threshold: CI with best practices)

Evidence:

• CI config found: .github/workflows/docs-lint.yml, .github/workflows/update-docs.yml, .github/workflows/release.yml, .github/workflows/agentready-assessment.yml, .github/workflows/claude-code-action.yml, .github/workflows/security.yml, .github/workflows/tests.yml, .github/workflows/continuous-learning.yml, .github/workflows/publish-pypi.yml
• Descriptive job/step names found
• No caching detected
• No parallelization detected

📝 Remediation Steps

Add or improve CI/CD pipeline configuration

1. Create CI config for your platform (GitHub Actions, GitLab CI, etc.)
2. Define jobs: lint, test, build
3. Use descriptive job and step names
4. Configure dependency caching
5. Enable parallel job execution
6. Upload artifacts: test results, coverage reports
7. Add status badge to README

Commands:

# Create GitHub Actions workflow
mkdir -p .github/workflows
touch .github/workflows/ci.yml

# Validate workflow
gh workflow view ci.yml

Examples:

# .github/workflows/ci.yml - Good example

name: CI Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    name: Lint Code
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'  # Caching

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run linters
        run: |
          black --check .
          isort --check .
          ruff check .

  test:
    name: Run Tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run tests with coverage
        run: pytest --cov --cov-report=xml

      - name: Upload coverage reports
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage.xml

  build:
    name: Build Package
    runs-on: ubuntu-latest
    needs: [lint, test]  # Runs after lint/test pass
    steps:
      - uses: actions/checkout@v4

      - name: Build package
        run: python -m build

      - name: Upload build artifacts
        uses: actions/upload-artifact@v3
        with:
          name: dist
          path: dist/

🎯 Next Steps

Priority Improvements (highest impact first):

1. Lock Files for Reproducibility (Tier 1) - +10.0 points potential
   • Add lock file for dependency reproducibility
2. Type Annotations (Tier 1) - +10.0 points potential
   • Add type annotations to function signatures
3. Conventional Commit Messages (Tier 2) - +3.0 points potential
   • Configure conventional commits with commitlint
4. Concise Documentation (Tier 2) - +3.0 points potential
   • Make documentation more concise and structured
5. Architecture Decision Records (ADRs) (Tier 3) - +1.5 points potential
   • Create Architecture Decision Records (ADRs) directory and document key decisions

---

📝 Assessment Metadata

• Tool Version: AgentReady v1.0.0
• Research Report: Bundled version
• Repository Snapshot: 5563470
• Assessment Duration: 1.1s

🤖 Generated with Claude Code

[github-actions]

🤖 AgentReady Assessment Report

Repository: agentready
Path: /home/runner/work/agentready/agentready
Branch: HEAD | Commit: 9517f100
Assessed: November 24, 2025 at 5:12 AM
AgentReady Version: 1.29.0
Run by: runner@runnervmg1sw1

---

📊 Summary

Metric	Value
Overall Score	70.0/100
Certification Level	Silver
Attributes Assessed	19/31
Attributes Not Assessed	12
Assessment Duration	1.2s

Languages Detected

• Python: 123 files
• Markdown: 96 files
• YAML: 19 files
• JSON: 9 files
• Shell: 6 files

Repository Stats

• Total Files: 287
• Total Lines: 170,282

🎖️ Certification Ladder

• 💎 Platinum (90-100)
• 🥇 Gold (75-89)
• 🥈 Silver (60-74) → YOUR LEVEL ←
• 🥉 Bronze (40-59)
• ⚠️ Needs Improvement (0-39)

📋 Detailed Findings

API Documentation

Attribute	Tier	Status	Score
OpenAPI/Swagger Specifications	T3	⊘ not_applicable	—

Build & Development

Attribute	Tier	Status	Score
One-Command Build/Setup	T2	✅ pass	100
One-Command Build/Setup	T2	⊘ not_applicable	—
Container/Virtualization Setup	T4	⊘ not_applicable	—

Code Organization

Attribute	Tier	Status	Score
Separation of Concerns	T2	✅ pass	97

Code Quality

Attribute	Tier	Status	Score
Type Annotations	T1	❌ fail	40
Cyclomatic Complexity Thresholds	T3	✅ pass	100
Semantic Naming	T3	✅ pass	100
Structured Logging	T3	❌ fail	0
Code Smell Elimination	T4	⊘ not_applicable	—

❌ Type Annotations

Measured: 32.3% (Threshold: ≥80%)

Evidence:

• Typed functions: 392/1212
• Coverage: 32.3%

📝 Remediation Steps

Add type annotations to function signatures

1. For Python: Add type hints to function parameters and return types
2. For TypeScript: Enable strict mode in tsconfig.json
3. Use mypy or pyright for Python type checking
4. Use tsc --strict for TypeScript
5. Add type annotations gradually to existing code

Commands:

# Python
pip install mypy
mypy --strict src/

# TypeScript
npm install --save-dev typescript
echo '{"compilerOptions": {"strict": true}}' > tsconfig.json

Examples:

# Python - Before
def calculate(x, y):
    return x + y

# Python - After
def calculate(x: float, y: float) -> float:
    return x + y

// TypeScript - tsconfig.json
{
  "compilerOptions": {
    "strict": true,
    "noImplicitAny": true,
    "strictNullChecks": true
  }
}

❌ Structured Logging

Measured: not configured (Threshold: structured logging library)

Evidence:

• No structured logging library found
• Checked files: pyproject.toml
• Using built-in logging module (unstructured)

📝 Remediation Steps

Add structured logging library for machine-parseable logs

1. Choose structured logging library (structlog for Python, winston for Node.js)
2. Install library and configure JSON formatter
3. Add standard fields: timestamp, level, message, context
4. Include request context: request_id, user_id, session_id
5. Use consistent field naming (snake_case for Python)
6. Never log sensitive data (passwords, tokens, PII)
7. Configure different formats for dev (pretty) and prod (JSON)

Commands:

# Install structlog
pip install structlog

# Configure structlog
# See examples for configuration

Examples:

# Python with structlog
import structlog

# Configure structlog
structlog.configure(
    processors=[
        structlog.stdlib.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer()
    ]
)

logger = structlog.get_logger()

# Good: Structured logging
logger.info(
    "user_login",
    user_id="123",
    email="user@example.com",
    ip_address="192.168.1.1"
)

# Bad: Unstructured logging
logger.info(f"User {user_id} logged in from {ip}")

Context Window Optimization

Attribute	Tier	Status	Score
CLAUDE.md Configuration Files	T1	✅ pass	100
File Size Limits	T2	⊘ not_applicable	—

Dependency Management

Attribute	Tier	Status	Score
Lock Files for Reproducibility	T1	❌ fail	0
Dependency Freshness & Security	T2	⊘ not_applicable	—

❌ Lock Files for Reproducibility

Measured: none (Threshold: at least one lock file)

Evidence:

• No lock files found

📝 Remediation Steps

Add lock file for dependency reproducibility

1. Use npm install, poetry lock, or equivalent to generate lock file

Commands:

npm install  # generates package-lock.json

Documentation

Attribute	Tier	Status	Score
Concise Documentation	T2	❌ fail	70
Inline Documentation	T2	✅ pass	100

❌ Concise Documentation

Measured: 276 lines, 40 headings, 38 bullets (Threshold: <500 lines, structured format)

Evidence:

• README length: 276 lines (excellent)
• Heading density: 14.5 per 100 lines (target: 3-5)
• 1 paragraphs exceed 10 lines (walls of text)

📝 Remediation Steps

Make documentation more concise and structured

1. Break long README into multiple documents (docs/ directory)
2. Add clear Markdown headings (##, ###) for structure
3. Convert prose paragraphs to bullet points where possible
4. Add table of contents for documents >100 lines
5. Use code blocks instead of describing commands in prose
6. Move detailed content to wiki or docs/, keep README focused

Commands:

# Check README length
wc -l README.md

# Count headings
grep -c '^#' README.md

Examples:

# Good: Concise with structure

## Quick Start
```bash
pip install -e .
agentready assess .

Features

• Fast repository scanning
• HTML and Markdown reports
• 25 agent-ready attributes

Documentation

See docs/ for detailed guides.

Bad: Verbose prose

This project is a tool that helps you assess your repository
against best practices for AI-assisted development. It works by
scanning your codebase and checking for various attributes that
make repositories more effective when working with AI coding
assistants like Claude Code...

[Many more paragraphs of prose...]

</details>

### Documentation Standards

| Attribute | Tier | Status | Score |
|-----------|------|--------|-------|
| README Structure | T1 | ✅ pass | 100 |
| Architecture Decision Records (ADRs) | T3 | ❌ fail | 0 |
| Architecture Decision Records | T3 | ⊘ not_applicable | — |

#### ❌ Architecture Decision Records (ADRs)

**Measured**: no ADR directory (Threshold: ADR directory with decisions)

**Evidence**:
- No ADR directory found (checked docs/adr/, .adr/, adr/, docs/decisions/)

<details><summary><strong>📝 Remediation Steps</strong></summary>


Create Architecture Decision Records (ADRs) directory and document key decisions

1. Create docs/adr/ directory in repository root
2. Use Michael Nygard ADR template or MADR format
3. Document each significant architectural decision
4. Number ADRs sequentially (0001-*.md, 0002-*.md)
5. Include Status, Context, Decision, and Consequences sections
6. Update ADR status when decisions are revised (Superseded, Deprecated)

**Commands**:

```bash
# Create ADR directory
mkdir -p docs/adr

# Create first ADR using template
cat > docs/adr/0001-use-architecture-decision-records.md << 'EOF'
# 1. Use Architecture Decision Records

Date: 2025-11-22

## Status
Accepted

## Context
We need to record architectural decisions made in this project.

## Decision
We will use Architecture Decision Records (ADRs) as described by Michael Nygard.

## Consequences
- Decisions are documented with context
- Future contributors understand rationale
- ADRs are lightweight and version-controlled
EOF

Examples:

# Example ADR Structure

```markdown
# 2. Use PostgreSQL for Database

Date: 2025-11-22

## Status
Accepted

## Context
We need a relational database for complex queries and ACID transactions.
Team has PostgreSQL experience. Need full-text search capabilities.

## Decision
Use PostgreSQL 15+ as primary database.

## Consequences
- Positive: Robust ACID, full-text search, team familiarity
- Negative: Higher resource usage than SQLite
- Neutral: Need to manage migrations, backups

</details>

### Git & Version Control

| Attribute | Tier | Status | Score |
|-----------|------|--------|-------|
| Conventional Commit Messages | T2 | ❌ fail | 0 |
| .gitignore Completeness | T2 | ✅ pass | 100 |
| Branch Protection Rules | T4 | ⊘ not_applicable | — |
| Issue & Pull Request Templates | T4 | ⊘ not_applicable | — |

#### ❌ Conventional Commit Messages

**Measured**: not configured (Threshold: configured)

**Evidence**:
- No commitlint or husky configuration

<details><summary><strong>📝 Remediation Steps</strong></summary>


Configure conventional commits with commitlint

1. Install commitlint
2. Configure husky for commit-msg hook

**Commands**:

```bash
npm install --save-dev @commitlint/cli @commitlint/config-conventional husky

Performance

Attribute	Tier	Status	Score
Performance Benchmarks	T4	⊘ not_applicable	—

Repository Structure

Attribute	Tier	Status	Score
Standard Project Layouts	T1	✅ pass	100
Issue & Pull Request Templates	T3	✅ pass	100
Separation of Concerns	T2	⊘ not_applicable	—

Security

Attribute	Tier	Status	Score
Security Scanning Automation	T4	⊘ not_applicable	—

Testing & CI/CD

Attribute	Tier	Status	Score
Test Coverage Requirements	T2	✅ pass	100
Pre-commit Hooks & CI/CD Linting	T2	✅ pass	100
CI/CD Pipeline Visibility	T3	❌ fail	60

❌ CI/CD Pipeline Visibility

Measured: basic config (Threshold: CI with best practices)

Evidence:

• CI config found: .github/workflows/docs-lint.yml, .github/workflows/update-docs.yml, .github/workflows/release.yml, .github/workflows/agentready-assessment.yml, .github/workflows/claude-code-action.yml, .github/workflows/security.yml, .github/workflows/tests.yml, .github/workflows/continuous-learning.yml, .github/workflows/publish-pypi.yml
• Descriptive job/step names found
• No caching detected
• No parallelization detected

📝 Remediation Steps

Add or improve CI/CD pipeline configuration

1. Create CI config for your platform (GitHub Actions, GitLab CI, etc.)
2. Define jobs: lint, test, build
3. Use descriptive job and step names
4. Configure dependency caching
5. Enable parallel job execution
6. Upload artifacts: test results, coverage reports
7. Add status badge to README

Commands:

# Create GitHub Actions workflow
mkdir -p .github/workflows
touch .github/workflows/ci.yml

# Validate workflow
gh workflow view ci.yml

Examples:

# .github/workflows/ci.yml - Good example

name: CI Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    name: Lint Code
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'  # Caching

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run linters
        run: |
          black --check .
          isort --check .
          ruff check .

  test:
    name: Run Tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run tests with coverage
        run: pytest --cov --cov-report=xml

      - name: Upload coverage reports
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage.xml

  build:
    name: Build Package
    runs-on: ubuntu-latest
    needs: [lint, test]  # Runs after lint/test pass
    steps:
      - uses: actions/checkout@v4

      - name: Build package
        run: python -m build

      - name: Upload build artifacts
        uses: actions/upload-artifact@v3
        with:
          name: dist
          path: dist/

🎯 Next Steps

Priority Improvements (highest impact first):

1. Lock Files for Reproducibility (Tier 1) - +10.0 points potential
   • Add lock file for dependency reproducibility
2. Type Annotations (Tier 1) - +10.0 points potential
   • Add type annotations to function signatures
3. Conventional Commit Messages (Tier 2) - +3.0 points potential
   • Configure conventional commits with commitlint
4. Concise Documentation (Tier 2) - +3.0 points potential
   • Make documentation more concise and structured
5. Architecture Decision Records (ADRs) (Tier 3) - +1.5 points potential
   • Create Architecture Decision Records (ADRs) directory and document key decisions

---

📝 Assessment Metadata

• Tool Version: AgentReady v1.0.0
• Research Report: Bundled version
• Repository Snapshot: 9517f10
• Assessment Duration: 1.2s

🤖 Generated with Claude Code

[github-actions]

🤖 AgentReady Assessment Report

Repository: agentready
Path: /home/runner/work/agentready/agentready
Branch: HEAD | Commit: 24779855
Assessed: November 24, 2025 at 6:59 PM
AgentReady Version: 2.5.0
Run by: runner@runnervmg1sw1

---

📊 Summary

Metric	Value
Overall Score	70.2/100
Certification Level	Silver
Attributes Assessed	19/31
Attributes Not Assessed	12
Assessment Duration	1.1s

Languages Detected

• Python: 131 files
• Markdown: 98 files
• YAML: 20 files
• JSON: 9 files
• Shell: 6 files

Repository Stats

• Total Files: 309
• Total Lines: 173,649

🎖️ Certification Ladder

• 💎 Platinum (90-100)
• 🥇 Gold (75-89)
• 🥈 Silver (60-74) → YOUR LEVEL ←
• 🥉 Bronze (40-59)
• ⚠️ Needs Improvement (0-39)

📋 Detailed Findings

API Documentation

Attribute	Tier	Status	Score
OpenAPI/Swagger Specifications	T3	⊘ not_applicable	—

Build & Development

Attribute	Tier	Status	Score
One-Command Build/Setup	T2	✅ pass	100
One-Command Build/Setup	T2	⊘ not_applicable	—
Container/Virtualization Setup	T4	⊘ not_applicable	—

Code Organization

Attribute	Tier	Status	Score
Separation of Concerns	T2	✅ pass	98

Code Quality

Attribute	Tier	Status	Score
Type Annotations	T1	❌ fail	39
Cyclomatic Complexity Thresholds	T3	✅ pass	100
Semantic Naming	T3	✅ pass	100
Structured Logging	T3	❌ fail	0
Code Smell Elimination	T4	⊘ not_applicable	—

❌ Type Annotations

Measured: 31.5% (Threshold: ≥80%)

Evidence:

• Typed functions: 419/1332
• Coverage: 31.5%

📝 Remediation Steps

Add type annotations to function signatures

1. For Python: Add type hints to function parameters and return types
2. For TypeScript: Enable strict mode in tsconfig.json
3. Use mypy or pyright for Python type checking
4. Use tsc --strict for TypeScript
5. Add type annotations gradually to existing code

Commands:

# Python
pip install mypy
mypy --strict src/

# TypeScript
npm install --save-dev typescript
echo '{"compilerOptions": {"strict": true}}' > tsconfig.json

Examples:

# Python - Before
def calculate(x, y):
    return x + y

# Python - After
def calculate(x: float, y: float) -> float:
    return x + y

// TypeScript - tsconfig.json
{
  "compilerOptions": {
    "strict": true,
    "noImplicitAny": true,
    "strictNullChecks": true
  }
}

❌ Structured Logging

Measured: not configured (Threshold: structured logging library)

Evidence:

• No structured logging library found
• Checked files: pyproject.toml
• Using built-in logging module (unstructured)

📝 Remediation Steps

Add structured logging library for machine-parseable logs

1. Choose structured logging library (structlog for Python, winston for Node.js)
2. Install library and configure JSON formatter
3. Add standard fields: timestamp, level, message, context
4. Include request context: request_id, user_id, session_id
5. Use consistent field naming (snake_case for Python)
6. Never log sensitive data (passwords, tokens, PII)
7. Configure different formats for dev (pretty) and prod (JSON)

Commands:

# Install structlog
pip install structlog

# Configure structlog
# See examples for configuration

Examples:

# Python with structlog
import structlog

# Configure structlog
structlog.configure(
    processors=[
        structlog.stdlib.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer()
    ]
)

logger = structlog.get_logger()

# Good: Structured logging
logger.info(
    "user_login",
    user_id="123",
    email="user@example.com",
    ip_address="192.168.1.1"
)

# Bad: Unstructured logging
logger.info(f"User {user_id} logged in from {ip}")

Context Window Optimization

Attribute	Tier	Status	Score
CLAUDE.md Configuration Files	T1	✅ pass	100
File Size Limits	T2	⊘ not_applicable	—

Dependency Management

Attribute	Tier	Status	Score
Lock Files for Reproducibility	T1	❌ fail	0
Dependency Freshness & Security	T2	⊘ not_applicable	—

❌ Lock Files for Reproducibility

Measured: none (Threshold: at least one lock file)

Evidence:

• No lock files found

📝 Remediation Steps

Add lock file for dependency reproducibility

1. Use npm install, poetry lock, or equivalent to generate lock file

Commands:

npm install  # generates package-lock.json

Documentation

Attribute	Tier	Status	Score
Concise Documentation	T2	❌ fail	70
Inline Documentation	T2	✅ pass	100

❌ Concise Documentation

Measured: 276 lines, 40 headings, 38 bullets (Threshold: <500 lines, structured format)

Evidence:

• README length: 276 lines (excellent)
• Heading density: 14.5 per 100 lines (target: 3-5)
• 1 paragraphs exceed 10 lines (walls of text)

📝 Remediation Steps

Make documentation more concise and structured

1. Break long README into multiple documents (docs/ directory)
2. Add clear Markdown headings (##, ###) for structure
3. Convert prose paragraphs to bullet points where possible
4. Add table of contents for documents >100 lines
5. Use code blocks instead of describing commands in prose
6. Move detailed content to wiki or docs/, keep README focused

Commands:

# Check README length
wc -l README.md

# Count headings
grep -c '^#' README.md

Examples:

# Good: Concise with structure

## Quick Start
```bash
pip install -e .
agentready assess .

Features

• Fast repository scanning
• HTML and Markdown reports
• 25 agent-ready attributes

Documentation

See docs/ for detailed guides.

Bad: Verbose prose

This project is a tool that helps you assess your repository
against best practices for AI-assisted development. It works by
scanning your codebase and checking for various attributes that
make repositories more effective when working with AI coding
assistants like Claude Code...

[Many more paragraphs of prose...]

</details>

### Documentation Standards

| Attribute | Tier | Status | Score |
|-----------|------|--------|-------|
| README Structure | T1 | ✅ pass | 100 |
| Architecture Decision Records (ADRs) | T3 | ❌ fail | 0 |
| Architecture Decision Records | T3 | ⊘ not_applicable | — |

#### ❌ Architecture Decision Records (ADRs)

**Measured**: no ADR directory (Threshold: ADR directory with decisions)

**Evidence**:
- No ADR directory found (checked docs/adr/, .adr/, adr/, docs/decisions/)

<details><summary><strong>📝 Remediation Steps</strong></summary>


Create Architecture Decision Records (ADRs) directory and document key decisions

1. Create docs/adr/ directory in repository root
2. Use Michael Nygard ADR template or MADR format
3. Document each significant architectural decision
4. Number ADRs sequentially (0001-*.md, 0002-*.md)
5. Include Status, Context, Decision, and Consequences sections
6. Update ADR status when decisions are revised (Superseded, Deprecated)

**Commands**:

```bash
# Create ADR directory
mkdir -p docs/adr

# Create first ADR using template
cat > docs/adr/0001-use-architecture-decision-records.md << 'EOF'
# 1. Use Architecture Decision Records

Date: 2025-11-22

## Status
Accepted

## Context
We need to record architectural decisions made in this project.

## Decision
We will use Architecture Decision Records (ADRs) as described by Michael Nygard.

## Consequences
- Decisions are documented with context
- Future contributors understand rationale
- ADRs are lightweight and version-controlled
EOF

Examples:

# Example ADR Structure

```markdown
# 2. Use PostgreSQL for Database

Date: 2025-11-22

## Status
Accepted

## Context
We need a relational database for complex queries and ACID transactions.
Team has PostgreSQL experience. Need full-text search capabilities.

## Decision
Use PostgreSQL 15+ as primary database.

## Consequences
- Positive: Robust ACID, full-text search, team familiarity
- Negative: Higher resource usage than SQLite
- Neutral: Need to manage migrations, backups

</details>

### Git & Version Control

| Attribute | Tier | Status | Score |
|-----------|------|--------|-------|
| Conventional Commit Messages | T2 | ❌ fail | 0 |
| .gitignore Completeness | T2 | ✅ pass | 100 |
| Branch Protection Rules | T4 | ⊘ not_applicable | — |
| Issue & Pull Request Templates | T4 | ⊘ not_applicable | — |

#### ❌ Conventional Commit Messages

**Measured**: not configured (Threshold: configured)

**Evidence**:
- No commitlint or husky configuration

<details><summary><strong>📝 Remediation Steps</strong></summary>


Configure conventional commits with commitlint

1. Install commitlint
2. Configure husky for commit-msg hook

**Commands**:

```bash
npm install --save-dev @commitlint/cli @commitlint/config-conventional husky

Performance

Attribute	Tier	Status	Score
Performance Benchmarks	T4	⊘ not_applicable	—

Repository Structure

Attribute	Tier	Status	Score
Standard Project Layouts	T1	✅ pass	100
Issue & Pull Request Templates	T3	✅ pass	100
Separation of Concerns	T2	⊘ not_applicable	—

Security

Attribute	Tier	Status	Score
Security Scanning Automation	T4	⊘ not_applicable	—

Testing & CI/CD

Attribute	Tier	Status	Score
Test Coverage Requirements	T2	✅ pass	100
Pre-commit Hooks & CI/CD Linting	T2	✅ pass	100
CI/CD Pipeline Visibility	T3	✅ pass	80

🎯 Next Steps

Priority Improvements (highest impact first):

1. Lock Files for Reproducibility (Tier 1) - +10.0 points potential
   • Add lock file for dependency reproducibility
2. Type Annotations (Tier 1) - +10.0 points potential
   • Add type annotations to function signatures
3. Conventional Commit Messages (Tier 2) - +3.0 points potential
   • Configure conventional commits with commitlint
4. Concise Documentation (Tier 2) - +3.0 points potential
   • Make documentation more concise and structured
5. Architecture Decision Records (ADRs) (Tier 3) - +1.5 points potential
   • Create Architecture Decision Records (ADRs) directory and document key decisions

---

📝 Assessment Metadata

• Tool Version: AgentReady v1.0.0
• Research Report: Bundled version
• Repository Snapshot: 2477985
• Assessment Duration: 1.1s

🤖 Generated with Claude Code

[github-actions]

🎉 This PR is included in version 2.6.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀