feat(assessors): implement File Size Limits assessor (Tier 2) #141

jeremyeder · 2025-11-25T15:41:27Z

Implement FileSizeLimitsAssessor to check for excessively large files that strain AI context windows. This Tier 2 critical assessor helps identify files that should be refactored into smaller modules.

Features:

Checks source files across 12+ programming languages
Flags files >1000 lines (fail) and 500-1000 lines (warn)
Provides proportional scoring based on percentage of large files
Includes actionable remediation guidance for refactoring

Also removed duplicate one_command_setup stub (already implemented in structure.py).

This brings implemented assessors to 24/31 (7 stubs remaining).

🤖 Generated with Claude Code

Description

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update
Refactoring (no functional changes)
Performance improvement
Test coverage improvement

Related Issues

Fixes #
Relates to #

Changes Made

Testing

Unit tests pass (pytest)
Integration tests pass
Manual testing performed
No new warnings or errors

Checklist

My code follows the project's code style
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

Summary by CodeRabbit

New Features
- File Size Limits assessor is now fully operational, scanning source code files and assessing file sizes against thresholds (500 and 1000+ lines) to evaluate code maintainability.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

Implement FileSizeLimitsAssessor to check for excessively large files that strain AI context windows. This Tier 2 critical assessor helps identify files that should be refactored into smaller modules. Features: - Checks source files across 12+ programming languages - Flags files >1000 lines (fail) and 500-1000 lines (warn) - Provides proportional scoring based on percentage of large files - Includes actionable remediation guidance for refactoring Also removed duplicate one_command_setup stub (already implemented in structure.py). This brings implemented assessors to 24/31 (7 stubs remaining). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

coderabbitai · 2025-11-25T15:41:39Z

Walkthrough

The FileSizeLimitsAssessor transitions from a stub to a fully implemented Tier 2 Critical assessor that scans source code files, counts lines, and computes scores based on file size thresholds. It is integrated into the assessor factory and replaces two removed stub entries in the assessor creation pipeline.

Changes

Cohort / File(s)	Summary
Assessor Factory Configuration `src/agentready/assessors/__init__.py`	Added `FileSizeLimitsAssessor` to public imports and integrated into `create_all_assessors()` factory (Tier 2 Critical). Updated implementation count from 6 to 7, stub count from 4 to 3.
Assessor Implementation `src/agentready/assessors/stub_assessors.py`	Implemented `FileSizeLimitsAssessor` class with methods for `attribute_id`, `tier`, `attribute`, and `assess`. Scans source files, counts lines, returns `not_applicable` if no sources found, computes score based on files >500 and >1000 lines. Removed stub instantiations for `one_command_setup` and `file_size_limits` from `create_stub_assessors()`.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Focus areas: Core logic in FileSizeLimitsAssessor.assess() method, particularly file scanning, line counting, and scoring thresholds
Verify: File extension patterns and line-count logic accuracy; remediation plan messaging for large vs. huge files

Poem

🐰 With whiskers twitched and code review keen,
I find the file-size assessor, lean!
From stub to life, five hundred lines divine,
A thousand crossed? This assessor will shine!
hippity-hop to cleaner code design! 🐇

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: implementing the FileSizeLimitsAssessor (Tier 2) assessor, which is the primary objective of the changeset.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch feat/one-command-setup-assessor

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

Provide your own instructions using the high_level_summary_instructions setting.
Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

📝 Description — Summarize the main change in 50–60 words, explaining what was done.

📓 References — List relevant issues, discussions, documentation, or related PRs.

📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.

📊 Contributor Summary — Include a Markdown table showing contributions:
| Contributor | Lines Added | Lines Removed | Files Changed |

✔️ Additional Notes — Add any extra reviewer context.
Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 6

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 244ba8b and 25ed2e5.

📒 Files selected for processing (2)

src/agentready/assessors/__init__.py (2 hunks)
src/agentready/assessors/stub_assessors.py (2 hunks)

🧰 Additional context used

📓 Path-based instructions (3)

**/assessors/*.py

📄 CodeRabbit inference engine (CLAUDE.md)

**/assessors/*.py: All new assessors must have unit tests with >80% code coverage maintained
All new assessor classes must inherit from BaseAssessor and implement required methods: attribute_id property, assess(repository) method, and optional is_applicable(repository) method
Use 'strategy pattern' for assessor implementations where each assessor is independent and self-contained
Assessors must implement graceful degradation: return 'skipped' if tools are missing rather than crashing
Assessor remediation guidance must be actionable with steps, tools, commands, examples, and citations
Use CLAUDEmdAssessor and READMEAssessor as reference implementations when creating new assessors
Use calculate_proportional_score() for partial compliance scoring in assessors

Files:

src/agentready/assessors/__init__.py
src/agentready/assessors/stub_assessors.py

**/*.py

📄 CodeRabbit inference engine (CLAUDE.md)

Python source code must be formatted with Black, imports sorted with isort, and linted with Ruff before commit

Files:

src/agentready/assessors/__init__.py
src/agentready/assessors/stub_assessors.py

src/**/*.py

📄 CodeRabbit inference engine (CLAUDE.md)

src/**/*.py: Use library-first architecture with no global state across all modules
Document all public APIs in docstrings with parameter descriptions, return types, and usage examples

Files:

src/agentready/assessors/__init__.py
src/agentready/assessors/stub_assessors.py

🧠 Learnings (8)

📓 Common learnings

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Applies to **/assessors/*.py : Use `CLAUDEmdAssessor` and `READMEAssessor` as reference implementations when creating new assessors

📚 Learning: 2025-11-25T15:15:10.808Z

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Applies to **/assessors/*.py : Use `CLAUDEmdAssessor` and `READMEAssessor` as reference implementations when creating new assessors

Applied to files:

src/agentready/assessors/__init__.py
src/agentready/assessors/stub_assessors.py

📚 Learning: 2025-11-25T15:15:10.808Z

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Applies to **/assessors/*.py : Use 'strategy pattern' for assessor implementations where each assessor is independent and self-contained

Applied to files:

src/agentready/assessors/__init__.py
src/agentready/assessors/stub_assessors.py

📚 Learning: 2025-11-25T15:15:10.808Z

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Applies to **/assessors/*.py : All new assessors must have unit tests with >80% code coverage maintained

Applied to files:

src/agentready/assessors/__init__.py
src/agentready/assessors/stub_assessors.py

📚 Learning: 2025-11-25T15:15:10.808Z

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Applies to **/assessors/*.py : All new assessor classes must inherit from BaseAssessor and implement required methods: `attribute_id` property, `assess(repository)` method, and optional `is_applicable(repository)` method

Applied to files:

src/agentready/assessors/__init__.py
src/agentready/assessors/stub_assessors.py

📚 Learning: 2025-11-25T15:15:10.808Z

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Read existing assessors before implementing new ones to understand patterns and avoid duplication

Applied to files:

src/agentready/assessors/__init__.py

📚 Learning: 2025-11-25T15:15:10.808Z

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Applies to **/assessors/*.py : Assessors must implement graceful degradation: return 'skipped' if tools are missing rather than crashing

Applied to files:

src/agentready/assessors/__init__.py

📚 Learning: 2025-11-25T15:15:10.808Z

Learnt from: CR
Repo: ambient-code/agentready PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T15:15:10.808Z
Learning: Applies to **/assessors/*.py : Assessor remediation guidance must be actionable with steps, tools, commands, examples, and citations

Applied to files:

src/agentready/assessors/stub_assessors.py

🧬 Code graph analysis (2)

src/agentready/assessors/__init__.py (1)

src/agentready/assessors/stub_assessors.py (1)

FileSizeLimitsAssessor (224-346)

src/agentready/assessors/stub_assessors.py (4)

src/agentready/assessors/base.py (1)

BaseAssessor (9-111)

src/agentready/models/attribute.py (1)

Attribute (7-51)

src/agentready/models/repository.py (1)

Repository (10-112)

src/agentready/models/finding.py (3)

Finding (50-158)

not_applicable (105-117)

Remediation (10-46)

🔇 Additional comments (3)

src/agentready/assessors/__init__.py (1)

31-31: LGTM! Clean integration of the new assessor.

The import, factory registration, and comment update correctly integrate FileSizeLimitsAssessor into the assessor creation pipeline, following the established patterns.

Also applies to: 62-68

src/agentready/assessors/stub_assessors.py (2)

224-246: LGTM! Class structure follows BaseAssessor pattern correctly.

The assessor inherits from BaseAssessor and implements all required properties (attribute_id, tier, attribute) with appropriate values for a Tier 2 Critical assessor.

326-343: LGTM! Remediation guidance is actionable.

The remediation provides concrete steps, tools, and a clear example showing how to split large files into smaller modules, following the coding guidelines for actionable remediation.

coderabbitai · 2025-11-25T15:45:25Z

src/agentready/assessors/stub_assessors.py

+class FileSizeLimitsAssessor(BaseAssessor):
+    """Tier 2 - File size limits for context window optimization."""
+
+    @property
+    def attribute_id(self) -> str:
+        return "file_size_limits"
+
+    @property
+    def tier(self) -> int:
+        return 2
+
+    @property
+    def attribute(self) -> Attribute:
+        return Attribute(
+            id=self.attribute_id,
+            name="File Size Limits",
+            category="Context Window Optimization",
+            tier=self.tier,
+            description="Files are reasonably sized for AI context windows",
+            criteria="<5% of files >500 lines, no files >1000 lines",
+            default_weight=0.03,
+        )
+
+    def assess(self, repository: Repository) -> Finding:
+        """Check for excessively large files that strain context windows.
+
+        Scoring:
+        - 100: All files <500 lines
+        - 75-99: Some files 500-1000 lines
+        - 0-74: Files >1000 lines exist
+        """
+        # Count files by size
+        large_files = []  # 500-1000 lines
+        huge_files = []  # >1000 lines
+        total_files = 0
+
+        # Check common source file extensions
+        extensions = {".py", ".js", ".ts", ".jsx", ".tsx", ".go", ".java", ".rb", ".rs", ".cpp", ".c", ".h"}
+
+        for ext in extensions:
+            pattern = f"**/*{ext}"
+            try:
+                from pathlib import Path
+                for file_path in repository.path.glob(pattern):
+                    if file_path.is_file():
+                        try:
+                            with open(file_path, "r", encoding="utf-8") as f:
+                                lines = len(f.readlines())
+                                total_files += 1
+
+                                if lines > 1000:
+                                    huge_files.append((file_path.relative_to(repository.path), lines))
+                                elif lines > 500:
+                                    large_files.append((file_path.relative_to(repository.path), lines))
+                        except (OSError, UnicodeDecodeError):
+                            # Skip files we can't read
+                            pass
+            except Exception:
+                pass
+
+        if total_files == 0:
+            return Finding.not_applicable(
+                self.attribute,
+                reason="No source files found to assess",
+            )
+
+        # Calculate score
+        if huge_files:
+            # Penalty for files >1000 lines
+            percentage_huge = (len(huge_files) / total_files) * 100
+            score = max(0, 70 - (percentage_huge * 10))
+            status = "fail"
+            evidence = [
+                f"Found {len(huge_files)} files >1000 lines ({percentage_huge:.1f}% of {total_files} files)",
+                f"Largest: {huge_files[0][0]} ({huge_files[0][1]} lines)",
+            ]
+        elif large_files:
+            # Partial credit for files 500-1000 lines
+            percentage_large = (len(large_files) / total_files) * 100
+            if percentage_large < 5:
+                score = 90
+                status = "pass"
+            else:
+                score = max(75, 100 - (percentage_large * 5))
+                status = "pass"
+
+            evidence = [
+                f"Found {len(large_files)} files 500-1000 lines ({percentage_large:.1f}% of {total_files} files)",
+            ]
+        else:
+            # Perfect score
+            score = 100.0
+            status = "pass"
+            evidence = [f"All {total_files} source files are <500 lines"]
+
+        return Finding(
+            attribute=self.attribute,
+            status=status,
+            score=score,
+            measured_value=f"{len(huge_files)} huge, {len(large_files)} large out of {total_files}",
+            threshold="<5% files >500 lines, 0 files >1000 lines",
+            evidence=evidence,
+            remediation=(
+                None
+                if status == "pass"
+                else Remediation(
+                    summary="Refactor large files into smaller, focused modules",
+                    steps=[
+                        "Identify files >1000 lines",
+                        "Split into logical submodules",
+                        "Extract classes/functions into separate files",
+                        "Maintain single responsibility principle",
+                    ],
+                    tools=["refactoring tools", "linters"],
+                    commands=[],
+                    examples=[
+                        "# Split large file:\n# models.py (1500 lines) → models/user.py, models/product.py, models/order.py"
+                    ],
+                    citations=[],
+                )
+            ),
+            error_message=None,
+        )
+


⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify unit tests exist for this new assessor.

The coding guidelines require "All new assessors must have unit tests with >80% code coverage maintained". No test file is visible in this PR for FileSizeLimitsAssessor.

Please run the following script to check if unit tests exist:

If tests don't exist, they should be added to meet the >80% coverage requirement. As per coding guidelines, use CLAUDEmdAssessor and READMEAssessor as reference implementations for test patterns.

🏁 Script executed:

#!/bin/bash # Description: Check for unit tests for FileSizeLimitsAssessor # Search for test files that reference FileSizeLimitsAssessor rg -n "FileSizeLimitsAssessor" --type=py -g "*test*.py" -C 3 # Also check for test files in typical test directories fd -e py -p "test.*file.*size" tests/ test/

Length of output: 51

🏁 Script executed:

#!/bin/bash # Description: Locate test directories and check for FileSizeLimitsAssessor tests # Find test directories in the repo echo "=== Test directories ===" fd -t d -n "test" | head -20 # Find all Python test files regardless of location echo -e "\n=== All Python test files ===" fd -e py -p "test_" | head -20 # Search for any file containing "FileSizeLimitsAssessor" echo -e "\n=== Files mentioning FileSizeLimitsAssessor ===" rg "FileSizeLimitsAssessor" --type=py -l # Check if tests/assessors directory exists echo -e "\n=== Check specific test paths ===" ls -la tests/assessors/ 2>/dev/null || echo "tests/assessors/ not found" ls -la src/agentready/assessors/test*.py 2>/dev/null || echo "No test files in assessors directory" # Look for any test file related to stub_assessors or file_size echo -e "\n=== Search for stub assessor tests ===" rg "stub_assessor" --type=py -l rg "FileSizeLimits\|file_size_limits" --type=py -l

Length of output: 1434

🏁 Script executed:

#!/bin/bash # Description: Check for stub assessor or FileSizeLimitsAssessor tests in detail # Check the test_assessors_structure.py file echo "=== Content of test_assessors_structure.py ===" cat tests/unit/test_assessors_structure.py | head -100 # Look for any test file that might test FileSizeLimitsAssessor or stub assessors echo -e "\n=== Search for assessor test patterns ===" rg "test.*FileSizeLimits|class.*FileSizeLimits|def.*file_size" --type=py # Check testing.py to see what testing utilities are available echo -e "\n=== First 50 lines of testing.py ===" head -50 src/agentready/assessors/testing.py # Look for any test file that imports stub_assessors echo -e "\n=== Files that import stub_assessors ===" rg "from.*stub_assessor|import.*stub_assessor" --type=py # Check if there's a conftest or test configuration echo -e "\n=== pytest configuration ===" cat pytest.ini 2>/dev/null || echo "No pytest.ini found" cat pyproject.toml 2>/dev/null | grep -A 20 "\[tool.pytest" || echo "No pytest config in pyproject.toml"

Length of output: 5639

🏁 Script executed:

#!/bin/bash # Description: Final verification - check if stub_assessors.py has any test coverage # Check if stub_assessors.py has pragma no cover or similar exclusions echo "=== Check for coverage pragmas in stub_assessors.py ===" rg "pragma: no cover|skip|xfail" src/agentready/assessors/stub_assessors.py # Verify what stub assessors are exported from __init__.py echo -e "\n=== Stub assessors exported ===" rg "FileSizeLimits" src/agentready/assessors/__init__.py -A 5 -B 5 # Check overall test file count for assessors echo -e "\n=== All test files matching assessor patterns ===" fd -e py "test_" tests/ | rg -i assessor

Length of output: 878

Add unit tests for FileSizeLimitsAssessor to meet >80% coverage requirement.

The coding guidelines mandate "All new assessors must have unit tests with >80% code coverage maintained." The repository's pytest configuration enforces --cov-fail-under=90, which will cause CI to fail if this assessor lacks test coverage. Currently, no unit tests exist for FileSizeLimitsAssessor.

Create a test file (e.g., tests/unit/test_file_size_limits_assessor.py) following the pattern of test_assessors_structure.py. Test scenarios should include:

Files below 500 lines (100 score)

Files between 500-1000 lines (75-99 score)

Files exceeding 1000 lines (0-74 score)

No source files found (not_applicable)

Unreadable files (graceful degradation)

Reference CLAUDEmdAssessor and READMEAssessor test patterns per guidelines.

🤖 Prompt for AI Agents

In src/agentready/assessors/stub_assessors.py around lines 224-347, the FileSizeLimitsAssessor has no unit tests and CI will fail due to coverage; create tests/unit/test_file_size_limits_assessor.py that follows existing assessor test patterns (e.g., CLAUDEmdAssessor), using pytest and tmp_path to build a temporary repo directory and a minimal Repository-like object with a .path attribute (Path) pointing to it, then cover these scenarios: (1) all source files <500 lines — assert score == 100 and status == "pass"; (2) some files 500-1000 lines — assert score in 75-99 and appropriate measured_value/evidence; (3) files >1000 lines — assert score in 0-74, status "fail", and evidence about huge files; (4) no source files — assert Finding.not_applicable; (5) unreadable files — create a file and make it unreadable (or mock open to raise UnicodeDecodeError/OSError) and assert assessor skips it and still returns correct result; parametrize inputs where useful and assert on measured_value, threshold, and remediation presence to exercise all branches so coverage exceeds the required threshold.

coderabbitai · 2025-11-25T15:45:26Z

src/agentready/assessors/stub_assessors.py

+        for ext in extensions:
+            pattern = f"**/*{ext}"
+            try:
+                from pathlib import Path
+                for file_path in repository.path.glob(pattern):
+                    if file_path.is_file():
+                        try:
+                            with open(file_path, "r", encoding="utf-8") as f:
+                                lines = len(f.readlines())
+                                total_files += 1
+
+                                if lines > 1000:
+                                    huge_files.append((file_path.relative_to(repository.path), lines))
+                                elif lines > 500:
+                                    large_files.append((file_path.relative_to(repository.path), lines))
+                        except (OSError, UnicodeDecodeError):
+                            # Skip files we can't read
+                            pass
+            except Exception:
+                pass


⚠️ Potential issue | 🟠 Major

Narrow exception handling to avoid hiding bugs.

The bare except Exception: on line 281 is too broad and can silently suppress programming errors, making debugging difficult. The inner exception handling (line 278) for file reading is appropriate, but the outer catch-all should be more specific or removed.

If the intent is to handle glob failures, be explicit:

for ext in extensions: pattern = f"**/*{ext}" try: - from pathlib import Path for file_path in repository.path.glob(pattern): if file_path.is_file(): try: with open(file_path, "r", encoding="utf-8") as f: lines = len(f.readlines()) total_files += 1 if lines > 1000: huge_files.append((file_path.relative_to(repository.path), lines)) elif lines > 500: large_files.append((file_path.relative_to(repository.path), lines)) except (OSError, UnicodeDecodeError): # Skip files we can't read pass - except Exception: - pass + except (OSError, PermissionError) as e: + # Skip directories we can't access + continue

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

for ext in extensions:

pattern = f"**/*{ext}"

try:

from pathlib import Path

for file_path in repository.path.glob(pattern):

if file_path.is_file():

try:

with open(file_path, "r", encoding="utf-8") as f:

lines = len(f.readlines())

total_files += 1

if lines > 1000:

huge_files.append((file_path.relative_to(repository.path), lines))

elif lines > 500:

large_files.append((file_path.relative_to(repository.path), lines))

except (OSError, UnicodeDecodeError):

# Skip files we can't read

pass

except Exception:

pass

for ext in extensions:

pattern = f"**/*{ext}"

try:

for file_path in repository.path.glob(pattern):

if file_path.is_file():

try:

with open(file_path, "r", encoding="utf-8") as f:

lines = len(f.readlines())

total_files += 1

if lines > 1000:

huge_files.append((file_path.relative_to(repository.path), lines))

elif lines > 500:

large_files.append((file_path.relative_to(repository.path), lines))

except (OSError, UnicodeDecodeError):

# Skip files we can't read

pass

except (OSError, PermissionError):

# Skip directories we can't access

continue

🤖 Prompt for AI Agents

In src/agentready/assessors/stub_assessors.py around lines 263 to 282, remove the broad `except Exception:` that silently swallows all errors; either delete the outer try/except entirely or narrow it to the specific errors you expect from the glob/Path operations (for example OSError and ValueError), and for any caught cases log a brief debug/error message and allow unexpected exceptions to propagate (re-raise) so real bugs aren’t hidden.

coderabbitai · 2025-11-25T15:45:26Z

src/agentready/assessors/stub_assessors.py

+        for ext in extensions:
+            pattern = f"**/*{ext}"
+            try:
+                from pathlib import Path


🛠️ Refactor suggestion | 🟠 Major

Move import to module level.

Importing Path inside the loop is inefficient and unnecessary since pathlib is already available. The import should be at the top of the file with other imports.

Apply this diff to move the import:

"""Stub implementations for remaining assessors - minimal but functional. These are simplified implementations to get the MVP working. Each can be enhanced later with more sophisticated detection and scoring logic. """ +from pathlib import Path + from ..models.attribute import Attribute from ..models.finding import Finding, Remediation

Then remove line 266 from the loop.

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents

In src/agentready/assessors/stub_assessors.py around line 266, an import statement "from pathlib import Path" is placed inside a loop; move that import to the module-level top with the other imports (ensuring you don't duplicate it if Path is already imported), then remove the in-loop import at line 266 so the loop uses the module-level Path reference.

coderabbitai · 2025-11-25T15:45:26Z

src/agentready/assessors/stub_assessors.py

+        if huge_files:
+            # Penalty for files >1000 lines
+            percentage_huge = (len(huge_files) / total_files) * 100
+            score = max(0, 70 - (percentage_huge * 10))
+            status = "fail"
+            evidence = [
+                f"Found {len(huge_files)} files >1000 lines ({percentage_huge:.1f}% of {total_files} files)",
+                f"Largest: {huge_files[0][0]} ({huge_files[0][1]} lines)",
+            ]
+        elif large_files:
+            # Partial credit for files 500-1000 lines
+            percentage_large = (len(large_files) / total_files) * 100
+            if percentage_large < 5:
+                score = 90
+                status = "pass"
+            else:
+                score = max(75, 100 - (percentage_large * 5))
+                status = "pass"
+
+            evidence = [
+                f"Found {len(large_files)} files 500-1000 lines ({percentage_large:.1f}% of {total_files} files)",
+            ]
+        else:
+            # Perfect score
+            score = 100.0
+            status = "pass"
+            evidence = [f"All {total_files} source files are <500 lines"]
+


🧹 Nitpick | 🔵 Trivial

Consider using calculate_proportional_score() helper for consistency.

The custom scoring logic works but diverges from the framework's recommended calculate_proportional_score() helper method, which provides consistent, maintainable scoring across assessors.

Based on coding guidelines, assessors should use calculate_proportional_score() for partial compliance scoring.

Example refactor (after fixing the status logic):

# For repositories with huge files (>1000 lines) if huge_files: percentage_huge = (len(huge_files) / total_files) * 100 # Score degrades as percentage increases (lower is better) score = self.calculate_proportional_score( measured_value=percentage_huge, threshold=5.0, # Allow up to 5% before severe penalty higher_is_better=False ) score = min(score, 70) # Cap at 70 for huge files status = "fail" # ... evidence ...

coderabbitai · 2025-11-25T15:45:26Z

src/agentready/assessors/stub_assessors.py

+            status = "fail"
+            evidence = [
+                f"Found {len(huge_files)} files >1000 lines ({percentage_huge:.1f}% of {total_files} files)",
+                f"Largest: {huge_files[0][0]} ({huge_files[0][1]} lines)",


⚠️ Potential issue | 🟡 Minor

Sort huge_files before reporting the largest.

Line 298 assumes huge_files[0] is the largest file, but the list is populated in discovery order, not sorted by size. This could report an arbitrary large file rather than the actual largest.

Apply this diff:

if huge_files: # Penalty for files >1000 lines + huge_files.sort(key=lambda x: x[1], reverse=True) percentage_huge = (len(huge_files) / total_files) * 100 score = max(0, 70 - (percentage_huge * 10)) status = "fail" evidence = [ f"Found {len(huge_files)} files >1000 lines ({percentage_huge:.1f}% of {total_files} files)", f"Largest: {huge_files[0][0]} ({huge_files[0][1]} lines)", ]

🤖 Prompt for AI Agents

In src/agentready/assessors/stub_assessors.py around line 298, the code assumes huge_files[0] is the largest but the list is in discovery order; sort huge_files by the size field (the second tuple element) in descending order before accessing index 0, e.g. sort with key=lambda t: t[1], reverse=True, and also handle the empty-list case safely (e.g., skip reporting or provide a fallback message) so the reported "Largest" truly reflects the largest file.

coderabbitai · 2025-11-25T15:45:26Z

src/agentready/assessors/stub_assessors.py

+        elif large_files:
+            # Partial credit for files 500-1000 lines
+            percentage_large = (len(large_files) / total_files) * 100
+            if percentage_large < 5:
+                score = 90
+                status = "pass"
+            else:
+                score = max(75, 100 - (percentage_large * 5))
+                status = "pass"


⚠️ Potential issue | 🔴 Critical

Fix status logic to match criteria.

The criteria states "<5% of files >500 lines", but when percentage_large >= 5%, the status remains "pass" (lines 307-308). This violates the stated criteria and will incorrectly report passing status for repositories that should fail.

Apply this diff:

elif large_files: # Partial credit for files 500-1000 lines percentage_large = (len(large_files) / total_files) * 100 if percentage_large < 5: score = 90 status = "pass" else: score = max(75, 100 - (percentage_large * 5)) - status = "pass" + status = "warn" # or "fail" depending on severity interpretation evidence = [ f"Found {len(large_files)} files 500-1000 lines ({percentage_large:.1f}% of {total_files} files)", ]

Note: The Finding model expects "pass", "fail", "skipped", "error", or "not_applicable" status values, so verify if "warn" is supported or use "fail" instead.

Committable suggestion skipped: line range outside the PR's diff.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2025-11-25T15:46:07Z

src/agentready/assessors/stub_assessors.py

+                score = 90
+                status = "pass"
+            else:
+                score = max(75, 100 - (percentage_large * 5))
+                status = "pass"


Fail when >5% of files exceed 500 lines

The FileSizeLimitsAssessor sets status = "pass" for any repository that only has 500–1000 line files, even when those files make up more than the stated threshold of 5% (percentage_large branch sets status to pass in both paths). Because the attribute criteria/threshold string says <5% files >500 lines, a repo with, for example, 30% of its files at ~800 lines will be marked passing and receive no remediation despite violating the stated limit, so oversized files silently slip through.

Useful? React with 👍 / 👎.

# [2.8.0](v2.7.1...v2.8.0) (2025-11-25) ### Features * **assessors:** implement File Size Limits assessor (Tier 2) ([#141](#141)) ([248467f](248467f))

github-actions · 2025-11-25T15:49:09Z

🎉 This PR is included in version 2.8.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@claude

@claude

This comment was marked as resolved.

Sign in to view

coderabbitai bot reviewed Nov 25, 2025

View reviewed changes

chatgpt-codex-connector bot reviewed Nov 25, 2025

View reviewed changes

jeremyeder merged commit 248467f into main Nov 25, 2025
18 of 19 checks passed

github-actions bot pushed a commit that referenced this pull request Nov 25, 2025

chore(release): 2.8.0 [skip ci]

96cee41

# [2.8.0](v2.7.1...v2.8.0) (2025-11-25) ### Features * **assessors:** implement File Size Limits assessor (Tier 2) ([#141](#141)) ([248467f](248467f))

github-actions bot added the released label Nov 25, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(assessors): implement File Size Limits assessor (Tier 2) #141

feat(assessors): implement File Size Limits assessor (Tier 2) #141

Uh oh!

jeremyeder commented Nov 25, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Nov 25, 2025 •

edited

Loading

Uh oh!

This comment was marked as resolved.

coderabbitai bot left a comment

Uh oh!

coderabbitai bot Nov 25, 2025

Uh oh!

coderabbitai bot Nov 25, 2025

Uh oh!

coderabbitai bot Nov 25, 2025

Uh oh!

coderabbitai bot Nov 25, 2025

Uh oh!

coderabbitai bot Nov 25, 2025

Uh oh!

coderabbitai bot Nov 25, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Nov 25, 2025

Uh oh!

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat(assessors): implement File Size Limits assessor (Tier 2) #141

feat(assessors): implement File Size Limits assessor (Tier 2) #141

Uh oh!

Conversation

jeremyeder commented Nov 25, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of Change

Related Issues

Changes Made

Testing

Checklist

Screenshots (if applicable)

Additional Notes

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks and finishing touches

Uh oh!

This comment was marked as resolved.

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jeremyeder commented Nov 25, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Nov 25, 2025 •

edited

Loading