fix: enrichment application and contract extraction fixes #94
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Fix enrichment not being applied when no source files changed - Check for enrichment before early exit in _check_incremental_changes - Mark bundle for regeneration when enrichment is provided - Ensure bundle is saved after enrichment is applied - Fix --force flag performance regression - Skip hash checking when --force is used for contract extraction - Process all features directly without expensive hash computation - Significantly improves performance for large bundles with --force - Fix type checking errors - Fix possibly unbound variables (is_test_mode, compute_file_hash, features_with_files) - Properly scope variables within conditional blocks - Ensure all variables are initialized before use - Add comprehensive test coverage - Add integration tests for enrichment and contract extraction bugs - Add unit tests for contract extraction logic - All tests now passing (11/11 integration tests) - Update version to 0.23.1 - Sync version across pyproject.toml, setup.py, src/__init__.py, src/specfact_cli/__init__.py - Update CHANGELOG.md with bug fixes and test coverage additions
- Remove progress.remove_task() calls for relationship and graph analysis - Keep final progress bars visible with completion state instead of removing them - Prevents blank lines from appearing when progress bars disappear - Progress bars now show final completion message and remain visible
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
SpecFact CLI Validation Report✅ All validations passed! |
- Add detailed progress updates showing which feature is currently being processed - For sequential mode: show feature name before and after processing - For parallel mode: show completed feature name and pending count - Progress now displays: 'Extracting contract from FEATURE-NAME... (X/Total, Y pending)' - Improves visibility during long-running contract extraction operations - Helps identify which features are taking longer to process
SpecFact CLI Validation Report✅ All validations passed! |
- Document current performance bottlenecks - Identify AST parsing as primary bottleneck - Propose file-level caching optimization (3-5x speedup) - Suggest batch processing and early exit optimizations - Estimate 5-10x overall speedup potential
… extraction - Add file-level AST caching to prevent redundant parsing (3-5x speedup) - Cache AST trees and file hashes for reuse across features - Invalidate cache when file content changes - Thread-safe cache operations - Add early exit optimization for non-API files (1.5-2x speedup) - Quick regex check before expensive AST parsing - Skip files without API endpoints (models, utilities) - Pre-compiled regex patterns for performance - Add comprehensive tests for optimizations: - Test AST caching prevents redundant parsing - Test early exit skips non-API files - Test cache invalidation on file changes Expected overall improvement: 5-10x speedup for contract extraction For SQLAlchemy (320 features): ~8 minutes -> ~45-90 seconds
SpecFact CLI Validation Report✅ All validations passed! |
- Early exit optimization was too aggressive for ORM/class-based codebases - SQLAlchemy doesn't use FastAPI/Flask decorators, so all files were skipped - Contract extractor also processes class-based APIs and interfaces - Disabled early exit to restore functionality (AST caching still provides 3-5x speedup) - Updated test to reflect that early exit detection works but is disabled in extraction Fixes: 0 contracts generated when using --force flag
SpecFact CLI Validation Report✅ All validations passed! |
…it method processing - Skip non-API class types: Protocol, TypedDict, Enum, ABC, Mixin, Base, Meta, Descriptor, Property - Skip classes that inherit from non-API base types - Filter methods more selectively: skip utility methods (processor, adapter, factory, etc.) - Limit methods processed per class to 15 (skip classes with more methods) - Only process methods that strongly suggest API endpoints (CRUD patterns or short names) Performance improvements: - FEATURE-TYPERESOLVE: Skips Protocol/TypedDict classes and TypeEngine utility methods - FEATURE-COLLECTIONADAPTER: Skips non-API classes, processes only relevant methods - Reduces processing time for large ORM/library codebases Expected improvement: 2-3x faster for features with many utility classes
SpecFact CLI Validation Report✅ All validations passed! |
- Fix interface extraction: Check for interfaces (ABC/Protocol with abstract methods) BEFORE skipping base classes - Interfaces should be processed for contract extraction - Non-interface ABC/Protocol classes are still skipped for performance - Fix progress callback tests: Update tests to expect two calls (total, then completed+description) - Progress callback now sets total on first call, then updates with completed count - Fix deprecation warnings: - Suppress ast.NameConstant deprecation warning (Python 3.8+ compatibility) - Replace datetime.utcnow() with datetime.now(UTC) for Python 3.11+ compatibility - Use timezone.utc fallback for older Python versions Fixes: - test_extract_interface_abstract_methods (was skipping ABC interfaces) - test_create_callback_with_prefix (expected single call, got two) - test_create_callback_without_prefix (expected single call, got two) - DeprecationWarning: ast.NameConstant (Python 3.14) - DeprecationWarning: datetime.utcnow() (future removal)
- Import timezone before try/except block to ensure UTC is defined - Add type: ignore comment to suppress false positive type checker warning - Fixes type checker error: 'UTC' is unbound in except block - Maintains backward compatibility with Python < 3.11
SpecFact CLI Validation Report✅ All validations passed! |
…essing Critical performance fix for large feature sets (320+ features): 1. **Moved file I/O outside lock**: File reading and hash calculation now happen outside the lock, eliminating I/O-bound blocking - Lock only held for cache lookups and updates (minimal scope) - Double-check pattern prevents race conditions 2. **Removed unnecessary locks from openapi_spec writes**: - Each feature has its own openapi_spec dict (no sharing) - Python dict assignment is atomic for single operations - Removed locks from: path initialization, schema addition, security schemes, operation addition - Only cache operations (shared across features) use lock now 3. **Separated cache lock**: Renamed _lock to _cache_lock for clarity - Cache is shared resource (needs protection) - openapi_spec dicts are per-feature (no shared lock needed) Performance impact: - Before: Lock held during file I/O (10-100ms per file) blocks all other threads - After: Lock only held for cache access (<1ms), file I/O happens in parallel - Expected: 5-10x faster for 320 features with parallel processing This fixes the 3-hour extraction time for 320 contracts by eliminating lock contention bottleneck.
SpecFact CLI Validation Report✅ All validations passed! |
- Reuse file content when checking hash vs parsing - Read file only once per cache check/parse cycle - Reduces I/O operations by 50% for cache hits - Maintains thread safety with minimal lock scope This addresses the performance issue where contract extraction was extremely slow (5+ hours) by eliminating redundant file I/O.
- Changed import from specfact_cli.models.project to specfact_cli.models.source_tracking - Fixes CrossHair import error in contract validation workflow - Resolves ImportError: cannot import name 'SourceTracking'
SpecFact CLI Validation Report✅ All validations passed! |
SpecFact CLI Validation Report✅ All validations passed! |
Collaborator
Author
|
Pipelines were successful in previous run without code modifications, just updating basedpyright settings. |
djm81
added a commit
that referenced
this pull request
Jan 9, 2026
* docs: improve documentation structure with unified command chains and cross-linking (#79) Co-authored-by: Dominikus Nold <[email protected]> * docs: add integrations overview guide (optional task 6.4) - Create integrations-overview.md with comprehensive overview of all integrations - Add links from integration guides to integrations-overview.md - Add link to integrations-overview.md in docs/README.md - Complete optional task 6.4 from improve-documentation-structure change * docs: fix linting errors in integrations-overview.md - Fix MD036 warnings by converting emphasis to proper headings - Fix MD040 warning by adding language specifier to code block * docs: simplify README and add links to new documentation - Update website links to specfact.com / .io / .dev - Add GitHub Pages docs link: https://nold-ai.github.io/specfact-cli/ - Remove version info section (avoids outdated info) - Simplify content - remove verbose sections, add links to docs instead - Add links to new documentation: - Command Chains Reference - Common Tasks Quick Reference - AI IDE Workflow Guide - Integrations Overview - Improve onboarding with clear path for new users * docs: add prominent SpecFact domain links with context - Add specfact.com, specfact.io, specfact.dev links prominently at top - Add domain purpose context (commercial, ecosystem, developer community) - Highlight specfact.dev for developers - Add GitHub Pages docs link - Improve user navigation to appropriate resources * docs: update Quick Start with correct IDE setup workflow - Add Step 2: Initialize IDE integration (specfact init --ide) - Update Step 3: Use slash commands in IDE or CLI - Add realistic timing expectations (10-15 min for typical repos) - Explain what init does (copies prompts, makes slash commands available) - Add link to AI IDE Workflow Guide - Remove unrealistic '60 seconds' claim * fix: correct heading level for SpecFact Domains section * docs: fix GitHub Pages permalinks for all documentation pages - Update permalinks to include full directory path (e.g., /reference/commands/ instead of /commands/) - Add frontmatter with permalinks to agile-scrum-workflows.md and reference/README.md - Add frontmatter with permalink to speckit-journey.md for consistency - All permalinks now match the Jekyll configuration pattern - Enables proper GitHub Pages URLs for platform-frontend sites * fix: resolve Jekyll build errors for GitHub Pages - Quote title in speckit-journey.md frontmatter to fix YAML parsing error - Wrap Jinja2 template code in {% raw %} tags in agile-scrum-workflows.md to prevent Jekyll from parsing it as Liquid syntax Fixes GitHub Pages build errors: - YAML Exception in speckit-journey.md (line 3) - Liquid syntax error in agile-scrum-workflows.md (line 708) * docs: add new pages to GitHub Pages navigation menu - Add Command Chains to Guides section - Add Agile/Scrum Workflows to Guides section - Add Reference Documentation index to Reference section These pages were missing from the navigation menu after fixing permalinks. * docs: add new pages to GitHub Pages sidebar navigation menu - Add Command Chains to Guides section (top of list) - Add Agile/Scrum Workflows to Guides section - Add Reference Documentation index to Reference section - Fix reference links to use correct permalinks (/reference/architecture/, etc.) The sidebar navigation menu is hardcoded in the layout file, so these pages need to be manually added to appear in the left sidebar. * feat: add Mermaid.js support for diagram rendering on GitHub Pages - Add Mermaid.js CDN script to layout - Add JavaScript to convert mermaid code blocks to renderable divs - Handle kramdown output format (pre > code.language-mermaid) - Initialize Mermaid with proper configuration Fixes Mermaid diagram rendering on GitHub Pages documentation. All mermaid code blocks will now render as interactive diagrams. * feat: align GitHub Pages styling with specfact.io design - Update color scheme to match specfact.io (dark theme with cyan accent) - Change primary colors: #64ffda (cyan), #0a192f (dark blue), #112240 (light dark) - Update Mermaid theme to dark with custom colors matching specfact.io - Add Inter and JetBrains Mono fonts to match specfact.io typography - Add Mermaid-specific CSS styling for better diagram appearance - Remove light mode support, use dark theme consistently Colors now match specfact.io: - Primary/Highlight: #64ffda (cyan) - Background: #0a192f (dark blue) - Text: #ccd6f6 (light blue-gray) - Code background: #1d2d50 (darker blue) Mermaid diagrams now use dark theme with cyan accents for better readability and visual consistency with specfact.io documentation site. * fix: improve YAML and code syntax highlighting for dark theme - Update Rouge syntax highlighting colors for dark theme readability - Use cyan (#64ffda) for literals, numbers, and constants - Use light green (#a8e6cf) for strings (better contrast on dark) - Use pink (#ff6b9d) for keywords and operators - Use purple (#c792ea) for functions and classes - Use yellow (#ffd93d) for variables - Use muted gray-blue (#8892b0) for comments - YAML keys now use cyan color for better visibility Fixes readability issues with YAML and other code blocks on dark background. * feat: update to custom domain docs.specfact.io - Update _config.yml: set baseurl to empty string for custom domain - Update _config.yml: set url to https://docs.specfact.io - Exclude assets/ from default permalink pattern to fix CSS path - Update README.md to use new docs.specfact.io domain - Fixes CSS 404 errors on custom domain * docs: add CNAME file for GitHub Pages custom domain Required for GitHub Pages to recognize docs.specfact.io as custom domain. This file must be in the repository root (not in docs/). * fix: update root _config.yml for custom domain - Set baseurl to empty string for custom domain - Set url to https://docs.specfact.io - Exclude assets/ from permalink pattern to fix CSS paths - This file is copied to docs/ by GitHub Pages workflow, so it must match docs/_config.yml Fixes CSS 404 error: /specfact-cli/assets/main.css -> /assets/main.css * fix: correct Reference navigation link in top menu - Change from /commands/ to /reference/ to match actual permalink - Fixes broken link in upper navigation menu * feat: Version 0.23.0 - Performance optimizations and progress reporting for large codebases (#92) * fix: enrichment application and contract extraction fixes (#94) * fix: enrichment application and contract extraction improvements - Fix enrichment not being applied when no source files changed - Check for enrichment before early exit in _check_incremental_changes - Mark bundle for regeneration when enrichment is provided - Ensure bundle is saved after enrichment is applied - Fix --force flag performance regression - Skip hash checking when --force is used for contract extraction - Process all features directly without expensive hash computation - Significantly improves performance for large bundles with --force - Fix type checking errors - Fix possibly unbound variables (is_test_mode, compute_file_hash, features_with_files) - Properly scope variables within conditional blocks - Ensure all variables are initialized before use - Add comprehensive test coverage - Add integration tests for enrichment and contract extraction bugs - Add unit tests for contract extraction logic - All tests now passing (11/11 integration tests) - Update version to 0.23.1 - Sync version across pyproject.toml, setup.py, src/__init__.py, src/specfact_cli/__init__.py - Update CHANGELOG.md with bug fixes and test coverage additions * fix: keep progress bars visible during enhanced analysis - Remove progress.remove_task() calls for relationship and graph analysis - Keep final progress bars visible with completion state instead of removing them - Prevents blank lines from appearing when progress bars disappear - Progress bars now show final completion message and remain visible * feat: show current feature/contract in contract extraction progress - Add detailed progress updates showing which feature is currently being processed - For sequential mode: show feature name before and after processing - For parallel mode: show completed feature name and pending count - Progress now displays: 'Extracting contract from FEATURE-NAME... (X/Total, Y pending)' - Improves visibility during long-running contract extraction operations - Helps identify which features are taking longer to process * docs: add contract extraction performance analysis - Document current performance bottlenecks - Identify AST parsing as primary bottleneck - Propose file-level caching optimization (3-5x speedup) - Suggest batch processing and early exit optimizations - Estimate 5-10x overall speedup potential * perf: implement AST caching and early exit optimizations for contract extraction - Add file-level AST caching to prevent redundant parsing (3-5x speedup) - Cache AST trees and file hashes for reuse across features - Invalidate cache when file content changes - Thread-safe cache operations - Add early exit optimization for non-API files (1.5-2x speedup) - Quick regex check before expensive AST parsing - Skip files without API endpoints (models, utilities) - Pre-compiled regex patterns for performance - Add comprehensive tests for optimizations: - Test AST caching prevents redundant parsing - Test early exit skips non-API files - Test cache invalidation on file changes Expected overall improvement: 5-10x speedup for contract extraction For SQLAlchemy (320 features): ~8 minutes -> ~45-90 seconds * fix: disable aggressive early exit that skipped all SQLAlchemy files - Early exit optimization was too aggressive for ORM/class-based codebases - SQLAlchemy doesn't use FastAPI/Flask decorators, so all files were skipped - Contract extractor also processes class-based APIs and interfaces - Disabled early exit to restore functionality (AST caching still provides 3-5x speedup) - Updated test to reflect that early exit detection works but is disabled in extraction Fixes: 0 contracts generated when using --force flag * perf: optimize class-based extraction to skip non-API classes and limit method processing - Skip non-API class types: Protocol, TypedDict, Enum, ABC, Mixin, Base, Meta, Descriptor, Property - Skip classes that inherit from non-API base types - Filter methods more selectively: skip utility methods (processor, adapter, factory, etc.) - Limit methods processed per class to 15 (skip classes with more methods) - Only process methods that strongly suggest API endpoints (CRUD patterns or short names) Performance improvements: - FEATURE-TYPERESOLVE: Skips Protocol/TypedDict classes and TypeEngine utility methods - FEATURE-COLLECTIONADAPTER: Skips non-API classes, processes only relevant methods - Reduces processing time for large ORM/library codebases Expected improvement: 2-3x faster for features with many utility classes * Apply format * fix: resolve CI test failures and deprecation warnings - Fix interface extraction: Check for interfaces (ABC/Protocol with abstract methods) BEFORE skipping base classes - Interfaces should be processed for contract extraction - Non-interface ABC/Protocol classes are still skipped for performance - Fix progress callback tests: Update tests to expect two calls (total, then completed+description) - Progress callback now sets total on first call, then updates with completed count - Fix deprecation warnings: - Suppress ast.NameConstant deprecation warning (Python 3.8+ compatibility) - Replace datetime.utcnow() with datetime.now(UTC) for Python 3.11+ compatibility - Use timezone.utc fallback for older Python versions Fixes: - test_extract_interface_abstract_methods (was skipping ABC interfaces) - test_create_callback_with_prefix (expected single call, got two) - test_create_callback_without_prefix (expected single call, got two) - DeprecationWarning: ast.NameConstant (Python 3.14) - DeprecationWarning: datetime.utcnow() (future removal) * fix: correct UTC import for type checking - Import timezone before try/except block to ensure UTC is defined - Add type: ignore comment to suppress false positive type checker warning - Fixes type checker error: 'UTC' is unbound in except block - Maintains backward compatibility with Python < 3.11 * perf: reduce lock contention in contract extraction for parallel processing Critical performance fix for large feature sets (320+ features): 1. **Moved file I/O outside lock**: File reading and hash calculation now happen outside the lock, eliminating I/O-bound blocking - Lock only held for cache lookups and updates (minimal scope) - Double-check pattern prevents race conditions 2. **Removed unnecessary locks from openapi_spec writes**: - Each feature has its own openapi_spec dict (no sharing) - Python dict assignment is atomic for single operations - Removed locks from: path initialization, schema addition, security schemes, operation addition - Only cache operations (shared across features) use lock now 3. **Separated cache lock**: Renamed _lock to _cache_lock for clarity - Cache is shared resource (needs protection) - openapi_spec dicts are per-feature (no shared lock needed) Performance impact: - Before: Lock held during file I/O (10-100ms per file) blocks all other threads - After: Lock only held for cache access (<1ms), file I/O happens in parallel - Expected: 5-10x faster for 320 features with parallel processing This fixes the 3-hour extraction time for 320 contracts by eliminating lock contention bottleneck. * perf: optimize AST cache to avoid redundant file reads - Reuse file content when checking hash vs parsing - Read file only once per cache check/parse cycle - Reduces I/O operations by 50% for cache hits - Maintains thread safety with minimal lock scope This addresses the performance issue where contract extraction was extremely slow (5+ hours) by eliminating redundant file I/O. * docs: update CHANGELOG for 0.23.1 with contract extraction performance fixes * Fix slowness bug in contract extraction * fix: correct SourceTracking import in profiling script - Changed import from specfact_cli.models.project to specfact_cli.models.source_tracking - Fixes CrossHair import error in contract validation workflow - Resolves ImportError: cannot import name 'SourceTracking' * Fix parallel processing of contract analysis * Revert venv config --------- Co-authored-by: Dominikus Nold <[email protected]> --------- Signed-off-by: Dom <[email protected]> Co-authored-by: Dominikus Nold <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes several critical bugs related to enrichment application and contract extraction:
Bug Fixes
Unhashable Type Error: Fixed
TypeError: unhashable type: 'Feature'by changingfeature_to_filesfromdict[Feature, list[Path]]todict[str, list[Path]]using feature keys instead of Feature objects.Enrichment Performance Regression: Fixed performance issue where applying enrichment reports forced full contract regeneration. Enrichment now only triggers contract extraction for new features, not all existing ones.
Contract Extraction Order: Moved enrichment application to occur before contract extraction to ensure new features from enrichment are present during contract extraction.
Force Flag Optimization: Optimized
--forceflag behavior to skip expensive hash checking and directly process all features when force mode is enabled.UI Progress Bar Fix: Fixed issue where progress bars during enhanced analysis would disappear leaving blank lines. Progress bars now remain visible with their final completion state.
Changes
_extract_contractsto use string keys instead of Feature objects_check_incremental_changesto not force full regeneration on enrichmentforceparameter to_extract_contractsto skip hash checkingprogress.remove_task()calls to keep progress bars visibleTesting
test_import_enrichment_contracts.pywith comprehensive integration teststest_import_contract_extraction.pywith unit testsVersion