Commit e98cd2c
authored
feat: integrate sidecar validation into CLI (#98)
* Merge dev into main (#96)
* docs: improve documentation structure with unified command chains and cross-linking (#79)
Co-authored-by: Dominikus Nold <[email protected]>
* docs: add integrations overview guide (optional task 6.4)
- Create integrations-overview.md with comprehensive overview of all integrations
- Add links from integration guides to integrations-overview.md
- Add link to integrations-overview.md in docs/README.md
- Complete optional task 6.4 from improve-documentation-structure change
* docs: fix linting errors in integrations-overview.md
- Fix MD036 warnings by converting emphasis to proper headings
- Fix MD040 warning by adding language specifier to code block
* docs: simplify README and add links to new documentation
- Update website links to specfact.com / .io / .dev
- Add GitHub Pages docs link: https://nold-ai.github.io/specfact-cli/
- Remove version info section (avoids outdated info)
- Simplify content - remove verbose sections, add links to docs instead
- Add links to new documentation:
- Command Chains Reference
- Common Tasks Quick Reference
- AI IDE Workflow Guide
- Integrations Overview
- Improve onboarding with clear path for new users
* docs: add prominent SpecFact domain links with context
- Add specfact.com, specfact.io, specfact.dev links prominently at top
- Add domain purpose context (commercial, ecosystem, developer community)
- Highlight specfact.dev for developers
- Add GitHub Pages docs link
- Improve user navigation to appropriate resources
* docs: update Quick Start with correct IDE setup workflow
- Add Step 2: Initialize IDE integration (specfact init --ide)
- Update Step 3: Use slash commands in IDE or CLI
- Add realistic timing expectations (10-15 min for typical repos)
- Explain what init does (copies prompts, makes slash commands available)
- Add link to AI IDE Workflow Guide
- Remove unrealistic '60 seconds' claim
* fix: correct heading level for SpecFact Domains section
* docs: fix GitHub Pages permalinks for all documentation pages
- Update permalinks to include full directory path (e.g., /reference/commands/ instead of /commands/)
- Add frontmatter with permalinks to agile-scrum-workflows.md and reference/README.md
- Add frontmatter with permalink to speckit-journey.md for consistency
- All permalinks now match the Jekyll configuration pattern
- Enables proper GitHub Pages URLs for platform-frontend sites
* fix: resolve Jekyll build errors for GitHub Pages
- Quote title in speckit-journey.md frontmatter to fix YAML parsing error
- Wrap Jinja2 template code in {% raw %} tags in agile-scrum-workflows.md
to prevent Jekyll from parsing it as Liquid syntax
Fixes GitHub Pages build errors:
- YAML Exception in speckit-journey.md (line 3)
- Liquid syntax error in agile-scrum-workflows.md (line 708)
* docs: add new pages to GitHub Pages navigation menu
- Add Command Chains to Guides section
- Add Agile/Scrum Workflows to Guides section
- Add Reference Documentation index to Reference section
These pages were missing from the navigation menu after fixing permalinks.
* docs: add new pages to GitHub Pages sidebar navigation menu
- Add Command Chains to Guides section (top of list)
- Add Agile/Scrum Workflows to Guides section
- Add Reference Documentation index to Reference section
- Fix reference links to use correct permalinks (/reference/architecture/, etc.)
The sidebar navigation menu is hardcoded in the layout file, so these
pages need to be manually added to appear in the left sidebar.
* feat: add Mermaid.js support for diagram rendering on GitHub Pages
- Add Mermaid.js CDN script to layout
- Add JavaScript to convert mermaid code blocks to renderable divs
- Handle kramdown output format (pre > code.language-mermaid)
- Initialize Mermaid with proper configuration
Fixes Mermaid diagram rendering on GitHub Pages documentation.
All mermaid code blocks will now render as interactive diagrams.
* feat: align GitHub Pages styling with specfact.io design
- Update color scheme to match specfact.io (dark theme with cyan accent)
- Change primary colors: #64ffda (cyan), #0a192f (dark blue), #112240 (light dark)
- Update Mermaid theme to dark with custom colors matching specfact.io
- Add Inter and JetBrains Mono fonts to match specfact.io typography
- Add Mermaid-specific CSS styling for better diagram appearance
- Remove light mode support, use dark theme consistently
Colors now match specfact.io:
- Primary/Highlight: #64ffda (cyan)
- Background: #0a192f (dark blue)
- Text: #ccd6f6 (light blue-gray)
- Code background: #1d2d50 (darker blue)
Mermaid diagrams now use dark theme with cyan accents for better
readability and visual consistency with specfact.io documentation site.
* fix: improve YAML and code syntax highlighting for dark theme
- Update Rouge syntax highlighting colors for dark theme readability
- Use cyan (#64ffda) for literals, numbers, and constants
- Use light green (#a8e6cf) for strings (better contrast on dark)
- Use pink (#ff6b9d) for keywords and operators
- Use purple (#c792ea) for functions and classes
- Use yellow (#ffd93d) for variables
- Use muted gray-blue (#8892b0) for comments
- YAML keys now use cyan color for better visibility
Fixes readability issues with YAML and other code blocks on dark background.
* feat: update to custom domain docs.specfact.io
- Update _config.yml: set baseurl to empty string for custom domain
- Update _config.yml: set url to https://docs.specfact.io
- Exclude assets/ from default permalink pattern to fix CSS path
- Update README.md to use new docs.specfact.io domain
- Fixes CSS 404 errors on custom domain
* docs: add CNAME file for GitHub Pages custom domain
Required for GitHub Pages to recognize docs.specfact.io as custom domain.
This file must be in the repository root (not in docs/).
* fix: update root _config.yml for custom domain
- Set baseurl to empty string for custom domain
- Set url to https://docs.specfact.io
- Exclude assets/ from permalink pattern to fix CSS paths
- This file is copied to docs/ by GitHub Pages workflow, so it must match docs/_config.yml
Fixes CSS 404 error: /specfact-cli/assets/main.css -> /assets/main.css
* fix: correct Reference navigation link in top menu
- Change from /commands/ to /reference/ to match actual permalink
- Fixes broken link in upper navigation menu
* feat: Version 0.23.0 - Performance optimizations and progress reporting for large codebases (#92)
* fix: enrichment application and contract extraction fixes (#94)
* fix: enrichment application and contract extraction improvements
- Fix enrichment not being applied when no source files changed
- Check for enrichment before early exit in _check_incremental_changes
- Mark bundle for regeneration when enrichment is provided
- Ensure bundle is saved after enrichment is applied
- Fix --force flag performance regression
- Skip hash checking when --force is used for contract extraction
- Process all features directly without expensive hash computation
- Significantly improves performance for large bundles with --force
- Fix type checking errors
- Fix possibly unbound variables (is_test_mode, compute_file_hash, features_with_files)
- Properly scope variables within conditional blocks
- Ensure all variables are initialized before use
- Add comprehensive test coverage
- Add integration tests for enrichment and contract extraction bugs
- Add unit tests for contract extraction logic
- All tests now passing (11/11 integration tests)
- Update version to 0.23.1
- Sync version across pyproject.toml, setup.py, src/__init__.py, src/specfact_cli/__init__.py
- Update CHANGELOG.md with bug fixes and test coverage additions
* fix: keep progress bars visible during enhanced analysis
- Remove progress.remove_task() calls for relationship and graph analysis
- Keep final progress bars visible with completion state instead of removing them
- Prevents blank lines from appearing when progress bars disappear
- Progress bars now show final completion message and remain visible
* feat: show current feature/contract in contract extraction progress
- Add detailed progress updates showing which feature is currently being processed
- For sequential mode: show feature name before and after processing
- For parallel mode: show completed feature name and pending count
- Progress now displays: 'Extracting contract from FEATURE-NAME... (X/Total, Y pending)'
- Improves visibility during long-running contract extraction operations
- Helps identify which features are taking longer to process
* docs: add contract extraction performance analysis
- Document current performance bottlenecks
- Identify AST parsing as primary bottleneck
- Propose file-level caching optimization (3-5x speedup)
- Suggest batch processing and early exit optimizations
- Estimate 5-10x overall speedup potential
* perf: implement AST caching and early exit optimizations for contract extraction
- Add file-level AST caching to prevent redundant parsing (3-5x speedup)
- Cache AST trees and file hashes for reuse across features
- Invalidate cache when file content changes
- Thread-safe cache operations
- Add early exit optimization for non-API files (1.5-2x speedup)
- Quick regex check before expensive AST parsing
- Skip files without API endpoints (models, utilities)
- Pre-compiled regex patterns for performance
- Add comprehensive tests for optimizations:
- Test AST caching prevents redundant parsing
- Test early exit skips non-API files
- Test cache invalidation on file changes
Expected overall improvement: 5-10x speedup for contract extraction
For SQLAlchemy (320 features): ~8 minutes -> ~45-90 seconds
* fix: disable aggressive early exit that skipped all SQLAlchemy files
- Early exit optimization was too aggressive for ORM/class-based codebases
- SQLAlchemy doesn't use FastAPI/Flask decorators, so all files were skipped
- Contract extractor also processes class-based APIs and interfaces
- Disabled early exit to restore functionality (AST caching still provides 3-5x speedup)
- Updated test to reflect that early exit detection works but is disabled in extraction
Fixes: 0 contracts generated when using --force flag
* perf: optimize class-based extraction to skip non-API classes and limit method processing
- Skip non-API class types: Protocol, TypedDict, Enum, ABC, Mixin, Base, Meta, Descriptor, Property
- Skip classes that inherit from non-API base types
- Filter methods more selectively: skip utility methods (processor, adapter, factory, etc.)
- Limit methods processed per class to 15 (skip classes with more methods)
- Only process methods that strongly suggest API endpoints (CRUD patterns or short names)
Performance improvements:
- FEATURE-TYPERESOLVE: Skips Protocol/TypedDict classes and TypeEngine utility methods
- FEATURE-COLLECTIONADAPTER: Skips non-API classes, processes only relevant methods
- Reduces processing time for large ORM/library codebases
Expected improvement: 2-3x faster for features with many utility classes
* Apply format
* fix: resolve CI test failures and deprecation warnings
- Fix interface extraction: Check for interfaces (ABC/Protocol with abstract methods) BEFORE skipping base classes
- Interfaces should be processed for contract extraction
- Non-interface ABC/Protocol classes are still skipped for performance
- Fix progress callback tests: Update tests to expect two calls (total, then completed+description)
- Progress callback now sets total on first call, then updates with completed count
- Fix deprecation warnings:
- Suppress ast.NameConstant deprecation warning (Python 3.8+ compatibility)
- Replace datetime.utcnow() with datetime.now(UTC) for Python 3.11+ compatibility
- Use timezone.utc fallback for older Python versions
Fixes:
- test_extract_interface_abstract_methods (was skipping ABC interfaces)
- test_create_callback_with_prefix (expected single call, got two)
- test_create_callback_without_prefix (expected single call, got two)
- DeprecationWarning: ast.NameConstant (Python 3.14)
- DeprecationWarning: datetime.utcnow() (future removal)
* fix: correct UTC import for type checking
- Import timezone before try/except block to ensure UTC is defined
- Add type: ignore comment to suppress false positive type checker warning
- Fixes type checker error: 'UTC' is unbound in except block
- Maintains backward compatibility with Python < 3.11
* perf: reduce lock contention in contract extraction for parallel processing
Critical performance fix for large feature sets (320+ features):
1. **Moved file I/O outside lock**: File reading and hash calculation now happen outside the lock, eliminating I/O-bound blocking
- Lock only held for cache lookups and updates (minimal scope)
- Double-check pattern prevents race conditions
2. **Removed unnecessary locks from openapi_spec writes**:
- Each feature has its own openapi_spec dict (no sharing)
- Python dict assignment is atomic for single operations
- Removed locks from: path initialization, schema addition, security schemes, operation addition
- Only cache operations (shared across features) use lock now
3. **Separated cache lock**: Renamed _lock to _cache_lock for clarity
- Cache is shared resource (needs protection)
- openapi_spec dicts are per-feature (no shared lock needed)
Performance impact:
- Before: Lock held during file I/O (10-100ms per file) blocks all other threads
- After: Lock only held for cache access (<1ms), file I/O happens in parallel
- Expected: 5-10x faster for 320 features with parallel processing
This fixes the 3-hour extraction time for 320 contracts by eliminating lock contention bottleneck.
* perf: optimize AST cache to avoid redundant file reads
- Reuse file content when checking hash vs parsing
- Read file only once per cache check/parse cycle
- Reduces I/O operations by 50% for cache hits
- Maintains thread safety with minimal lock scope
This addresses the performance issue where contract extraction
was extremely slow (5+ hours) by eliminating redundant file I/O.
* docs: update CHANGELOG for 0.23.1 with contract extraction performance fixes
* Fix slowness bug in contract extraction
* fix: correct SourceTracking import in profiling script
- Changed import from specfact_cli.models.project to specfact_cli.models.source_tracking
- Fixes CrossHair import error in contract validation workflow
- Resolves ImportError: cannot import name 'SourceTracking'
* Fix parallel processing of contract analysis
* Revert venv config
---------
Co-authored-by: Dominikus Nold <[email protected]>
---------
Signed-off-by: Dom <[email protected]>
Co-authored-by: Dominikus Nold <[email protected]>
* feat: integrate sidecar validation into CLI
- Add sidecar validation commands (validate sidecar init/run)
- Integrate framework detection (Django, FastAPI, DRF, pure Python)
- Add route/schema extraction and contract population
- Implement CrossHair and Specmatic validation runners
- Add environment manager detection (venv, poetry, uv, hatch, pip)
- Update documentation (README, guides, command reference)
- Add comprehensive test coverage (unit, integration, E2E)
- Update version to 0.24.0 and CHANGELOG
Closes #97
* feat: add CrossHair summary, Specmatic auto-skip, and repro sidecar integration
- Add CrossHair summary reporting with JSON file generation
- Add Specmatic auto-skip when no service configuration detected
- Integrate sidecar validation into repro command with --sidecar flag
- Add AST-based unannotated code detection
- Add safe defaults for repro mode (TimeoutConfig.safe_defaults_for_repro)
- Add deterministic inputs support for CrossHair
- Update documentation for all new features
- Add comprehensive test coverage (35 new tests)
Phases 9, 10, and 11 complete:
- Phase 9: CrossHair Summary Reporting ✅
- Phase 10: Specmatic Auto-Skip ✅
- Phase 11: Repro Integration ✅
Closes #55, #56, #57
* chore: remove accidentally committed harness_contracts.py
* fix: sanitize harness function names and preserve PATH in CrossHair runner
- Fix harness generator to sanitize all non-identifier characters (/, {, }, etc.)
in operation IDs to create valid Python function names
- Fix CrossHair runner to preserve PATH and other environment variables
by using os.environ.copy() as base instead of empty dict
- Add unit tests for both fixes
Fixes issues identified in code review:
- P2: Badge Sanitize operationId-derived harness names
- P1: Badge Preserve PATH when launching CrossHair
* test: improve harness sanitization tests
* fix: explicitly define boolean option flags for Typer help text
- Add --run-crosshair/--no-run-crosshair syntax to make both options visible
- Add --run-specmatic/--no-run-specmatic syntax for consistency
- Fixes test_validate_sidecar_run_help assertion failure
Typer only shows --no-flag in help for boolean options with default=True
unless both flags are explicitly specified using --flag/--no-flag syntax.
---------
Signed-off-by: Dom <[email protected]>
Co-authored-by: Dominikus Nold <[email protected]>1 parent 61b4e88 commit e98cd2c
File tree
53 files changed
+5138
-16
lines changed- docs
- _layouts
- guides
- reference
- src
- specfact_cli
- commands
- validators/sidecar
- frameworks
- tests
- e2e
- integration
- commands
- specfact_cli/validators/sidecar
- unit/specfact_cli/validators/sidecar
- frameworks
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
53 files changed
+5138
-16
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
12 | 145 | | |
13 | 146 | | |
14 | 147 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
93 | 93 | | |
94 | 94 | | |
95 | 95 | | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
96 | 100 | | |
97 | 101 | | |
98 | 102 | | |
| |||
157 | 161 | | |
158 | 162 | | |
159 | 163 | | |
| 164 | + | |
160 | 165 | | |
161 | | - | |
| 166 | + | |
| 167 | + | |
162 | 168 | | |
163 | 169 | | |
164 | 170 | | |
| |||
188 | 194 | | |
189 | 195 | | |
190 | 196 | | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
191 | 203 | | |
192 | 204 | | |
193 | 205 | | |
| |||
227 | 239 | | |
228 | 240 | | |
229 | 241 | | |
| 242 | + | |
230 | 243 | | |
231 | 244 | | |
232 | 245 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
190 | 190 | | |
191 | 191 | | |
192 | 192 | | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
193 | 199 | | |
194 | 200 | | |
195 | 201 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
141 | 141 | | |
142 | 142 | | |
143 | 143 | | |
| 144 | + | |
144 | 145 | | |
145 | 146 | | |
146 | 147 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
45 | 48 | | |
46 | 49 | | |
47 | 50 | | |
| |||
298 | 301 | | |
299 | 302 | | |
300 | 303 | | |
301 | | - | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
302 | 363 | | |
303 | 364 | | |
304 | 365 | | |
| |||
349 | 410 | | |
350 | 411 | | |
351 | 412 | | |
352 | | - | |
| 413 | + | |
353 | 414 | | |
354 | 415 | | |
355 | 416 | | |
| |||
402 | 463 | | |
403 | 464 | | |
404 | 465 | | |
405 | | - | |
| 466 | + | |
406 | 467 | | |
407 | 468 | | |
408 | 469 | | |
| |||
453 | 514 | | |
454 | 515 | | |
455 | 516 | | |
456 | | - | |
| 517 | + | |
457 | 518 | | |
458 | 519 | | |
459 | 520 | | |
| |||
506 | 567 | | |
507 | 568 | | |
508 | 569 | | |
509 | | - | |
| 570 | + | |
510 | 571 | | |
511 | 572 | | |
512 | 573 | | |
| |||
558 | 619 | | |
559 | 620 | | |
560 | 621 | | |
561 | | - | |
| 622 | + | |
562 | 623 | | |
563 | 624 | | |
564 | 625 | | |
| |||
670 | 731 | | |
671 | 732 | | |
672 | 733 | | |
| 734 | + | |
673 | 735 | | |
674 | 736 | | |
675 | 737 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
160 | 188 | | |
161 | 189 | | |
162 | 190 | | |
| |||
0 commit comments