Skip to content

Commit e98cd2c

Browse files
authored
feat: integrate sidecar validation into CLI (#98)
* Merge dev into main (#96) * docs: improve documentation structure with unified command chains and cross-linking (#79) Co-authored-by: Dominikus Nold <[email protected]> * docs: add integrations overview guide (optional task 6.4) - Create integrations-overview.md with comprehensive overview of all integrations - Add links from integration guides to integrations-overview.md - Add link to integrations-overview.md in docs/README.md - Complete optional task 6.4 from improve-documentation-structure change * docs: fix linting errors in integrations-overview.md - Fix MD036 warnings by converting emphasis to proper headings - Fix MD040 warning by adding language specifier to code block * docs: simplify README and add links to new documentation - Update website links to specfact.com / .io / .dev - Add GitHub Pages docs link: https://nold-ai.github.io/specfact-cli/ - Remove version info section (avoids outdated info) - Simplify content - remove verbose sections, add links to docs instead - Add links to new documentation: - Command Chains Reference - Common Tasks Quick Reference - AI IDE Workflow Guide - Integrations Overview - Improve onboarding with clear path for new users * docs: add prominent SpecFact domain links with context - Add specfact.com, specfact.io, specfact.dev links prominently at top - Add domain purpose context (commercial, ecosystem, developer community) - Highlight specfact.dev for developers - Add GitHub Pages docs link - Improve user navigation to appropriate resources * docs: update Quick Start with correct IDE setup workflow - Add Step 2: Initialize IDE integration (specfact init --ide) - Update Step 3: Use slash commands in IDE or CLI - Add realistic timing expectations (10-15 min for typical repos) - Explain what init does (copies prompts, makes slash commands available) - Add link to AI IDE Workflow Guide - Remove unrealistic '60 seconds' claim * fix: correct heading level for SpecFact Domains section * docs: fix GitHub Pages permalinks for all documentation pages - Update permalinks to include full directory path (e.g., /reference/commands/ instead of /commands/) - Add frontmatter with permalinks to agile-scrum-workflows.md and reference/README.md - Add frontmatter with permalink to speckit-journey.md for consistency - All permalinks now match the Jekyll configuration pattern - Enables proper GitHub Pages URLs for platform-frontend sites * fix: resolve Jekyll build errors for GitHub Pages - Quote title in speckit-journey.md frontmatter to fix YAML parsing error - Wrap Jinja2 template code in {% raw %} tags in agile-scrum-workflows.md to prevent Jekyll from parsing it as Liquid syntax Fixes GitHub Pages build errors: - YAML Exception in speckit-journey.md (line 3) - Liquid syntax error in agile-scrum-workflows.md (line 708) * docs: add new pages to GitHub Pages navigation menu - Add Command Chains to Guides section - Add Agile/Scrum Workflows to Guides section - Add Reference Documentation index to Reference section These pages were missing from the navigation menu after fixing permalinks. * docs: add new pages to GitHub Pages sidebar navigation menu - Add Command Chains to Guides section (top of list) - Add Agile/Scrum Workflows to Guides section - Add Reference Documentation index to Reference section - Fix reference links to use correct permalinks (/reference/architecture/, etc.) The sidebar navigation menu is hardcoded in the layout file, so these pages need to be manually added to appear in the left sidebar. * feat: add Mermaid.js support for diagram rendering on GitHub Pages - Add Mermaid.js CDN script to layout - Add JavaScript to convert mermaid code blocks to renderable divs - Handle kramdown output format (pre > code.language-mermaid) - Initialize Mermaid with proper configuration Fixes Mermaid diagram rendering on GitHub Pages documentation. All mermaid code blocks will now render as interactive diagrams. * feat: align GitHub Pages styling with specfact.io design - Update color scheme to match specfact.io (dark theme with cyan accent) - Change primary colors: #64ffda (cyan), #0a192f (dark blue), #112240 (light dark) - Update Mermaid theme to dark with custom colors matching specfact.io - Add Inter and JetBrains Mono fonts to match specfact.io typography - Add Mermaid-specific CSS styling for better diagram appearance - Remove light mode support, use dark theme consistently Colors now match specfact.io: - Primary/Highlight: #64ffda (cyan) - Background: #0a192f (dark blue) - Text: #ccd6f6 (light blue-gray) - Code background: #1d2d50 (darker blue) Mermaid diagrams now use dark theme with cyan accents for better readability and visual consistency with specfact.io documentation site. * fix: improve YAML and code syntax highlighting for dark theme - Update Rouge syntax highlighting colors for dark theme readability - Use cyan (#64ffda) for literals, numbers, and constants - Use light green (#a8e6cf) for strings (better contrast on dark) - Use pink (#ff6b9d) for keywords and operators - Use purple (#c792ea) for functions and classes - Use yellow (#ffd93d) for variables - Use muted gray-blue (#8892b0) for comments - YAML keys now use cyan color for better visibility Fixes readability issues with YAML and other code blocks on dark background. * feat: update to custom domain docs.specfact.io - Update _config.yml: set baseurl to empty string for custom domain - Update _config.yml: set url to https://docs.specfact.io - Exclude assets/ from default permalink pattern to fix CSS path - Update README.md to use new docs.specfact.io domain - Fixes CSS 404 errors on custom domain * docs: add CNAME file for GitHub Pages custom domain Required for GitHub Pages to recognize docs.specfact.io as custom domain. This file must be in the repository root (not in docs/). * fix: update root _config.yml for custom domain - Set baseurl to empty string for custom domain - Set url to https://docs.specfact.io - Exclude assets/ from permalink pattern to fix CSS paths - This file is copied to docs/ by GitHub Pages workflow, so it must match docs/_config.yml Fixes CSS 404 error: /specfact-cli/assets/main.css -> /assets/main.css * fix: correct Reference navigation link in top menu - Change from /commands/ to /reference/ to match actual permalink - Fixes broken link in upper navigation menu * feat: Version 0.23.0 - Performance optimizations and progress reporting for large codebases (#92) * fix: enrichment application and contract extraction fixes (#94) * fix: enrichment application and contract extraction improvements - Fix enrichment not being applied when no source files changed - Check for enrichment before early exit in _check_incremental_changes - Mark bundle for regeneration when enrichment is provided - Ensure bundle is saved after enrichment is applied - Fix --force flag performance regression - Skip hash checking when --force is used for contract extraction - Process all features directly without expensive hash computation - Significantly improves performance for large bundles with --force - Fix type checking errors - Fix possibly unbound variables (is_test_mode, compute_file_hash, features_with_files) - Properly scope variables within conditional blocks - Ensure all variables are initialized before use - Add comprehensive test coverage - Add integration tests for enrichment and contract extraction bugs - Add unit tests for contract extraction logic - All tests now passing (11/11 integration tests) - Update version to 0.23.1 - Sync version across pyproject.toml, setup.py, src/__init__.py, src/specfact_cli/__init__.py - Update CHANGELOG.md with bug fixes and test coverage additions * fix: keep progress bars visible during enhanced analysis - Remove progress.remove_task() calls for relationship and graph analysis - Keep final progress bars visible with completion state instead of removing them - Prevents blank lines from appearing when progress bars disappear - Progress bars now show final completion message and remain visible * feat: show current feature/contract in contract extraction progress - Add detailed progress updates showing which feature is currently being processed - For sequential mode: show feature name before and after processing - For parallel mode: show completed feature name and pending count - Progress now displays: 'Extracting contract from FEATURE-NAME... (X/Total, Y pending)' - Improves visibility during long-running contract extraction operations - Helps identify which features are taking longer to process * docs: add contract extraction performance analysis - Document current performance bottlenecks - Identify AST parsing as primary bottleneck - Propose file-level caching optimization (3-5x speedup) - Suggest batch processing and early exit optimizations - Estimate 5-10x overall speedup potential * perf: implement AST caching and early exit optimizations for contract extraction - Add file-level AST caching to prevent redundant parsing (3-5x speedup) - Cache AST trees and file hashes for reuse across features - Invalidate cache when file content changes - Thread-safe cache operations - Add early exit optimization for non-API files (1.5-2x speedup) - Quick regex check before expensive AST parsing - Skip files without API endpoints (models, utilities) - Pre-compiled regex patterns for performance - Add comprehensive tests for optimizations: - Test AST caching prevents redundant parsing - Test early exit skips non-API files - Test cache invalidation on file changes Expected overall improvement: 5-10x speedup for contract extraction For SQLAlchemy (320 features): ~8 minutes -> ~45-90 seconds * fix: disable aggressive early exit that skipped all SQLAlchemy files - Early exit optimization was too aggressive for ORM/class-based codebases - SQLAlchemy doesn't use FastAPI/Flask decorators, so all files were skipped - Contract extractor also processes class-based APIs and interfaces - Disabled early exit to restore functionality (AST caching still provides 3-5x speedup) - Updated test to reflect that early exit detection works but is disabled in extraction Fixes: 0 contracts generated when using --force flag * perf: optimize class-based extraction to skip non-API classes and limit method processing - Skip non-API class types: Protocol, TypedDict, Enum, ABC, Mixin, Base, Meta, Descriptor, Property - Skip classes that inherit from non-API base types - Filter methods more selectively: skip utility methods (processor, adapter, factory, etc.) - Limit methods processed per class to 15 (skip classes with more methods) - Only process methods that strongly suggest API endpoints (CRUD patterns or short names) Performance improvements: - FEATURE-TYPERESOLVE: Skips Protocol/TypedDict classes and TypeEngine utility methods - FEATURE-COLLECTIONADAPTER: Skips non-API classes, processes only relevant methods - Reduces processing time for large ORM/library codebases Expected improvement: 2-3x faster for features with many utility classes * Apply format * fix: resolve CI test failures and deprecation warnings - Fix interface extraction: Check for interfaces (ABC/Protocol with abstract methods) BEFORE skipping base classes - Interfaces should be processed for contract extraction - Non-interface ABC/Protocol classes are still skipped for performance - Fix progress callback tests: Update tests to expect two calls (total, then completed+description) - Progress callback now sets total on first call, then updates with completed count - Fix deprecation warnings: - Suppress ast.NameConstant deprecation warning (Python 3.8+ compatibility) - Replace datetime.utcnow() with datetime.now(UTC) for Python 3.11+ compatibility - Use timezone.utc fallback for older Python versions Fixes: - test_extract_interface_abstract_methods (was skipping ABC interfaces) - test_create_callback_with_prefix (expected single call, got two) - test_create_callback_without_prefix (expected single call, got two) - DeprecationWarning: ast.NameConstant (Python 3.14) - DeprecationWarning: datetime.utcnow() (future removal) * fix: correct UTC import for type checking - Import timezone before try/except block to ensure UTC is defined - Add type: ignore comment to suppress false positive type checker warning - Fixes type checker error: 'UTC' is unbound in except block - Maintains backward compatibility with Python < 3.11 * perf: reduce lock contention in contract extraction for parallel processing Critical performance fix for large feature sets (320+ features): 1. **Moved file I/O outside lock**: File reading and hash calculation now happen outside the lock, eliminating I/O-bound blocking - Lock only held for cache lookups and updates (minimal scope) - Double-check pattern prevents race conditions 2. **Removed unnecessary locks from openapi_spec writes**: - Each feature has its own openapi_spec dict (no sharing) - Python dict assignment is atomic for single operations - Removed locks from: path initialization, schema addition, security schemes, operation addition - Only cache operations (shared across features) use lock now 3. **Separated cache lock**: Renamed _lock to _cache_lock for clarity - Cache is shared resource (needs protection) - openapi_spec dicts are per-feature (no shared lock needed) Performance impact: - Before: Lock held during file I/O (10-100ms per file) blocks all other threads - After: Lock only held for cache access (<1ms), file I/O happens in parallel - Expected: 5-10x faster for 320 features with parallel processing This fixes the 3-hour extraction time for 320 contracts by eliminating lock contention bottleneck. * perf: optimize AST cache to avoid redundant file reads - Reuse file content when checking hash vs parsing - Read file only once per cache check/parse cycle - Reduces I/O operations by 50% for cache hits - Maintains thread safety with minimal lock scope This addresses the performance issue where contract extraction was extremely slow (5+ hours) by eliminating redundant file I/O. * docs: update CHANGELOG for 0.23.1 with contract extraction performance fixes * Fix slowness bug in contract extraction * fix: correct SourceTracking import in profiling script - Changed import from specfact_cli.models.project to specfact_cli.models.source_tracking - Fixes CrossHair import error in contract validation workflow - Resolves ImportError: cannot import name 'SourceTracking' * Fix parallel processing of contract analysis * Revert venv config --------- Co-authored-by: Dominikus Nold <[email protected]> --------- Signed-off-by: Dom <[email protected]> Co-authored-by: Dominikus Nold <[email protected]> * feat: integrate sidecar validation into CLI - Add sidecar validation commands (validate sidecar init/run) - Integrate framework detection (Django, FastAPI, DRF, pure Python) - Add route/schema extraction and contract population - Implement CrossHair and Specmatic validation runners - Add environment manager detection (venv, poetry, uv, hatch, pip) - Update documentation (README, guides, command reference) - Add comprehensive test coverage (unit, integration, E2E) - Update version to 0.24.0 and CHANGELOG Closes #97 * feat: add CrossHair summary, Specmatic auto-skip, and repro sidecar integration - Add CrossHair summary reporting with JSON file generation - Add Specmatic auto-skip when no service configuration detected - Integrate sidecar validation into repro command with --sidecar flag - Add AST-based unannotated code detection - Add safe defaults for repro mode (TimeoutConfig.safe_defaults_for_repro) - Add deterministic inputs support for CrossHair - Update documentation for all new features - Add comprehensive test coverage (35 new tests) Phases 9, 10, and 11 complete: - Phase 9: CrossHair Summary Reporting ✅ - Phase 10: Specmatic Auto-Skip ✅ - Phase 11: Repro Integration ✅ Closes #55, #56, #57 * chore: remove accidentally committed harness_contracts.py * fix: sanitize harness function names and preserve PATH in CrossHair runner - Fix harness generator to sanitize all non-identifier characters (/, {, }, etc.) in operation IDs to create valid Python function names - Fix CrossHair runner to preserve PATH and other environment variables by using os.environ.copy() as base instead of empty dict - Add unit tests for both fixes Fixes issues identified in code review: - P2: Badge Sanitize operationId-derived harness names - P1: Badge Preserve PATH when launching CrossHair * test: improve harness sanitization tests * fix: explicitly define boolean option flags for Typer help text - Add --run-crosshair/--no-run-crosshair syntax to make both options visible - Add --run-specmatic/--no-run-specmatic syntax for consistency - Fixes test_validate_sidecar_run_help assertion failure Typer only shows --no-flag in help for boolean options with default=True unless both flags are explicitly specified using --flag/--no-flag syntax. --------- Signed-off-by: Dom <[email protected]> Co-authored-by: Dominikus Nold <[email protected]>
1 parent 61b4e88 commit e98cd2c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+5138
-16
lines changed

CHANGELOG.md

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,139 @@ All notable changes to this project will be documented in this file.
99

1010
---
1111

12+
## [0.24.0] - 2026-01-09
13+
14+
### Added (0.24.0)
15+
16+
- **Sidecar Validation CLI Integration**: Native CLI integration for sidecar validation workflow
17+
- **New Command**: `specfact validate sidecar init <bundle-name> <repo-path>` - Initialize sidecar workspace for validation
18+
- **New Command**: `specfact validate sidecar run <bundle-name> <repo-path>` - Run complete sidecar validation workflow
19+
- **Framework Detection**: Automatic detection of Django, FastAPI, DRF, and pure Python frameworks
20+
- **Route Extraction**: Framework-specific route and schema extraction (Django URLs, FastAPI routes, DRF serializers)
21+
- **Contract Population**: Automatic population of OpenAPI contracts with extracted routes and schemas
22+
- **Harness Generation**: CrossHair harness generation from populated contracts
23+
- **Tool Execution**: Integration with CrossHair symbolic execution and Specmatic contract testing
24+
- **Environment Manager Support**: Automatic detection and use of hatch, poetry, uv, and pip environments
25+
- **Venv Detection**: Automatic detection and configuration of `.venv` and `venv` virtual environments
26+
- **Progress Reporting**: Rich console progress indicators for long-running operations
27+
- **Backward Compatibility**: Full compatibility with existing template-based sidecar workspaces
28+
29+
- **CrossHair Summary Reporting**: Enhanced CrossHair output parsing and reporting
30+
- **Summary Parser**: New `crosshair_summary.py` module for parsing CrossHair stdout/stderr
31+
- **Summary File Generation**: Automatic generation of `crosshair-summary-{timestamp}.json` files
32+
- **Console Display**: Formatted summary line showing confirmed/not confirmed/violations counts
33+
- **Integration**: Summary parsing integrated into sidecar validation orchestrator
34+
- **Testing**: Comprehensive unit tests (15 tests) covering various output scenarios
35+
36+
- **Specmatic Auto-Skip**: Intelligent detection and skipping of Specmatic when no service is available
37+
- **Service Detection**: New `has_service_configuration()` function to detect available service endpoints
38+
- **Auto-Skip Logic**: Specmatic automatically skipped when no `test_base_url`, `host`/`port`, or app server configuration detected
39+
- **Clear Messaging**: User-friendly warning messages when Specmatic is skipped
40+
- **Manual Override**: `--run-specmatic` flag to force execution even without service configuration
41+
- **Testing**: Unit tests (8 tests) for service detection and auto-skip logic
42+
- **Documentation**: Updated sidecar validation guide and command reference with auto-skip behavior
43+
44+
- **Repro Sidecar Integration**: Sidecar validation integrated into `specfact repro` command
45+
- **New Options**: `--sidecar` and `--sidecar-bundle <name>` options for repro command
46+
- **Unannotated Code Detection**: AST-based detection of functions without icontract/beartype decorators
47+
- New `unannotated_detector.py` module with AST parsing
48+
- Detects unannotated functions across repositories
49+
- Skips test files and harness files automatically
50+
- Comprehensive unit tests (7 tests)
51+
- **Safe Defaults**: Automatic application of safe timeout defaults for repro mode
52+
- `TimeoutConfig.safe_defaults_for_repro()` method with conservative timeouts
53+
- CrossHair timeout: 30s (vs 60s default)
54+
- Per-path timeout: 5s
55+
- Per-condition timeout: 2s
56+
- Unit tests (2 tests) for safe defaults
57+
- **Deterministic Inputs**: Support for deterministic inputs from harness `inputs.json`
58+
- `use_deterministic_inputs` flag in CrossHairConfig
59+
- Per-path and per-condition timeout support in CrossHair runner
60+
- Automatic application in repro mode
61+
- **Integration Tests**: Comprehensive integration tests (3 tests) for repro sidecar workflow
62+
- **Documentation**: Updated command reference and sidecar validation guide with repro integration examples
63+
64+
- **New Python Modules**: Complete sidecar validation package
65+
- `src/specfact_cli/validators/sidecar/` - Sidecar validation orchestrator and utilities
66+
- `src/specfact_cli/validators/sidecar/frameworks/` - Framework-specific extractors (Django, FastAPI, DRF)
67+
- `src/specfact_cli/commands/validate.py` - Validation command group with sidecar subcommands
68+
- `src/specfact_cli/validators/sidecar/crosshair_summary.py` - CrossHair output parsing and summary generation
69+
- `src/specfact_cli/validators/sidecar/unannotated_detector.py` - AST-based unannotated code detection
70+
- `tests/unit/specfact_cli/validators/sidecar/test_crosshair_summary.py` - Summary parser tests
71+
- `tests/unit/specfact_cli/validators/sidecar/test_specmatic_runner_auto_skip.py` - Auto-skip logic tests
72+
- `tests/unit/specfact_cli/validators/sidecar/test_unannotated_detector.py` - Unannotated detection tests
73+
- `tests/unit/specfact_cli/validators/sidecar/test_timeout_config_safe_defaults.py` - Safe defaults tests
74+
- `tests/integration/commands/test_repro_sidecar.py` - Repro sidecar integration tests
75+
76+
- **Environment Manager Integration**: Enhanced environment detection for sidecar validation
77+
- Venv detection and Python path configuration
78+
- PYTHONPATH building with venv site-packages, source directories, and repo root
79+
- Tool execution with environment manager prefixes (hatch run, poetry run, uv run)
80+
81+
- **Testing**: Comprehensive test coverage (69 sidecar-related tests, ≥80% coverage)
82+
- Unit tests for all framework extractors and core workflow components
83+
- Integration tests for CLI commands and backward compatibility
84+
- End-to-end tests for complete validation workflows
85+
- Verification tests against real-world repositories
86+
- 35 new tests for CrossHair summary, Specmatic auto-skip, unannotated detection, safe defaults, and repro integration
87+
88+
### Changed (0.24.0)
89+
90+
- **Init Command**: Updated `specfact init --install-deps` to include sidecar validation tools
91+
- Added comment about sidecar validation tools (crosshair-tool already included)
92+
- Note: specmatic may need separate installation (Java-based tool)
93+
94+
- **CrossHair Runner**: Enhanced with per-path and per-condition timeout support
95+
- Added `per_path_timeout` and `per_condition_timeout` parameters
96+
- Support for deterministic inputs via `inputs_path` parameter
97+
- Improved timeout handling for long-running symbolic execution paths
98+
99+
- **Sidecar Orchestrator**: Extended to support unannotated code detection
100+
- Added `unannotated_functions` parameter to `run_sidecar_validation()`
101+
- Integration with CrossHair summary generation
102+
- Enhanced result dictionary with unannotated function information
103+
104+
- **Repro Command**: Extended with sidecar validation integration
105+
- Automatic unannotated code detection when `--sidecar` flag is used
106+
- Automatic application of safe defaults for repro mode
107+
- Sidecar results displayed in repro output
108+
109+
- **TimeoutConfig Model**: Added safe defaults factory method
110+
- New `safe_defaults_for_repro()` class method
111+
- Conservative timeout values for repro mode
112+
- Per-path and per-condition timeout defaults
113+
114+
- **CrossHairConfig Model**: Added configuration flags
115+
- `use_deterministic_inputs` flag for deterministic input support
116+
- `safe_defaults` flag for safe default application
117+
118+
### Fixed (0.24.0)
119+
120+
- **Flask Framework Detection**: Fixed incorrect detection of Flask as Django
121+
- Added Flask pattern detection before Django `urls.py` check
122+
- Framework detection accuracy improved from 85.7% (6/7) to 100% (7/7)
123+
- Flask correctly detected as `PURE_PYTHON`
124+
125+
### Documentation (0.24.0)
126+
127+
- **New Guides**: Added comprehensive documentation for sidecar validation
128+
- `docs/guides/sidecar-validation.md` - Complete user guide with examples
129+
- `docs/reference/commands.md` - Updated with `validate sidecar` commands
130+
- Verification and test results documentation
131+
132+
- **Sidecar Validation Guide**: Updated with new features
133+
- Auto-skip behavior documentation with examples
134+
- Troubleshooting section for Specmatic auto-skip
135+
- Repro integration section with usage examples
136+
- Safe defaults documentation
137+
138+
- **Command Reference**: Updated with repro sidecar options
139+
- Added `--sidecar` and `--sidecar-bundle` options to repro command
140+
- Updated examples with sidecar usage
141+
- Added sidecar to tool requirements list
142+
143+
---
144+
12145
## [0.23.1] - 2026-01-07
13146

14147
### Fixed (0.23.1)

README.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,10 @@ specfact init --ide cursor --install-deps
9393
```bash
9494
# Analyze legacy codebase (most common use case)
9595
specfact import from-code my-project --repo .
96+
97+
# Or validate external codebase without modifying source (sidecar validation)
98+
specfact validate sidecar init my-project /path/to/repo
99+
specfact validate sidecar run my-project /path/to/repo
96100
```
97101

98102
**⏱️ Timing:** Analysis typically takes **10-15 minutes** for typical repositories (e.g., `specfact-cli` itself with several hundred features & contracts). Smaller codebases may complete in 2-5 minutes. Large codebases (3000+ features) may take 15-30 minutes, but progress reporting shows real-time status. The analysis performs AST parsing, Semgrep pattern detection, and Specmatic integration.
@@ -157,8 +161,10 @@ specfact import from-code my-project --repo .
157161
- **Prevent regressions** with runtime contract validation
158162
- **CI/CD integration** - Block bad code from merging
159163
- **Works offline** - No cloud required
164+
- **Sidecar validation** - Validate external codebases without modifying source code
160165

161-
👉 **[Command Reference](docs/reference/commands.md)** - All enforcement commands
166+
👉 **[Command Reference](docs/reference/commands.md)** - All enforcement commands
167+
👉 **[Sidecar Validation Guide](docs/guides/sidecar-validation.md)** - Validate external codebases
162168

163169
### 👥 Team Collaboration
164170

@@ -188,6 +194,12 @@ specfact import from-code my-project --repo .
188194

189195
👉 **[Brownfield Modernization Guide](docs/guides/brownfield-engineer.md)** - Complete walkthrough
190196

197+
### 1.5. Validating External Codebases (Sidecar Validation) 🆕
198+
199+
**Problem:** Need to validate third-party libraries or legacy codebases without modifying source code
200+
201+
👉 **[Sidecar Validation Guide](docs/guides/sidecar-validation.md)** - Validate external codebases with contract testing
202+
191203
### 2. Working with a Team
192204

193205
**Problem:** Need team collaboration with role-based workflows
@@ -227,6 +239,7 @@ specfact import from-code my-project --repo .
227239
- **[AI IDE Workflow](docs/guides/ai-ide-workflow.md)****NEW** - AI-assisted development
228240
- **[Agile/Scrum Workflows](docs/guides/agile-scrum-workflows.md)** ⭐ - Team collaboration
229241
- **[Integrations Overview](docs/guides/integrations-overview.md)****NEW** - All integrations
242+
- **[Sidecar Validation](docs/guides/sidecar-validation.md)** 🆕 - Validate external codebases without modifying source
230243
- **[Use Cases](docs/guides/use-cases.md)** - Common scenarios
231244

232245
### Integration Guides

docs/README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,6 +190,12 @@ specfact enforce sdd --bundle my-project
190190
- [Dependency Management](guides/agile-scrum-workflows.md#dependency-management) - Track story and feature dependencies
191191
- [Conflict Resolution](guides/agile-scrum-workflows.md#conflict-resolution) - Persona-aware merge conflict resolution
192192

193+
#### Validation & Quality
194+
195+
- [Sidecar Validation](guides/sidecar-validation.md) 🆕 - Validate external codebases without modifying source code
196+
- [Contract Testing Workflow](guides/contract-testing-workflow.md) - Contract testing patterns
197+
- [Specmatic Integration](guides/specmatic-integration.md) - API contract testing
198+
193199
#### General Guides
194200

195201
- [UX Features](guides/ux-features.md) - Progressive disclosure, context detection, intelligent suggestions, templates

docs/_layouts/default.html

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,7 @@ <h2 class="docs-sidebar-title">
141141
<li><a href="{{ '/guides/agile-scrum-workflows/' | relative_url }}">Agile/Scrum Workflows</a></li>
142142
<li><a href="{{ '/brownfield-engineer/' | relative_url }}">Modernizing Legacy Code</a></li>
143143
<li><a href="{{ '/brownfield-journey/' | relative_url }}">Brownfield Journey</a></li>
144+
<li><a href="{{ '/guides/sidecar-validation/' | relative_url }}">Sidecar Validation</a></li>
144145
<li><a href="{{ '/ux-features/' | relative_url }}">UX Features</a></li>
145146
<li><a href="{{ '/use-cases/' | relative_url }}">Use Cases</a></li>
146147
<li><a href="{{ '/ide-integration/' | relative_url }}">IDE Integration</a></li>

docs/guides/command-chains.md

Lines changed: 70 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,9 @@ Command chains are sequences of SpecFact CLI commands that work together to achi
1616

1717
**Why use command chains?** Instead of learning individual commands in isolation, command chains show you how to combine commands to solve real-world problems. They provide context, decision points, and links to detailed guides.
1818

19-
This document covers all 9 identified command chains:
19+
This document covers all 10 identified command chains:
2020

21-
- **6 Mature Chains**: Well-established workflows with comprehensive documentation
21+
- **7 Mature Chains**: Well-established workflows with comprehensive documentation
2222
- **3 Emerging Chains**: AI-assisted workflows that integrate with IDE slash commands
2323

2424
---
@@ -42,6 +42,9 @@ Start: What do you want to accomplish?
4242
├─ Develop or validate API contracts?
4343
│ └─ → API Contract Development Chain
4444
45+
├─ Validate external codebase without modifying source?
46+
│ └─ → Sidecar Validation Chain
47+
4548
├─ Promote a plan through stages to release?
4649
│ └─ → Plan Promotion & Release Chain
4750
@@ -298,7 +301,65 @@ graph TD
298301

299302
---
300303

301-
## 5. Plan Promotion & Release Chain
304+
## 5. Sidecar Validation Chain
305+
306+
**Goal**: Validate external codebases (libraries, APIs, frameworks) without modifying source code.
307+
308+
**When to use**: You need to validate third-party libraries, legacy codebases, or APIs where you don't control the implementation.
309+
310+
**Command Sequence**:
311+
312+
```bash
313+
# Step 1: Initialize sidecar workspace
314+
specfact validate sidecar init <bundle-name> <repo-path>
315+
316+
# Step 2: Run sidecar validation workflow
317+
specfact validate sidecar run <bundle-name> <repo-path>
318+
319+
# Step 3: Review validation results
320+
# Results are saved to .specfact/projects/<bundle>/reports/sidecar/
321+
```
322+
323+
**Workflow Diagram**:
324+
325+
```mermaid
326+
graph TD
327+
A[External Repository] -->|validate sidecar init| B[Initialize Workspace]
328+
B --> C[Detect Framework]
329+
C --> D[Extract Routes/Schemas]
330+
D --> E[Populate Contracts]
331+
E --> F[Generate Harness]
332+
F -->|CrossHair| G[Symbolic Execution]
333+
F -->|Specmatic| H[Contract Testing]
334+
G --> I[Validation Results]
335+
H --> I
336+
```
337+
338+
**Decision Points**:
339+
340+
- **After `validate sidecar init`**: Review detected framework and configuration. Adjust if needed.
341+
- **Framework detection**: System automatically detects Django, FastAPI, DRF, or pure Python. Verify detection is correct.
342+
- **Tool execution**: Use `--no-run-crosshair` or `--no-run-specmatic` to skip specific tools if not needed.
343+
- **After `validate sidecar run`**: Review validation results. Fix issues in contracts or harness if needed.
344+
345+
**Expected Outcomes**:
346+
347+
- Sidecar workspace initialized with framework-specific configuration
348+
- Routes and schemas extracted from framework patterns
349+
- OpenAPI contracts populated with extracted data
350+
- CrossHair harness generated from contracts
351+
- Validation results (CrossHair analysis, Specmatic testing)
352+
- Reports saved to sidecar reports directory
353+
354+
**Related Guides**:
355+
356+
- [Sidecar Validation Guide](sidecar-validation.md) - Complete sidecar validation walkthrough
357+
- [Command Reference - Validate Commands](../reference/commands.md#validate---validation-commands) - Command reference
358+
- [Framework Detection](../reference/commands.md#framework-detection) - Supported frameworks
359+
360+
---
361+
362+
## 6. Plan Promotion & Release Chain
302363

303364
**Goal**: Promote a plan through stages (draft → review → approved → released) and manage versions.
304365

@@ -349,7 +410,7 @@ graph LR
349410

350411
---
351412

352-
## 6. Code-to-Plan Comparison Chain
413+
## 7. Code-to-Plan Comparison Chain
353414

354415
**Goal**: Detect and resolve drift between code and specifications.
355416

@@ -402,7 +463,7 @@ graph TD
402463

403464
---
404465

405-
## 7. AI-Assisted Code Enhancement Chain (Emerging)
466+
## 8. AI-Assisted Code Enhancement Chain (Emerging)
406467

407468
**Goal**: Use AI IDE integration to enhance code with contracts and validate them.
408469

@@ -453,7 +514,7 @@ graph TD
453514

454515
---
455516

456-
## 8. Test Generation from Specifications Chain (Emerging)
517+
## 9. Test Generation from Specifications Chain (Emerging)
457518

458519
**Goal**: Generate tests from specifications using AI assistance.
459520

@@ -506,7 +567,7 @@ graph TD
506567

507568
---
508569

509-
## 9. Gap Discovery & Fixing Chain (Emerging)
570+
## 10. Gap Discovery & Fixing Chain (Emerging)
510571

511572
**Goal**: Discover gaps in specifications and fix them using AI assistance.
512573

@@ -558,7 +619,7 @@ graph TD
558619

559620
---
560621

561-
## 10. SDD Constitution Management Chain
622+
## 11. SDD Constitution Management Chain
562623

563624
**Goal**: Manage Spec-Driven Development (SDD) constitutions for Spec-Kit compatibility.
564625

@@ -670,6 +731,7 @@ The following commands are now integrated into documented workflows:
670731
- [Command Reference](../reference/commands.md) - Complete command documentation
671732
- [Agile/Scrum Workflows](agile-scrum-workflows.md) - Team collaboration patterns
672733
- [Brownfield Engineer Guide](brownfield-engineer.md) - Legacy modernization guide
734+
- [Sidecar Validation Guide](sidecar-validation.md) - Validate external codebases
673735
- [Spec-Kit Journey](speckit-journey.md) - Spec-Kit integration
674736
- [OpenSpec Journey](openspec-journey.md) - OpenSpec integration
675737
- [Team Collaboration Workflow](team-collaboration-workflow.md) - Team collaboration guide

docs/guides/common-tasks.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,34 @@ specfact spec backward-compat --spec openapi.yaml --previous-spec openapi-v1.yam
157157

158158
---
159159

160+
### I want to validate an external codebase without modifying source
161+
162+
**Recommended**: [Sidecar Validation Chain](command-chains.md#5-sidecar-validation-chain)
163+
164+
**Command**: `validate sidecar init``validate sidecar run`
165+
166+
**Quick Example**:
167+
168+
```bash
169+
# Initialize sidecar workspace
170+
specfact validate sidecar init legacy-api /path/to/django-project
171+
172+
# Run validation workflow
173+
specfact validate sidecar run legacy-api /path/to/django-project
174+
```
175+
176+
**What it does**:
177+
178+
- Detects framework (Django, FastAPI, DRF, pure Python)
179+
- Extracts routes and schemas from framework patterns
180+
- Populates OpenAPI contracts automatically
181+
- Generates CrossHair harness for symbolic execution
182+
- Runs CrossHair and Specmatic validation
183+
184+
**Detailed Guide**: [Sidecar Validation Guide](sidecar-validation.md)
185+
186+
---
187+
160188
### I want to generate tests from API specifications
161189

162190
**Recommended**: `spec generate-tests`

0 commit comments

Comments
 (0)