Skip to content

Bug: Enrichment application causes performance regression and unhashable type errors #95

@djm81

Description

@djm81

Describe the Bug

When applying enrichment reports to plan bundles, several critical bugs were encountered:

  1. Unhashable Type Error: TypeError: unhashable type: 'Feature' occurred during contract extraction when using Feature objects as dictionary keys
  2. Performance Regression: Applying enrichment reports forced full contract regeneration for all features, even when only metadata changed
  3. Contract Extraction Order: New features from enrichment were not present during contract extraction due to incorrect execution order
  4. Force Flag Inefficiency: The --force flag still performed expensive hash checking instead of directly processing all features
  5. UI Progress Bar Disappearance: Progress bars during enhanced analysis would disappear leaving blank lines

To Reproduce

Steps to reproduce the behavior:

# Import a large codebase
specfact import from-code sqlalchemy-validation --repo ~/git/specfact-validation/sqlalchemy

# Apply enrichment report
specfact import from-code sqlalchemy-validation --repo ~/git/specfact-validation/sqlalchemy \
  --enrichment ~/git/specfact-validation/sqlalchemy/.specfact/projects/sqlalchemy-validation/reports/enrichment/*.enrichment.md \
  --force

Example:

  • Import SQLAlchemy codebase (700+ features)
  • Apply enrichment report with --enrichment flag
  • Observe TypeError: unhashable type: 'Feature' error during contract extraction
  • Observe full contract regeneration taking 30+ minutes even when only metadata changed
  • Observe progress bars disappearing and leaving blank lines

Expected Behavior

  • Enrichment should only trigger contract extraction for new features, not all existing ones
  • Feature objects should not be used as dictionary keys (use string keys instead)
  • Enrichment should be applied before contract extraction
  • --force flag should skip hash checking and directly process all features
  • Progress bars should remain visible with final completion state

Actual Behavior

  • Full contract regeneration triggered on enrichment application (30+ minutes for 320 features)
  • TypeError: unhashable type: 'Feature' when using Feature objects as dictionary keys
  • Contract extraction occurred before enrichment application, missing new features
  • --force flag still performed expensive hash checking for all features
  • Progress bars disappeared leaving blank lines during enhanced analysis

Environment

  • OS: Linux
  • Python Version: 3.11+
  • SpecFact CLI Version: 0.23.0 (before fix)
  • Installation Method: from source (hatch)

Command Output

TypeError: unhashable type: 'Feature'
  File "src/specfact_cli/commands/import_cmd.py", line 862, in _extract_contracts
    feature_to_files: dict[Feature, list[Path]] = {}

Performance regression observed:

  • 320 features, 5 minutes but only 11% complete
  • 30+ minutes on extracting contracts after applying enrichment with --force

Codebase Context (for brownfield issues)

  • Project Type: SQLAlchemy (ORM framework)
  • Codebase Size: ~700 features, ~3000+ files
  • Python Version in Target Codebase: 3.11+

Additional Context

These bugs were discovered during validation of the SQLAlchemy codebase (700+ features). The performance regression was particularly severe, causing contract extraction to take 30+ minutes when it should have been much faster for metadata-only changes.

Resolution: Fixed in PR #94 (version 0.23.1):

  • Changed feature_to_files to use string keys instead of Feature objects
  • Modified incremental change detection to not force full regeneration on enrichment
  • Reordered enrichment application to occur before contract extraction
  • Optimized --force flag to skip hash checking
  • Fixed progress bar visibility issues

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or request

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions