Skip to content

Pair expected fixtures with canonical sources - #3

Merged
tmusser merged 7 commits into
mainfrom
agent/pair-fixtures-with-sources
Jul 10, 2026
Merged

Pair expected fixtures with canonical sources#3
tmusser merged 7 commits into
mainfrom
agent/pair-fixtures-with-sources

Conversation

@tmusser

@tmusser tmusser commented Jul 10, 2026

Copy link
Copy Markdown
Owner

What changed

  • added one canonical numbered *.source.txt file for each *.expected.json fixture
  • standardized fixture anchors to Line N and Lines N-M
  • made the schema validator require one-to-one source/expected pairs
  • validate that every record anchor resolves inside the paired source
  • reject anchored records with no substantive language overlap
  • require follow-up actions to remain explicitly proposed
  • documented the fixture-pair convention in examples/README.md

Why

The existing JSON fixtures were schema-valid and semantically plausible, but they were not paired with canonical source inputs. Anchors such as Thread or Alex line 5 could pass even when no validator proved that the referenced source existed or supported the record.

This change turns the fixtures from output-shape examples into grounded source/output evidence.

Pair contract

<name>.source.txt
<name>.expected.json

Source files contain sequentially numbered lines. Expected records must use Line N or Lines N-M, reference existing lines, and retain substantive overlap with those lines.

Validation

All checks passed on the final head:

  • installer smoke tests
  • repository structure validation
  • example validation
  • schema and fixture-pair validation
  • numbered source parsing
  • anchor format and range validation
  • substantive source-overlap checks
  • fixture-specific classification rules
  • diff formatting

@tmusser
tmusser marked this pull request as ready for review July 10, 2026 19:31
@tmusser
tmusser merged commit efae294 into main Jul 10, 2026
2 checks passed
@tmusser
tmusser deleted the agent/pair-fixtures-with-sources branch July 10, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant