Skip to content

Strengthen fixture coverage and simplify validation commands#11

Merged
GsCommand merged 1 commit intomainfrom
codex/fix-validation-surface-credibility
Mar 20, 2026
Merged

Strengthen fixture coverage and simplify validation commands#11
GsCommand merged 1 commit intomainfrom
codex/fix-validation-surface-credibility

Conversation

@GsCommand
Copy link
Contributor

Motivation

  • The v1.1.0 example surface contained copy-paste contaminated invalid fixtures (wrong verb, identical toy failures and placeholder hashes) that undermined validation credibility.
  • The goal was to make examples intentional: realistic valid receipts, single-failure invalid request coverage, and an unambiguous validation command topology so a new contributor can trust and run validation quickly.

Description

  • Replaced and expanded the entire JSON fixture set under examples/v1.1.0/commons/*/json/{valid,invalid} to provide per-verb disciplined coverage (minimal + richer valid requests, canonical ok+error receipts, and targeted invalid cases such as missing-verb, missing-input, empty-input, invalid-version, extra-property, invalid-mode, wrong-input-type, and explicit wrong-verb-literal cases). Representative files: examples/v1.1.0/commons/*/json/invalid/001-missing-verb.request.invalid.json and */json/valid/900-ok.receipt.valid.json.
  • Made valid receipts look operationally credible by using well-formed sha256: digests, CID-shaped result_cid values, and realistic example detached signatures (replaced 1111.. placeholders).
  • Simplified validation command topology in package.json so npm run validate is the single top-level command, with two distinct subcommands kept: validate:schemas and validate:examples; removed the redundant ajv-run.mjs convenience alias.
  • Hardened scripts/validate-examples.mjs to be deterministic and more diagnostic: deterministic file sort, explicit detection of request vs receipt files, per-file AJV error formatting, per-verb and per-version summary counts, and safer schema preloading.
  • Updated contributor docs (README.md, ONBOARDING.md) to describe the canonical workflow and added a concise Fixture discipline section that records rules for realistic valid examples and targeted invalid fixtures.
  • Files changed (representative): package.json, scripts/validate-examples.mjs, README.md, ONBOARDING.md, deletion of scripts/ajv-run.mjs, and all JSON fixtures under examples/v1.1.0/commons/*/json/{valid,invalid} (173 paths touched in the commit).
  • No schema edits were required; existing v1.1.0 schemas validated all new fixtures as intended.

Testing

  • Ran npm run validate:schemas and all schemas compiled successfully (45 schemas compiled).
  • Ran npm run validate:examples and the new v1.1.0 fixture matrix passed; summary printed per-verb/per-version totals (example output included per-verb counts).
  • Ran npm run validate (full pass) and it completed successfully (schema compile + examples verification).
  • Ran a small Python verification to count fixtures and assert no unintentional cross-verb wrong-verb literals; results: v1.1.0 totals = 40 valid (20 requests, 20 receipts) and 100 invalid (80 requests, 20 receipts) and no accidental wrong-verb mismatches.

Notes on scope and remaining gaps

  • No protocol semantics were invented; request schemas are intentionally flat in v1.1.0 so coverage focuses on scalar/boundary behaviors; nested-object validation will require schema changes when semantics demand them.
  • The repo now demonstrates intentional fixtures, realistic receipts, and a clear validation workflow so a skeptical protocol engineer can reproduce and trust the validation surface.

Codex Task

@GsCommand GsCommand merged commit 5a76f76 into main Mar 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant