Skip to content

0053-accuracy-docs-committed.md

Latest commit

 

History

History
52 lines (42 loc) · 2.61 KB

0053-accuracy-docs-committed.md

File metadata and controls

52 lines (42 loc) · 2.61 KB
Raw

ADR-0053: docs/accuracy.md is committed, not generated at docs-build time

Date: 2026-08-09 Status: Accepted

Context

Issue #1266 asks for a generated Markdown table of measured accuracy bounds (docs/accuracy.md), rendered from the differential-testing harness reports (scripts/difftest-special.py, scripts/difftest-dist.py — #1264/#1265). The harness itself deliberately commits no reference literals and runs entirely out-of-band from npm test (ADR-0052): it takes minutes, requires a Python + mpmath environment, and its answer can legitimately drift as mpmath itself is upgraded.

docs/accuracy.md is a different artifact from the raw JSON report: it is the human-facing claim a downstream user reads to reason about error propagation ("accurate to X ULP for |x| <= Y"). Two placements were considered:

  1. Commit it to the repository, regenerated by a documented command (npm run accuracy) and updated by hand whenever a maintainer chooses to re-run the sweep.
  2. Generate it at docs-build time (npm run docs), so it is always as fresh as the latest harness run in whatever environment builds the docs site.

Decision

docs/accuracy.md is committed, exactly as the issue's own recommendation states. It is regenerated by npm run accuracy (which runs both harness sweeps and then scripts/generate-accuracy-docs.js) and reviewed like any other source change — not wired into npm run docs or any build/CI step in this issue.

Consequences

Easier:

  • The bounds table is readable directly on GitHub without anyone running the Python + mpmath harness locally — the same accessibility the committed test/precision-*.js gates already give the correctness contract.
  • A change in measured accuracy (a regression, or a fix landing) shows up as an ordinary reviewable diff in the PR that caused it, the same way CHANGELOG.md entries do.
  • No CI or docs-build dependency on a Python + mpmath environment being available at build time.

Harder:

  • The committed table can go stale: nothing currently fails a build if docs/accuracy.md drifts from what a fresh npm run accuracy would produce. todo.md already tracks a follow-up ("Staleness check on docs/accuracy.md — regenerate in CI and fail if the committed copy is out of date") for closing this gap; it is deliberately out of scope here, the same way scheduled CI for the harness itself was deferred from #1264 to #1267.
  • Regenerating requires a maintainer (or a future CI job) to actually run npm run accuracy and commit the result — it is not automatic on every push, unlike a docs-build-time render.