Skip to content

scripts/gates.test.mjs: fixtures rewrite tracked deploy files in place while sibling test files read them — the required gates check reds at random on unrelated PRs #1478

Description

@DmitriyG228

Value this issue delivers

The gate script tests give the same verdict on the same tree every run: a green gates context on an unchanged tree means the tree is green, and a red one means the PR broke something.

Diagnoses #1106 (the report; it contributes the symptom and closes into this one when this ships).

Why this matters

gates is a required context on main. Its static job's last step runs node --test scripts/*.test.mjs release/*.test.mjs, and that step reds at random on PRs that touch nothing near it: #1106 recorded a docs-only PR blocked by it, and a maintainer measured it at roughly one run in three on a laptop against an untouched worktree at a5ba8e952. The failure text names deploy/lite/Dockerfile.lite, so each red sends whoever reads it hunting a Lite regression that does not exist. Every re-run costs a CI cycle and a person's attention, and a required check that is a coin flip stops being a signal.

Where we are (honest)

Verified at f94c8fa74 (tip of minutes-mcp-viewer, 2026-09-03). The report (#1106, 2026-08-10, main at 9fef84bb) predates this tree; the mechanism below is present unchanged today, so this is a confirmed present-tense defect, not an inherited symptom.

  • scripts/gates.test.mjs:131-138withEdited() rewrites a tracked file in the checkout in place (writeFileSync(join(ROOT, relPath), edited)), runs a gate as a subprocess, then writes the original bytes back. Nine call sites (:153-244) edit deploy/lite/Dockerfile.lite, deploy/compose/docker-compose.yml, deploy/helm/charts/vexa/values.yaml, deploy/helm/charts/vexa/templates/job-minio-init.yaml and image-licenses.json. Three of them replace the literal supervisor postgresql-client in Dockerfile.lite.
  • scripts/sbom.test.mjs:50-53 — reads the same tracked Dockerfile.lite and asserts that exact literal is present ('fixture setup did not alter Dockerfile.lite'). It already does its own fixture correctly: it writes a copy to a temp dir and points scripts/sbom.mjs at it through SBOM_LITE_DOCKERFILE (scripts/sbom.mjs:159).
  • .github/workflows/gates.yml:52node --test scripts/*.test.mjs release/*.test.mjs. node --test runs test files concurrently (default concurrency = available cores − 1), so the sbom read can land inside a gates.test write window. Each file passes alone; only the concurrent run reds.
  • scripts/gates.mjs:15ROOT = process.cwd(); gate:image-licenses (:465,480-482,530,557) and gate:runtime-parity (:594-596,602) read every input relative to it. Neither takes a path override. Nothing in either gate depends on .git.
  • Measured on a 128-core host: the flake itself did not reproduce in 23 baseline runs (3 at default concurrency, 10 pinned to 4 cores, 10 pinned to 2). A 5 ms poller on the tracked file during node --test scripts/gates.test.mjs shows three mutation windows per run, 45-80 ms each, at +1.3 s to +1.7 s — the window exists on every run; whether a sibling's read lands in it is scheduling. The poller is the discriminating instrument for this defect, not the pass/fail rate.
  • Same class, not in scope: withPlanted() (scripts/gates.test.mjs:38-50) and scripts/check-isolation.test.mjs:33 create new files in the checkout for gate:db-budget / gate:isolation. No test file reads those paths concurrently today, and git grep --untracked reading the working tree is the point of those fixtures (the file's header explains why). Left as they are.

Deployments to validate (D12b)

None. This is CI tooling: the change is confined to a test harness and is exercised by the gates workflow's static job on every PR. No Lite, compose, k8s or hosted run applies, and Lite needs no run of its own.

Docs surface (D6c)

No docs impact, argued: the gate-script test lane is described only by the comment at .github/workflows/gates.yml:47-51 and the one-line scripts/README.md; neither describes fixture mechanics, and docs/ does not reference the test files. The gates' documented behaviour (docs/docs/governance/architecture.mdx gate table) is unchanged — only their tests' inputs move.

The components

  • C1 — the gate-test fixtures stop writing the checkout

C1 · the gate-test fixtures stop writing the checkout

Target: ONE seam — scripts/gates.test.mjs (withEdited / runGate).
Value: no test file writes a file another test file reads; the checkout has zero writers during node --test.
Prepared solution: withEdited edits a private shadow of the working tree instead of the checkout, and runGate runs the working tree's scripts/gates.mjs with cwd set to the shadow. Since gates.mjs resolves every path from process.cwd(), that one redirection covers all five edited surfaces and every gate. The shadow is built once per file on first use (mkdtemp + copy of git ls-files -z --cached --others --exclude-standard, regular files only — so an uncommitted deploy edit is part of the fixture exactly as it would be for the gate, and nothing git-ignored is copied) and removed in a node:test after hook. Vacuity controls keep running against the checkout; the shadow is a mirror of it, so the "same tree" premise holds. Call sites are untouched.
Along the way:

  • Env overrides instead (SBOM_LITE_DOCKERFILE style): five surfaces plus a templates directory would each need one, and each new gate input would need another — the cwd redirection needs none. Rejected.
  • Serialize the lane (--test-concurrency=1): hides the race, keeps the class, slows the whole step. Rejected (and the report asked for a fix, not a flaky mark).
  • A throwaway git worktree per test: checks out HEAD, not the working tree, so an uncommitted deploy edit would be invisible to its own fixture. Rejected in favour of the ls-files copy.
  • The listing can name a file a sibling test planted and already removed → skip it (lstat fails).
  • git absent from the runner → the copy fails loudly; every lane this runs in is a git checkout.
  • Cost: ~2,500 files / ~28 MB copied once per file; measured +0.35 s on the file's ~1.5 s.
    Early validation: node --test scripts/gates.test.mjs red→green on the new harness test (below); the poller shows zero mutation windows.

The acceptance table

# Observation Negative control (shown RED) Anchor
A1 A 5 ms poller on deploy/lite/Dockerfile.lite during node --test scripts/gates.test.mjs reports windows=0 Same poller at base: windows=3 (45-80 ms each) base f94c8fa74; head sha in the PR
A2 New test withEdited (#1106): the checkout is untouched while a fixture runs — reads the tracked file from inside the fixture callback and asserts the committed bytes — is green Same test appended to base withEdited: not ok … the tracked Dockerfile.lite changed under a running fixture base f94c8fa74; head sha in the PR
A3 node --test scripts/*.test.mjs release/*.test.mjs passes 10 consecutive runs at head, pinned to 4 cores (the CI runner's shape) The reporter's ~1/3 red rate at a5ba8e952; on the measuring host the base did not red in 23 runs — A1 is the control there head sha in the PR; run logs in the PR bundle
A- No-regression: all 20 tests in scripts/gates.test.mjs green (every RED fixture still reds through the shadow); touched lane + repo gates green at head CI gates at head sha

How this issue closes

A parser-class observation: no live human bar. A non-author maintainer runs A1 and A3 on their own machine; the reporter of #1106, on the laptop where the flake reproduced 1-in-3, is the preferred signer.

Principle check (D7)

The principle indicted is P23 — one writer per data carrier (docs/docs/governance/architecture.mdx), applied one level down: a tracked file in the checkout had two test files on it, one writing and one reading, with no coordination, and the failure was plausible rather than loud. No gate covers the test lane's own inputs; gate:dataflow models the software's carriers, not the CI checkout. That is a constitution finding, recorded here and not extended by this issue.

Authorship

Written to be handed to one contributor. No agent co-author trailers.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind: fix-requestPrepared fix: solution included; closes via its acceptance floorstate: awaiting-evaluationChange staged; needs a non-author instrumented validation (D9)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions