Skip to content

The gate script tests give the same verdict on the same tree every run: fixtures edit a private shadow, never the checkout - #1480

Open
DmitriyG228 wants to merge 1 commit into
minutes-mcp-viewerfrom
gates-test-shadow-tree
Open

DmitriyG228 wants to merge 1 commit into
minutes-mcp-viewerfrom
gates-test-shadow-tree

Conversation

@DmitriyG228

Copy link
Copy Markdown
Member

Delivers issue: #1478 (diagnoses the report #1106)

Contribution rights

  • Independent: I created this contribution, or otherwise have the right to submit it
    under Apache-2.0, and it is not owned or controlled by an employer, client, or other entity.
  • Employer/client authorization required: an employer, client, or other entity owns or
    may control this contribution. I am requesting Vexa's private corporate-authorization process.
  • Unsure: I need a private rights review before merge.

Observation bundle

  • C1 · locate the writer — ran: grep -n writeFileSync scripts/*.test.mjs release/*.test.mjs at base f94c8fa74 · saw: scripts/gates.test.mjs:136 writes join(ROOT, relPath) for five tracked deploy surfaces; scripts/sbom.test.mjs:51 reads deploy/lite/Dockerfile.lite and asserts the literal three of those fixtures replace; every other writer targets a mkdtemp dir · concluded: two test files on one tracked file, one writing, one reading, run concurrently by node --test.
  • C1 · measure the window — ran: a 5 ms poller on the tracked file during node --test scripts/gates.test.mjs at base · saw: MUTATED +1330ms / restored +1392ms, +1509/+1553, +1659/+1741windows=3 · concluded: 45–80 ms per run in which any sibling read sees a fixture as the tree. The pass/fail rate on this 128-core host was 0/23 at base (3 default, 10 pinned to 4 cores, 10 to 2), so the poller, not the rate, is the instrument here.
  • C1 · confirm the redirection is sufficient — ran: read gate:image-licenses and gate:runtime-parity in scripts/gates.mjs · saw: every input resolved from ROOT = process.cwd() (:465,480-482,530,557,594-596,602), no .git dependency · concluded: pointing the subprocess cwd at a shadow copy covers all five surfaces without a per-file override.
  • C1 · fix — withEdited edits a per-file shadow (git ls-files -z --cached --others --exclude-standard copy, built once, removed in after()); runGate runs the working tree's scripts/gates.mjs with cwd = shadow. Call sites untouched. Rejected on the way: env overrides (five surfaces + a templates dir), --test-concurrency=1 (hides the race, keeps the class), a throwaway git worktree (HEAD, not the working tree — an uncommitted deploy edit would be invisible to its own fixture).
  • C1 · prove — ran: poller at head · saw: windows=0, leftover shadows: 0 · ran: node --test scripts/gates.test.mjs · saw: 20/20 (every RED fixture still reds through the shadow; file time 1.46 s → 1.82 s) · ran: the new harness test appended to the base withEdited in a throwaway worktree · saw: not ok … the tracked Dockerfile.lite changed under a running fixture (19 pass / 1 fail) · ran: taskset -c 0-3 node --test scripts/*.test.mjs release/*.test.mjs ×10 at head · saw: 10/10 pass=134 fail=0.

Acceptance floor

Row Evidence
A1 poller: base f94c8fa74windows=3 (45–80 ms each); head → windows=0
A2 withEdited (#1106): the checkout is untouched while a fixture runs — the edit lives in the shadow only: RED against base withEdited (not ok 20 … changed under a running fixture), GREEN at head
A3 node --test scripts/*.test.mjs release/*.test.mjs, 4-core pin, 10 consecutive runs at head: 10/10 green, 134 tests each
A- scripts/gates.test.mjs 20/20 at head; full script-test lane green ×10; gates CI on this PR

Docs diff (D6c)

None. The lane is described only by the comment at .github/workflows/gates.yml:47-51 and the one line in scripts/README.md; neither states fixture mechanics, and docs/ does not reference the test files. The gates' documented behaviour is unchanged — only their tests' inputs move. The mechanism is explained in the file header where the next reader of withEdited will look.

Security checks

  • Diff confined to scripts/gates.test.mjs; no dependency, lockfile or runtime change — dependency/licence scan not applicable, pnpm gate:licenses runs in CI regardless.
  • Secrets: git diff -U0 | grep -Ei '(api[_-]?key|secret|token|password|BEGIN (RSA|OPENSSH)|AKIA…)' → no matches. gitleaks is not installed on the measuring host.
  • SAST: not applicable to a node:test harness; node --check clean.
  • The shadow is a mkdtemp dir of tracked + untracked-unignored files only; nothing git-ignored (secrets files are ignored) is copied, and it is removed when the file's tests finish.

Validation request

Any non-author maintainer; the reporter of #1106, on the laptop where the flake reproduced ~1 in 3, is the preferred signer. Watch: A1 (the poller from the bundle, or any 5 ms read loop on deploy/lite/Dockerfile.lite during node --test scripts/gates.test.mjs) and A3 (10 consecutive full-lane runs).
D12b: no deployment applies — CI tooling only. Provenance of the measurements: fresh git worktree of Vexa-ai/vexa at base f94c8fa74 on a 128-core Linux host, Node v22.14.0, no env deltas; a second throwaway worktree at the same base for the A2 negative control.

Authorship

Submitted by the maintainer. Tooling disclosure: diagnosed, measured and drafted with an agent; no co-author trailers.

…ckout

withEdited() rewrote tracked deploy files in place and restored them after the
gate ran. node --test runs test files concurrently, and sbom.test.mjs reads
deploy/lite/Dockerfile.lite and asserts the very marker the runtime-parity
fixture replaces, so its read could land inside a ~50 ms write window: the
required gates check redded at random on unrelated PRs (#1106).

The fixture now edits a per-file shadow of the working tree (git ls-files copy,
built once, removed in after()) and runs the working tree's gates.mjs with cwd
set to the shadow — gates.mjs resolves every input from process.cwd(), so the
one redirection covers all five edited surfaces. The checkout has no writer.

Adds a harness test that reads the tracked file from inside the fixture and
asserts the committed bytes (red on the old withEdited).
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

🃏 Merge card — #1480

check what it needs
Value missing state: value-signed (the value sign-off)
Diff maintainer self-review — @DmitriyG228 holds the commit bit (no separate non-author review required)

Not mergeable yet — every row above must be accepted before merge (choke point 1). Fill in what's ❌ above, then this clears automatically.

How a PR reaches merge: the merge bar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant