Skip to content

gate:dataflow: gitignored build residue under core/<dom>/src is not an unregistered module — gates.mjs all runs twice in one worktree (#1657) - #1658

Open
DmitriyG228 wants to merge 1 commit into
mainfrom
fix/gates-dataflow-skippable
Open

DmitriyG228 wants to merge 1 commit into
mainfrom
fix/gates-dataflow-skippable

Conversation

@DmitriyG228

Copy link
Copy Markdown
Member

Delivers issue: #1657

Contribution rights

  • Independent: I created this contribution, or otherwise have the right to submit it
    under Apache-2.0, and it is not owned or controlled by an employer, client, or other entity.
  • Employer/client authorization required: an employer, client, or other entity owns or
    may control this contribution. I am requesting Vexa's private corporate-authorization process.
  • Unsure: I need a private rights review before merge.

Observation bundle

  • C1 · locate the walker — ran: grep -n "skippable\|lsdirs" scripts/gates.mjs at base c02e88985 · saw: skippable at :23, honoured by walkDirs (:57), findFile (:75) and scanEnvReads (:1129); gateDataflow's lsdirs (:842) a raw readdirSync + isDirectory · concluded: one population, two definitions — the completeness walker is the only reader that never filters.
  • C1 · reproduce — ran: python3 -m compileall -q core/flows/src && node scripts/gates.mjs dataflow at base · saw: completeness: 'core/flows/src/__pycache__' exists on disk but is not registered in architecture.calm.json; mkdir core/flows/src/.venv core/flows/src/zz.egg-info → three rows; git status empty throughout · concluded: gitignored residue is read as modules. Checked gate:db-schema (python3 scripts/schema_digest.py): ast parse, no imports, no bytecode — the residue comes from the pytest gates (gate:python and gate:health run uv run pytest in core/flows, whose tests/conftest.py puts src on sys.path).
  • C1 · fix — lsdirs filters with !skippable(n) before the isDirectory test; skippable also recognises *.egg-info (.gitignore:6), so gate:readme and every other walkDirs() user close the same hole at once. Two lines of logic; the comments state the population, not the history.
  • C1 · prove — ran: the same plant at head 98f798988 (residue present) · saw: ✓ gate:dataflow — 99 nodes · 68 edges · 11 carriers · complete + sealed, ✓ gate:arch-report — every modularity principle maps to a green gate · ran: node --test --test-reporter=tap scripts/gates.test.mjs at head · saw: 25/25 (21 existing + 4 new) · ran: the 4 new rows against base scripts/gates.mjs · saw: not ok on the __pycache__ row and on the .venv/*.egg-info row, ok on the vacuity control and on the negative control · ran: git push through .githooks/pre-push · saw: all 14 fast gates green, dataflow among them, after contract-conformance's pytest had run in the same worktree.
  • C1 · the line — git cherry-pick 98f798988 onto minutes-mcp-viewer ad97be131: scripts/gates.mjs applies cleanly; the scripts/gates.test.mjs hunk meets the Release 0.10.0-260419-0052 #209 gate:python section that follows the same anchor there — a one-hunk placement at the next main→line merge. The line's run result follows as a comment.

Acceptance floor

Row Evidence
A1 residue planted (__pycache__, .venv, zz_probe.egg-info under core/flows/src) → gate:dataflow ✓ and gate:arch-report ✓ at head 98f798988; same plant at base c02e88985✗ completeness: 'core/flows/src/__pycache__' … plus one row each for .venv and the egg-info
A2 rows 23–24 of scripts/gates.test.mjs ok at head; the same rows against base gates.mjs: not ok 2, not ok 3 (2 pass / 2 fail of the 4)
A3 row 25 negative control: a real package dir in the same place still reds, named by pathok at head; the message reads completeness: 'core/flows/src/zz_planted_pkg' exists on disk but is not registered
A- node --test scripts/gates.test.mjs 25/25 at head after pnpm install --frozen-lockfile (a worktree without node_modules reds gate:schema and config-contract on ajv — the #1107 hint case, unrelated); the pre-push fast subset green; gates CI on this PR

Docs diff (D6c)

None. docs/docs/governance/architecture.mdx:138 states what gate:dataflow enforces and that is unchanged; no page describes the walker's population; .gitignore already names the residue. Tooling-only, so no changelog fragment (docs/changelog.d/README.md).

Security checks

  • Diff confined to scripts/gates.mjs and scripts/gates.test.mjs; no dependency, lockfile or runtime change — dependency/licence scan not applicable; pnpm gate:licenses ran green in the pre-push subset.
  • Secrets: git diff c02e88985..98f798988 -U0 | grep -Ei '(api[_-]?key|secret|token|password|BEGIN (RSA|OPENSSH)|AKIA)' → no matches.
  • SAST: not applicable to a gate script and its node:test harness; node --check clean on both files.
  • The new fixtures create empty directories under core/flows/src in the checkout and remove only what they created (the withPlanted prune discipline); a pre-existing __pycache__ is left as found.

Validation request

Any non-author maintainer, on a worktree where gate:python or pnpm gates has run: A1 is node scripts/gates.mjs dataflow && node scripts/gates.mjs arch-report without deleting bytecode first. The operator who hit this during the v0.13.1-alpha.2 freeze is the preferred signer.
D12b: no deployment applies — CI tooling only. Provenance of the measurements: fresh git worktree of Vexa-ai/vexa at base c02e88985 on macOS (Darwin 25.5), Node v24.13.0, Python 3.13.11, pnpm 11.7.0, no env deltas beyond ONNXRUNTIME_NODE_INSTALL=skip for the install; a second throwaway worktree at the same base for the red-at-base rows.

Authorship

Submitted by the maintainer. Tooling disclosure: diagnosed, drafted and measured with an agent; no co-author trailers.

…walkDirs()

gate:dataflow requires every directory under a src-laid-out domain's src/,
and its listing never consulted skippable(). Any pytest gate run in
core/flows compiles core/flows/src/*.py into core/flows/src/__pycache__/,
so the next dataflow run — and gate:arch-report, which re-runs it — redded
on gitignored bytecode as an unregistered module: `gates.mjs all` could not
be run twice in one worktree, and the pre-push hook's fast subset carries
dataflow, so the residue blocked pushes.

lsdirs now filters with skippable() before the isDirectory test, and
skippable() recognises *.egg-info alongside the dot-dirs and the SKIP set —
the residue .gitignore already keeps out of the tree — so every walkDirs()
user closes the same hole at once.

Regression rows in scripts/gates.test.mjs plant __pycache__, .venv and an
*.egg-info under core/flows/src (green) and a real-looking package in the
same place (RED, named by path): the population is narrowed to modules, the
anti-drift guard is not disarmed.

Delivers #1657.

Signed-off-by: DmitriyG228 <2280905@gmail.com>
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

🃏 Merge card — #1658

check what it needs
Value missing state: value-signed (the value sign-off)
Diff maintainer self-review — @DmitriyG228 holds the commit bit (no separate non-author review required)

Not mergeable yet — every row above must be accepted before merge (choke point 1). Fill in what's ❌ above, then this clears automatically.

How a PR reaches merge: the merge bar.

@DmitriyG228

Copy link
Copy Markdown
Member Author

The line, measured. git cherry-pick 98f798988 onto minutes-mcp-viewer at ad97be131: scripts/gates.mjs applies cleanly; scripts/gates.test.mjs conflicts because the line carries the #209 gate:python section directly after the same anchor (the runtime-parity RED: the bare apt install row). Placement: the 49-line block goes in ahead of that section's banner (before the line's line 276), which is the order a main→line merge produces once the hunk is placed. With that one placement the line's diff is the same 2 files, +58 −2; pnpm install --frozen-lockfile then node --test --test-reporter=tap scripts/gates.test.mjs on the line: 30/30 — the 4 new rows and the 5 #209 uv rows included. Nothing pushed for the line: it takes main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs: none PR touches a product surface but needs no docs change (D6c waiver, give reason)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant