gate:dataflow: the completeness walker counts gitignored build residue (`__pycache__`, `.venv`, `*.egg-info`) under `core/<dom>/src` as an unregistered module — `gates.mjs all` cannot run twice in one worktree

[DmitriyG228]

Value this issue delivers

node scripts/gates.mjs all gives the same verdict on the same tree every run: a gate:dataflow red means a module is missing from architecture.calm.json, never that a Python gate ran earlier in the same worktree.

Why this matters

gate:dataflow's completeness check is the anti-drift guard on architecture.calm.json (P23), and gate:arch-report re-runs it as its P23 row. In gates.mjs all, dataflow runs fourth and is green on a fresh worktree; the pytest gates run after it (gate:python runs uv run pytest -q in every Python package, core/flows included, and core/flows/tests puts core/flows/src on sys.path), and importing the five top-level modules there writes core/flows/src/__pycache__/. gate:arch-report, later in the same run, re-runs dataflow and reds with:

completeness: 'core/flows/src/__pycache__' exists on disk but is not registered in architecture.calm.json

Every later run in that worktree reds at dataflow itself, and dataflow is in the pre-push hook's fast subset (.githooks/pre-push, FAST_GATES), so the residue then blocks every push from that worktree. The only remedy is find . -name __pycache__ -type d -exec rm -rf {} + by hand. It cost operator time during the v0.13.1-alpha.2 freeze on 2026-09-07. CI never sees it: each job gets a fresh checkout, the static job's gate:dataflow step runs before its gate script tests step (whose gate:python rows run over a temp fixture tree), and the checkout's pytest runs in the separate python job.

Where we are (honest)

Verified at c02e88985 (tip of main, 2026-09-07). minutes-mcp-viewer at ad97be131 carries the identical code (scripts/gates.mjs:893); the line takes main, so the fix lands there at the next merge.

• scripts/gates.mjs:22-23 — SKIP + skippable(name): dot-dirs plus node_modules, dist, .turbo, __pycache__, test-results, playwright-report, coverage. walkDirs() (:56-69), findFile() (:74-84) and scanEnvReads() (:1129) all honour it.
• scripts/gates.mjs:842 — gateDataflow's lsdirs(p) is a raw readdirSync filtered only by isDirectory(); it never consults skippable. It feeds every entry of the required set (:845-873): core/<dom>/services/*, modules/*, contracts/*.vN, core/<dom>/src/* for src-laid-out domains, deploy/contracts/*.vN, clients/*, clients/<cl>/src/*.
• core/flows is src-laid-out: five top-level modules (flows_config.py, flows_queue.py, …) and seven packages under core/flows/src, no services/ or modules/, and core/flows/src is not itself a node — so :863 requires every directory directly under core/flows/src, __pycache__ included once any import compiles those modules.
• .gitignore:4-6 — .venv/, __pycache__/, *.egg-info/. .venv and __pycache__ are already skippable names; *.egg-info is covered nowhere in gates.mjs, so walkDirs() users (gate:readme asks it for a README.md) have the same hole one size smaller.
• Reproduced at c02e88985 (macOS, Node v24.13.0, Python 3.13.11):

  node scripts/gates.mjs dataflow                       # ✓ 98 nodes · 68 edges · 11 carriers
  python3 -m compileall -q core/flows/src               # what any import of those modules leaves behind
  node scripts/gates.mjs dataflow                       # ✗ completeness: 'core/flows/src/__pycache__' …
  mkdir core/flows/src/.venv core/flows/src/zz.egg-info
  node scripts/gates.mjs dataflow                       # ✗ three completeness rows, one per dir
  git status --short                                    # empty — all three are ignored
  

• Checked and NOT a source: gate:db-schema (python3 scripts/schema_digest.py) parses with ast and imports nothing; it leaves no bytecode. The residue comes from the pytest gates.

Deployments to validate (D12b)

None — repo tooling. The change is exercised by the gates workflow's static job (gate:dataflow step and the gate script tests step). No Lite, compose, k8s or hosted run applies, and Lite needs no run of its own.

Docs surface (D6c)

No docs impact, argued: docs/docs/governance/architecture.mdx:138 states what gate:dataflow enforces — the chart covers every service/module/contract/client — and that is unchanged; only gitignored build residue leaves the walker's population, and .gitignore already names it. No page describes the walker's population. Tooling-only, so no changelog fragment (docs/changelog.d/README.md: repo-tooling / test-only changes take docs: none).

The components

• C1 — the completeness walker reads the same population as walkDirs()

C1 · one population for every walker in gates.mjs

Target: ONE seam — scripts/gates.mjs (skippable and gateDataflow's lsdirs).
Value: a name walkDirs() would skip is never an unregistered module: gate:dataflow is green after any Python gate has run in the same worktree, and still red on a real unregistered package.
Prepared solution: lsdirs filters with !skippable(n) before the isDirectory test. skippable additionally recognises *.egg-info (setuptools' editable-install residue, gitignored beside __pycache__), which closes the same hole for every walkDirs() user at once. Regression rows in scripts/gates.test.mjs, same plant-and-run-the-real-gate discipline as the file's other rows: plant core/flows/src/__pycache__, .venv, zz.egg-info → gate:dataflow green; plant core/flows/src/zz_planted_pkg (a real-looking package) → RED naming the path, so the guard is narrowed, not disarmed.
Along the way:

• Ask git check-ignore instead of the name list: git's answer is the truth, but it forks a process per entry and makes the verdict depend on .gitignore state rather than on the one list every other walker in the file reads. Rejected; the list stays the single population.
• Honour .gateignore in lsdirs as walkDirs() does: a vendored subtree under core/<dom>/src would then also drop out of the register — a different question, not in scope.
• The planted-dir fixtures write the checkout, as withPlanted does. The gate script tests give the same verdict on the same tree every run: fixtures edit a private shadow, never the checkout #1480 moves this file's fixtures to a shadow tree; these rows plant into whatever root runGate is handed, so they follow it on rebase. A __pycache__ that already exists when a row runs (bytecode from an earlier gate) is left as found — the row still asserts the property.
  Early validation: node --test scripts/gates.test.mjs red→green on the new rows.

The acceptance table

#	Observation	Negative control (shown RED)	Anchor
A1	In one worktree: python3 -m compileall -q core/flows/src && node scripts/gates.mjs dataflow && node scripts/gates.mjs arch-report → both green	Same commands at base: completeness: 'core/flows/src/__pycache__' exists on disk but is not registered	base c02e88985; head sha in the PR
A2	New rows in scripts/gates.test.mjs: __pycache__, .venv and *.egg-info planted under core/flows/src leave gate:dataflow green	The same rows run against base scripts/gates.mjs: red	base c02e88985; head sha in the PR
A3	Negative-control row: core/flows/src/zz_planted_pkg planted → gate:dataflow RED, the message names the path	this row IS the control — it must red at head	head sha in the PR
A-	No-regression: node --test scripts/*.test.mjs release/*.test.mjs green at head; the gates static job green at head	—	CI gates at head sha

How this issue closes

A parser-class observation, no live human bar. A non-author maintainer runs A1 in a worktree where gate:python has run; the operator who hit it during the v0.13.1-alpha.2 freeze is the preferred signer.

Authorship

Written to be handed to one contributor. No agent co-author trailers.

[DmitriyG228]

Delivered as #1658 (base main, head 98f798988). Rung: PR open. scripts/gates.mjs cherry-picks cleanly onto minutes-mcp-viewer (ad97be131); the scripts/gates.test.mjs hunk needs a one-hunk placement there, ahead of the #209 section that follows the same anchor. Next move: a non-author runs A1 — node scripts/gates.mjs dataflow && node scripts/gates.mjs arch-report in a worktree where gate:python has run, without deleting bytecode first; the operator who hit it during the v0.13.1-alpha.2 freeze is the preferred signer.