feat(shell): redis console store with cross-language streaming integ

[zechengz]

Follow-up to #611. The ConsoleStore contract was shaped like a stream on purpose; this lands the second implementation to prove it, so a job's console can be read from another process (or another language) while the job runs, and makes the console backend a config-level switch.

RedisConsoleStore (both languages)

mirage/shell/console/redis/ and @struktoai/mirage-node shell/console/redis/, one Redis stream per job. Each package holds store + constants + append.lua, following the cas.lua precedent: the Lua ships beside its module (importlib.resources in Python; readFileSync(new URL(...)) plus a tsup dist copy in TypeScript), and the two copies are pinned byte-identical by a test.

• Schema. Chunk seq maps to stream id (seq+1)-0; channel, payload and timestamp ride as entry fields (c/d/t). append.lua (INCR + explicit-id XADD) hands out dense seqs atomically, so a kill marker racing a runner's last emit cannot collide on an id. Both languages write byte-identical schema, and each side's unit tests assert the same pinned raw-wire shape, so a reader in the other language attaches to a stream this one wrote by construction.
• Blocking reads. Python wait parks server-side (XREAD BLOCK) in short rounds so a local close() is noticed within one round. TypeScript polls the seq counter instead: node-redis serializes commands on one connection, so a server-side block would wedge the job's own appends behind it. Documented in place, mirroring the RAM store's own py/ts asymmetry note.
• No retention trim in this version, so read_from never reports a truncated cursor; the contract's truncation channel stays for stores that do trim.

The backend is selected at runtime, not compiled in

• Programmatic: JobTable takes a console_factory / consoleFactory (job id -> JobConsole), threaded through Workspace as a constructor knob beside the other store overrides.
• Config / CLI: a console: block in the workspace YAML ({type: ram} default, {type: redis, url, key_prefix}), a discriminated block exactly like cache: / index: / store:, validated by both loaders and covered by the shared accept/reject fixtures (integ/fixtures/config/*.json). The config-built factory mints a fresh nonce beside each job id, because ids restart at 1 when the table empties and a reused stream would replay the previous job's chunks.
• The default is untouched: no factory means an in-memory console per job. The redis module is imported only by its own package in Python (guarded extra, same as the redis store blocks), and the npm redis package stays a lazily-loaded optional peer.

Retention, addressing and the terminal chunk

Three properties the store enforces itself, all inside append.lua so they are atomic with the write:

• A console has an address. The factory mints a per-job nonce (job ids restart at 1 when the table empties, so a prefix keyed on the id alone would replay the previous job's chunks), and that prefix is public: RedisConsoleStore.key_prefix / keyPrefix, reachable as job.console.store. That is what an embedder hands to the process that should attach.
• Retention is bounded. ttl_seconds (default one day, null to opt out) issues an EXPIRE on every append, so a console expires that long after its job's last write whether or not the job ended cleanly.
• The ending is terminal in the store, not just in the process. The script sets an ended key with the CONTROL chunk and refuses later appends, so an emit that raced a kill past JobConsole's local guard is dropped server-side rather than landing past the ending where no follower would read it. That matches the RAM store and emit's documented after-the-ending semantics.

Ownership

The job table tracks what the factory builds and the workspace closes it at teardown (close_consoles / closeConsoles, after kill_all): a config-provisioned store keeps a client open per job, invisible to the embedder, and in Node an open client holds the process alive. A console still outlives its table entry, so a reap never closes one, and default in-memory consoles are never tracked.

The _settle shield

The kill path in _settle emits while the task is already unwinding from a cancel. RAM never suspends there; Redis does, and a second cancellation mid-write would lose the Killed marker readers are parked on. The marker + ending write is now wrapped in asyncio.shield. The regression test was verified both ways: it times out on the unshielded code and passes with the shield.

Battery coverage instead of bespoke scripts

A new console-redis target (ram mounts, consoles on Redis via the target's console block) rides the shared battery beside ram: integ/console/jobs.json runs the same four job-control cases (& wait output adoption, wait $! exit codes, kill by pid, background writes landing) on both hosts x both backends from one JSON file. The runners gained the console case dir, the target key, and a refusal for a console block on any non-ram resource, so a silently RAM-consoled "redis" target cannot read as covered.

job_table is a package now

shell/job_table.py and shell/job_table.ts split into types (Job, JobStatus, JobRunner, ConsoleFactory), constants (KILLED_EXIT_CODE) and table (JobTable, cancel_job/settle), mirroring the console package's layout in both languages. Import paths are unchanged for every caller; the tests moved to match (tests/shell/job_table/test_table.py, shell/job_table/table.test.ts). Layout parity holds at its committed baseline.

Tests

• tests/shell/console/redis/ and node shell/console/redis/store.test.ts (skipped without REDIS_URL): dense seqs, cursor+limit reads, wake-on-append, close releases a parked waiter, a follow across two store instances sharing only the key prefix, the pinned raw wire schema, drop-after-ending, TTL applied and absent, and the byte-identity of the two append.lua copies.
• test_job_table.py / job_table.test.ts: close_consoles releases factory stores and leaves default consoles alone; the second-cancel shield test.
• Config: python test_loader.py and server config.test.ts cover the redis form (factory with fresh keys) and the ram form (no factory), plus the shared accept/reject fixtures (which now pin a non-string url and a zero ttl_seconds as rejected by both loaders).
• Gates: full python suite, TS core/node/server suites, the ram + console-redis batteries in both hosts, layout parity at baseline, pre-commit clean.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[zechengz]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c95061fb35

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".