Pinard is a multi-agent orchestration system. Components communicate exclusively via NATS JetStream.
| Component | Code | Runs as |
|---|---|---|
| Daemon | cmd/aoc/daemon.go |
self-supervising background process (aoc daemon start, PID file in .state/daemon.pid) |
| Régisseur | pi-extension/pinard/ (no PINARD_PARCELLE) |
Pi extension (Opus), conductor tmux session, window 0 [régisseur] — vignoble general lane + overview |
| Maître | pi-extension/pinard/ (PINARD_PARCELLE=<name>) |
Pi extension (Opus), one tmux window per parcelle in the conductor session |
| Vendangeur 🧺 (worker) | pi-extension/worker/ |
Pi extension (Sonnet), spawned tmux sessions |
| CLI | cmd/aoc/ |
Go binary (~/.local/bin/aoc) |
| Launcher | bin/pinard |
Shell script (starts the régisseur, --maitre <parcelle>, or --worker) |
Terminology — vendangeur vs worker. The harvester role is shown as vendangeur 🧺 in all user-facing surfaces (régisseur dashboard, status/session lines,
aoc dashboard,aoc status, tool labels, slash-command help). The code keeps the nameworker— identifiers, thepi-extension/worker/extension dir,WORKER_*env vars, themodels.workerconfig key, the--workerlauncher flag,worker-policy, tool names (list_workers/kill_worker/interrupt_worker), and NATS durable names are all unchanged. When this doc says "worker" it means the code-level concept; the UI says "vendangeur".
The daemon is always running (started via aoc daemon start, no systemd dependency). It handles: MR watching, issue watching, schedule evaluation, auto-spawn, direct dispatch to worker inboxes, and maître liveness. The régisseur/maîtres are optional — they provide LLM-powered orchestration and user interaction but the system works without them.
One vignoble = one régisseur + N per-parcelle maîtres + M workers:
- Régisseur (
conductorsession, window[régisseur]): the general lane — vignoble overview + unparceled / untriaged / vignoble-level events. Persistent session at.state/regisseur-session.jsonl(not underparcelles/— it is not a workstream). Does not ingest the per-parcelle agent-events firehose (IS_MAITREgates that consumer) — it uses the KV overview (list_parcelles) plus the notifications/issues/schedules consumers. - Maître (per parcelle): a parcelle-scoped conductor. Consumes only
pinard.<v>.parcelles.<parcelle>.agents.*.events.>. Conductor-grade model. Persistent, resumable session atparcelles/<parcelle>/session.jsonl(pi --session, resume-if-exists). Always runs autonomously on its parcelle's events (its durable JetStream consumer ingests and steers the session regardless of tmux attach state); attaching to its window lets you watch and steer it interactively. Runs as a tmux window; not a separate tmux server or a subagent-package child. - Attach = native tmux window switch (
aoc maitre attach --parcelle <name>→ spawns-if-missing thenselect-window; also the/parcelle <name>command andattach_parcelletool). "Steer" a maître = type in its window. - Liveness: the daemon owns it —
recoverMaitres(in orphan recovery) ensures a window exists for every parcelle with live work, when theconductorsession is running. Idle-exit is deferred (active maîtres stay warm).
- Daemon publishes events to JetStream streams (guaranteed delivery)
- Daemon dispatches actionable events (
pipeline_failed,review_comment,main_pipeline_failed,tag_pipeline_failed) directly to worker inboxes — no conductor dependency - Conductor receives events for visibility/dashboard but does NOT dispatch to workers
auto-pinardlabeled issues are spawned by the daemon, not the conductor
- Spawn: daemon (
autoSpawnForIssue) or conductor (spawn_agenttool) callsaoc spawn - Work: worker reads issue/prompt, makes changes, opens MR, calls
track_mr - Monitor: daemon watches MR (pipeline, reviews, approvals)
- Auto-merge: daemon merges when CI passes + approved + no unresolved reviewer threads
- Post-merge: daemon monitors main pipeline via
merge_commit_sha - Reap:
mrs.go'sreapWorkeris the single deterministic teardown point (kills tmux + removes KV + drops the MR-watch entry). It fires at the terminal condition — MR merged and (post-merge main pipeline succeeded, or post-merge monitoring is disabled), or MR closed — uniformly for process and non-process workers (no more "let the process self-terminate"). It does not fire while amain_pipeline_failedfollow-up is outstanding. (Event-driven respawn + location-aware liveness are not-yet-implemented.)
A worker can run on a different machine that has no vignoble directory (e.g. an HPC
node inside the genomics Singularity sandbox), talking to the vignoble's conductor purely
over NATS. Launch with pinard --worker --vignoble-name <name> (instead of --vignoble <path>): the launcher skips vignoble-dir resolution and all $VIGNOBLE-derived setup.
- NATS creds come from
~/.config/pinard/credentials.yaml(viaaoc env-exports) — not the vignoble.--vignoble-nameonly supplies the NATS namespace (NATS_VIGNOBLE). - Standalone requires an explicit concrete
--model <id>(model resolution needs the vignoble) and gets babysitter args from--args, or synthesizes them via the vignoble-freeaoc vigne-args --repo <group/proj> …. - The process definition (build/release) is baked into the project repo at
pinard/<proc>/process.jsand resolves from$(pwd)/pinard/<proc>/process.js. - Resume after failure: babysitter resumes a run from
<runs-dir>/<RUN_ID>/run.json, so a standalone worker must get a stable RUN_ID (defaults to<project>-<process>, not the PID-based session) and a persistent--runs-dir(env vars don't crosssingularity --containall;--runs-diris an arg, point it at a bound persistent path). Re-running with the same RUN_ID continues where it left off. Daemon-side orphan-recovery does not manage remote runs. - Entry point for genomics:
~/genomics-workers/pinard/genomics-build/run.sh(env-driven:VIGNOBLE_NAME,MODEL,PROJECT,PROCESS,REPO, …).
- Main inbox (
…inbox): actionable work via JetStream. Durable, survives restarts. - BTW (
…btw): parallel questions. Real-time core NATS, no persistence. - Interrupt (
…interrupt): cancels current turn. Real-time core NATS.
- All event publish MUST use JetStream publish (not core NATS) for stream-captured subjects
- Daemon ensures streams exist on connect (creates/updates if missing)
- Agent subjects are parcelle-scoped via a literal
parcellessegment:pinard.<v>.parcelles.<parcelle>.agents.<sid>.{events.<type>,inbox,btw,interrupt}. Agent events always carry a parcelle (a worker is spawned with one, defaulting to the project). Build/parse them with the centralized helpers ininternal/pnats/subjects.go(Go) and theagentBase/parseAgentSubjecthelpers in the extensions — never hand-format. Vignoble-level subjects (issues/schedules/notifications) are not parcelle-scoped (régisseur-consumed). - Stream subjects use wildcards:
pinard.*.parcelles.*.agents.*.events.>(all vignobles/parcelles); maître consumers filter topinard.<v>.parcelles.<parcelle>.agents.*.events.> - Consumers are durable,
deliver_policy: "all"for catch-up on restart - BTW and interrupt channels are real-time only (no stream)
- One tmux server per vignoble (socket
pinard-<vignoble>). Theconductorsession holds the[régisseur]window (the general lane) + one window per active parcelle maître. Workers are flat tmux sessions on the same server. - Worker session names are parcelle-leading:
<parcelle>--<project>-<id><rand>(workerSessionNameincmd_spawn.go), sotmux ls/ the prefix+fpicker is self-describing and filterable by parcelle. The vignoble token is dropped (socket already scopes it). Any name/parcelle used as a tmux target is passed throughsession.SanitizeName(forbids./:/whitespace). - Two-level view:
tmux -L pinard-<v> ls→ theconductorsession + workers;tmux -L pinard-<v> list-windows -t conductor(aoc maitre list) → maîtres.
- Conductor:
<vignoble>/.pi/agent/pi-permissions.jsonc— bash: allow all - Workers:
<worktree>/.pi/agent/pi-permissions.jsonc— bash: allow, external_directory: deny - Per-vigne override:
<vignoble>/vignes/<project>/.pi/agent/pi-permissions.jsonc(symlinked into worktree at spawn)
- The daemon self-daemonizes:
aoc daemon startre-execsaoc daemondetached (Setsid), logging tologs/aoc-daemon.log, and records.state/daemon.pid.aoc daemon {stop,restart,status}manage it;aoc statusreads the PID file + liveness (signal 0). - The detached child inherits secrets from
~/.config/pinard/envand an augmentedPATH(nvm node bin so it can findpi) — seedaemonChildEnvincmd_daemon_ctl.go. - Hot-reload is in-process: a
reloadertick polls the mtimes of theaocbinary,vignes.yaml, andschedules.yaml; on any change itsyscall.Execs itself in place (same PID, same log fd, same cwd) — equivalent to the old systemd restart. - No boot persistence: the daemon does not auto-start on login.
aoc initstarts it once (aoc daemon restart) and tears down any legacy systemd units (removeLegacySystemdUnits).
GIT_AUTHOR_*= user (fromgit config --global) — they directed the workGIT_COMMITTER_*= pinard service account (from credentials.yaml) — it pushedGITLAB_TOKEN(not GLAB_TOKEN) for glab API calls as pinard user
Browser read-only view of a vendangeur's tmux session, reached via a signed link. Phases 1 (signed-link transport) and 2 (OIDC SSO + operator discovery) are built; the control-room index and writable "steer" are later phases (not built yet).
Transport (never JetStream — terminal bytes are ephemeral):
browser ⇄ WebSocket ⇄ gateway (k8s) ⇄ core NATS ⇄ responder (tmux host) ⇄ tmux attach -r
- Gateway (
cmd/webterm-gateway,internal/webterm/gateway.go): k8s service serving the embedded xterm.js frontend + WebSocket. Verifies the signed link, mints a short-lived HMAC grant, and bridges the WS to per-viewer NATS subjects. No inbound path to the tmux host is needed. Deployed bycharts/pinard-webterm-gatewayatHost(pinard.example.com) && PathPrefix(/sessions)(webentrypoint, internal-only host). - Responder (
internal/webterm/responder.go): runs on every tmux host — in-process in the daemon (pinard host) and viaaoc webterm-responderon standalone/HPC hosts (started bybin/pinard --worker). Subscribes topinard.<v>.webterm.req, verifies the gateway grant before attaching (anyone with NATS creds could publish), thentmux attach -rvia an embedded PTY (creack/pty) and streams frames back with flow control (coalescing, bounded buffer, rate cap). Tears the PTY down on disconnect/idle/target-exit. - Signed links (
internal/webterm/link.go):…/sessions?v=&target=&exp=&sig=(sig = hmac-sha256(v|target|exp), wherevis the vignoble). Built Go-side byaoc track-mr(auto-post on MRs, gated bywebterm.post_links),aoc webterm-link(CLI), and the régisseur/maître/webterm [name]slash command (on-demand, picker when no name). Once SSO is enabled a posted/shared link is not anonymously usable — the viewer must still authenticate; the signature scopes them to that one session. - Multi-vignoble: one gateway serves many vignobles. The link + grant carry the
vignoble
v; per request the gateway checksvis served, routes topinard.<v>.webterm.*, and resolves the owner forv. The served set is self-maintaining: withgateway.vignoblesempty (default) it serves any vignoble that has published an owner to thepinard-vignoblesKV — a new vignoble's daemon publishes on startup, so no gateway redeploy; unknown vignoble → 403. An explicitgateway.vignobles/WEBTERM_VIGNOBLESlist is an optional allowlist. A single (per-tenant) secret verifies all of a tenant's vignobles — the vignoble is bound into the signature; responders reject a grant whose vignoble ≠ their own. (Per-tenant secrets / isolation = the separatemulti-tenant-webtermchange.) - Subjects (
internal/webterm/subjects.go): per-viewer…webterm.<id>.{out,in,ctl,evt}. - Auth (Phase 2,
internal/webterm/auth.go): whenwebterm.authis configured (issuer + client_id) the gateway runs the Cognito OIDC login flow itself (Authorization Code + PKCE, public client — no secret; not oauth2-proxy/ Traefik), validates the ID token against the pool JWKS (iss/aud/exp/token_use=id), and stores a stateless HMAC-signed session cookie. Absent → Phase-1 signed-link-only. Routes:/sessions/auth/{login,callback}. - Authorization (
authz.go, deny-by-default): operator (JWTpreferred_username== vignoble owner) → any target; viewer (valid signed link) → that target; else denied. Owner is discovered from thepinard-vignoblesKV (operator.go), published by the daemon/responder fromcredentials.yamlnats.user(or theowner:override) — no manual mapping. Never uses the ADgroupsclaim. - Control-room index (
web/index.html,gateway.go):/sessionswith notargetserves an authed operator-only two-pane index — sidebar of the vignobles you own (OwnerStore.OwnedBy), main pane of the selected vignoble's live sessions (régisseur, maître windows, vendangeurs) enumerated over NATS (ListSubject→ respondertmux list-sessions/list-windows; grant-verified, silent on failure). APIs:/sessions/api/{vignobles,sessions}. Vendangeur rows are enriched with parcelle (name prefix) + state (pinard-agentsKV). Each row links to the read-only terminal view. Viewers (scoped-link holders) never see it. - Secrets (
credentials.yamlwebterm:block):base_url,link_secret[_env](link builder ↔ gateway),grant_secret[_env](gateway ↔ every responder),link_ttl,idle_timeout,max_viewers,post_links; andauth:withissuer,client_id,redirect_url(default<base_url>/sessions/auth/callback, must be registered in Cognito),scopes,cookie_secret[_env],session_ttl. The gateway needs link+grant+cookie secrets; a responder needs only the grant secret. - Writable "steer" (
authz.go,responder.go,web/term.html): operator-only, opt-in via?mode=rw(a toggle in the terminal header). The gateway mints aModeRWgrant only for an operator who requested it (viewers are always RO), forwards browser keystrokes toInSubject, and the responder attaches without-rand writes input to the PTY — single-writer, gated on the grant at both ends; audited asmode=rw. - Grouped-session navigation (
responder.go): a window target (session:window, e.g.conductor:3) is viewed via a per-viewer grouped session (tmux new-session -t <base>+select-window), so opening a régisseur/maître window doesn't move the operator's active window; torn down with the viewer. Plain vendangeur sessions attach directly. - Known limitations: one shared NATS connection (a dedicated account for terminal
traffic is a follow-up). Grouped sessions pin
window-size manualso viewer dimensions do not affect the operator's terminal.
cmd/aoc/ — Go CLI binary
main.go — cobra root command
daemon.go — `aoc daemon` (all watchers as goroutines)
cmd_spawn.go — `aoc spawn` (worktree + tmux + policy + env)
cmd_config.go — `aoc config set/get` (dot-path YAML manipulation)
cmd_status.go — `aoc status` (tracked MRs, issues, workers, schedules)
cmd_init.go — `aoc init` (scaffold vignoble; starts daemon, removes legacy systemd units)
cmd_daemon_ctl.go — `aoc daemon start/stop/restart/status` (self-daemonize + PID file)
cmd_notify.go — `aoc notify`
cmd_track_mr.go — `aoc track-mr`, `aoc untrack-mr`
cmd_maitre.go — `aoc maitre spawn/attach/list` (per-parcelle maître windows)
cmd_webterm.go — `aoc webterm-responder` (host responder), `aoc webterm-link` (signed URL)
cmd/webterm-gateway/ — k8s web-terminal gateway binary (HTTP+WS bridge over NATS)
internal/
config/ — credentials.yaml, vignes.yaml, schedules.yaml
state/ — Write-through state (atomic file writes, flock, reload-before-update)
pnats/ — NATS client (WebSocket, JetStream publish, stream ensure, KV); subjects.go = parcelle-scoped subject builders/parser
gitlab/ — GitLab API (direct HTTP, MR/issue/note/pipeline/approval/discussion)
git/ — Git operations (fetch, pull, worktree)
session/ — tmux session/window management (SpawnWorker, EnsureWindow, SanitizeName)
cron/ — Cron expression matching
watcher/ — MR, issue, scheduler, orphan/maître recovery logic
webterm/ — Web terminal (Phase 1): signed links, HMAC grants, NATS subjects, host responder, k8s gateway + embedded xterm.js
bin/
pinard — Launcher (régisseur, --maitre <parcelle>, or --worker mode)
pi-extension/
pinard/ — Régisseur/maître extension (JetStream consumers, tools); parcelle-scoped when PINARD_PARCELLE is set
index.ts — Entry point (event handling, tools, NATS setup)
lib/
logic.ts — Pure functions (formatEventMessage, buildDedupeKey, resolveEventParcelle)
classify.ts — Event classification + subject builders (shared with tests)
worker/ — Worker extension (inbox consumer, aoc_notify, KV state)
index.ts — Entry point
shared/ — Shared tools (read_issue, update_issue, track_mr, proxy provider)
charts/ — Helm charts (pinard-nats, pinard-engram, pinard-website, pinard-webterm-gateway)
tests/ — Four-layer test harness
vignoble-<name>/
vignes.yaml — Vigne registry + models config
schedules.yaml — Scheduled spawns (cron)
.state/ — mr-watcher.yaml, issue-watcher.yaml, scheduler-runs.yaml, daemon.pid
.pi/agent/ — Conductor pi-permissions.jsonc
.engram/ — Engram memory database (per-vignoble)
logs/ — aoc-daemon.log, conductor.log, nats-events.log
vignes/ — Per-vigne config (VIGNE.md, .pi/agent/pi-permissions.jsonc)
PINARD.md — Conductor system prompt (symlink to pinard repo)
Each vignoble has a local-first Engram memory store at .engram/engram.db. Cloud replication to https://engram.example.com is opt-in, configured in credentials.yaml:
engram:
server: https://engram.example.com
cloud_token_env: ENGRAM_CLOUD_TOKEN # env var holding the bearer token
# cloud_token: <literal> # alternative: literal valuePer-vignoble store isolation: bin/pinard derives a deterministic ENGRAM_PORT for each vignoble — cksum(vignoble-name) % 1000 + 7500 — and exports it before launching pi and any child processes. gentle-engram (the mem_* MCP backend) checks this port rather than the global default (7437), so a stray engram serve on 7437 cannot capture another vignoble's memories. If nothing is on the computed port, engram serve is spawned with the inherited ENGRAM_DATA_DIR and ENGRAM_PORT, giving fully isolated per-vignoble storage. An explicit ENGRAM_PORT in the environment overrides the computed value (for operators who need to pin a port). The launcher warns at startup if a serve is detected on 7437 (informational) or if the computed port is already occupied by a serve for a different vignoble (collision warning).
Token source-of-truth: ENGRAM_CLOUD_TOKEN env var (exported by aoc env-exports from credentials.yaml). The .engram/cloud.json file intentionally has an empty token field — engram cloud config only accepts --server, not --token. Engram autosync reads ENGRAM_CLOUD_TOKEN from the environment directly. An empty token in cloud.json is not a mismatch — it is expected.
Startup flush: bin/pinard calls engram sync --cloud --project <vignoble> after enrollment on every start. This drains any pending mutations accumulated while autosync was unavailable (e.g., daemon restart without the token set). Non-fatal — failure logs a warning; autosync retries in the background.
Periodic flush: A background _engram_sync_loop (every 5 min by default, overridable via ENGRAM_SYNC_INTERVAL in seconds) runs while the agent is alive, pushing writes that accumulate during a long session. Self-exits when the parent shell exits.
Exit flush: After pi exits, _run_with_engram_flush fires a final engram sync --cloud to drain any writes made in the last interval before the process dies.
Manual flush: engram sync --cloud --project <name> (requires ENGRAM_CLOUD_TOKEN and ENGRAM_CLOUD_SERVER in env).
Autosync: ENGRAM_CLOUD_AUTOSYNC=1 is set by the launcher; the engram MCP server (running under pi) picks it up and replicates each mem_save in the background.
Failure visibility: enroll and sync failures are now logged as warnings to stderr (instead of being silently swallowed), so replication problems are visible in the session log.
gitlab_host: gitlab.example.com
gitlab_group: exohub
auto_merge: true
models:
conductor:
id: claude-opus-4-6
worker:
id: claude-sonnet-4-6
vignes:
exo-cli:
path: ~/exo-cli
repo: exohub/exo-cli
auto_merge: true
model:
id: claude-opus-4-6 # override worker model for this vignecd cmd/aoc && make install # builds + installs to ~/.local/bin/aoc
go test ./internal/... ./cmd/aoc/ # Go tests
cd tests && npm test # TS unit + integration + contract tests
cd pi-extension && npm run typecheck # type-check the Pi extensionsPi loads extension .ts files untranspiled — there is no build step, so a
typo like proj is not defined would otherwise only surface at runtime inside a
worker. pi-extension/tsconfig.json + npm run typecheck (tsc --noEmit) is the
only gate that catches this. A pre-commit hook (scripts/pre-commit-typecheck.sh,
installed by ./install) blocks commits touching pi-extension/*.ts when it fails;
bypass with git commit --no-verify.
pi-extension/pinard-pi-augment.d.ts augments Pi's ExtensionAPI with
emit, setStatus, and a custom-event on() overload — capabilities Pinard uses
at runtime that the published @earendil-works/pi-coding-agent types omit. Without
it the type-check drowns in false errors. The devDeps in pi-extension/package.json
are pinned to the runtime Pi version so the check reflects what actually runs;
when bumping Pi, bump them together and trim the augment file if upstream types
have since caught up.
Runtime versions: Pi ≥0.80.2 (currently 0.80.6) and Node 24 (>=24 <26).
The Node floor is driven by the pi-sessions package (Pi's own floor is 22.19). The
global Pi install / make dist-vendored runtime and nvm default must be on these —
keep them in lockstep with pi-extension/package.json. .nvmrc pins 24.
make dist produces dist/pinard-linux-x64.run — a single self-extracting
makeself archive (~172M) for Linux/glibc
x64 only. It bundles the static aoc binary, the launcher, the Pi extensions
(with node_modules), and a vendored Node + Pi runtime under runtime/, so the
target machine needs no Node/nvm/npm — only the thin host CLIs tmux git glab fzf.
dist/build.shstages the tree, vendorsnode+ the global Pi install, prunes non-Linux native artifacts (darwin*/win32*prebuilds) and non-runtime bloat (deps/babysitter/library, docs, dev typecheck tooling), then packages via the patched vendored makeself indist/.makeself/(see its README — stock makeself truncates archives over ~100k files; the patch usestar --null -T -). GNU tar format is required for deepnode_modulespaths.dist/pinard-setup.shis the post-extract hook: installs to$PINARD_HOME(default~/.pinard), symlinksbin/{aoc,pinard,pinard-picker}into~/.local/bin, and scaffolds user config (credentials template, pi-permissions, worker-policy, CA certs). It shares this logic withinstall(dev-from-source path).- The launcher (
bin/pinard) resolves its runtime as bundled > nvm > PATH, and delegates all YAML/JSON parsing toaocsubcommands (resolve-model,vigne-args,env-exports) — nopython3at runtime. Keep the bundled Pi version in lockstep with the type-check devDeps. - Build temp goes to
/data/storage/tmp(overridePINARD_DIST_TMPDIR);/tmpis too small for the ~1G staging tarball.
| Layer | What it tests |
|---|---|
Unit (tests/unit/) |
Pure functions: dedup keys, formatting, tool ownership |
Integration (tests/integration/) |
NATS consumer lifecycle, KV, auth |
Contract (tests/contract/) |
Full event flows with real NATS |
Go unit (internal/*/..._test.go) |
State, watcher logic, cron, config |