feat([wire-proactive-cos-speech-to-real-triggers]): proactive CoS speech on error/task/notification events

.changelog/NEXT.md

@@ -10,6 +10,7 @@
 - **Multiple, ordered code-review agents per task.** The custom-task form (`TaskAddForm`) and scheduled-task form (`ScheduleTab`) now expose an ordered multi-reviewer picker (new `client/src/components/cos/ReviewerPicker.jsx`) instead of a single reviewer dropdown: click reviewers to add them (run order = click order, numbered badges), reorder with ↑↓, remove with ✕, plus a **stop-mode** selector (`run all` / `stop on first fix` / `stop on first clean`) and a **reviewer-applies** toggle. Task metadata gains `reviewers: string[]` (ordered, replacing the single `reviewer`), `reviewStopMode`, and `reviewerApplies`; a read-side `normalizeReviewers()` (server `server/lib/validation.js` + client mirror in `constants.js`) keeps legacy single-`reviewer` tasks/schedules working. The agent prompts thread the list through as slashdo `--review-with a,b,c [--review-stop-on-findings|--review-stop-on-clean] [--reviewer-applies]` on the self-completion path, and the system-spawned review-loop follow-up agent runs each reviewer in the configured order (`buildReviewWithArgs()` builds the flag string; Copilot is pre-requested only when it *leads* the list — otherwise the follow-up requests it at its turn so it reviews the post-fix diff — and the follow-up still runs the CLI reviewers on non-GitHub forges where Copilot is unavailable).
 
 - **Universes list/table index — CRUD all universes at a glance.** New `client/src/pages/Universes.jsx` lists every universe in a table (desktop) / card stack (mobile) with canon-entity count, linked-series count, last-updated, and per-row Share / Sync-to-peer / two-click delete (optimistic remove + rollback, no `window.confirm`) — mirroring the Series Pipeline index. Series counts come from a client-side join against `listPipelineSeries()` (reverse of the join `Pipeline.jsx` already does); canon counts sum the `characters`/`places`/`objects` trunks. The PipelineSeries "Open" link beside the Linked-Universe picker now opens the actually-linked universe instead of a blank builder.
+- **[wire-proactive-cos-speech-to-real-triggers] The assistant can now speak up on its own.** With proactive voice enabled, your Chief of Staff speaks first on three kinds of moments — a critical system error, a new task becoming ready, and a high-priority notification — instead of only talking back when you address it. Each kind is rate-limited on its own timer so a burst can't talk over you, and all of it still respects quiet hours and the proactive-voice switch.
 
 ## Changed
 - **[dedupe-hhmm-time-window-helpers] Shared time-of-day helpers.** Voice quiet-hours and time-windowed dashboard layouts now share one implementation for parsing HH:MM times and deciding whether the current time falls inside a window, so the two can't quietly drift apart.

PLAN.md

@@ -7,7 +7,6 @@ For project goals, see [GOALS.md](./GOALS.md). For completed work, see [.changel
 - [ ] [extract-compare-helpers-once-eight-callers] **Extract `equalByKeys` / `equalListByKeys` once `useAutoRefetch`'s `compare` reaches ~8 widget callers.** With the dashboard rollout (PR #421) there are now 5 widget callers (Backup, Chief of Staff, Decision Log, Goal Progress, Proactive Alerts) on top of EpisodeVideoStage / Brain / Digital Twin from PR #425, using two emerging shapes — (a) scalar-object key equality (BackupWidget status, DecisionLogWidget last24Hours, ProactiveAlertsWidget counts) and (b) array equality by length + per-item key tuple (BackupWidget snapshots, GoalProgressWidget goals, ProactiveAlertsWidget alerts list, DecisionLogWidget impactfulDecisions). Inline forms are noisy but each has an explanatory comment that an `equalByKeys(prev, next, ['status', 'lastRun', …])` would lose. Defer until enough more callers appear that the abstraction pays for itself, then add helpers to `client/src/lib/compareHelpers.js` (register in `index.js` + `README.md`) and migrate the flat callers — leave CosDashboardWidget's nested-optional-chain shape inline. Surfaced by /simplify reuse review during `[useautorefetch-compare-caller-rollout-widgets]` claim 2026-05-21.
 - [ ] [codex5-onboarding-capability-map] **[P2][ONBOARDING]** Capability map of connected systems. One page showing each integration's status: Providers (per-provider configured/available/throttled), Calendar, Brain/memory embeddings, Voice (mic + TTS), Tailscale + HTTPS, Genome/health imports, Telegram/messages, App registry/PM2. Each row links to the relevant settings page. Doubles as a setup checklist and a runtime health overview.
 - [ ] [migrate-two-remaining-local-formatters] **Migrate two remaining local formatters surfaced during the module-discovery PR.** Two additional local format definitions were found that aren't in the original tracker: (1) `client/src/components/calendar/EventDetail.jsx:10` defines `formatDateTime(dateStr, isAllDay)` with an all-day branch the shared `formatDateTime` lacks — extend the shared helper to take `{ allDay: true }` or keep this local and rename to `formatEventDateTime`. (2) `client/src/components/cos/tabs/TaskItem.jsx:78` defines `formatDurationMin(mins)` with a `~` approximation prefix (`~3h 30m`) — extend shared `formatDurationMin` with an `{ approximate: true }` option. Low priority; visual semantics need to be preserved exactly.
-- [ ] [wire-proactive-cos-speech-to-real-triggers] **Wire proactive CoS speech to real triggers.** Plumbing landed (`POST /api/voice/speak` + `voice:speak` socket event); hook to high-severity `errorEvents`, `task:ready`, and `notificationEvents` with per-source rate-limits.
 - [ ] [optimize-voice-ui-index-text-payload-lazy-only-run] **Optimize `voice:ui:index` text payload.** Lazy: only run `extractVisibleText` when server requests via `voice:ui:read-request`. Keep current behavior as fallback.
 - [ ] [voice-agent-explicit-long-term-memory-routing-on] **Voice agent — explicit long-term memory routing.** On retrieval-shaped voice turns, inject top-N relevant memories into the system prompt via `brain_search`.
 - [ ] [flux2-multi-reference-python-runner] **FLUX.2 multi-reference Python runner.** The UI + server contract for multi-reference editing shipped 2026-05-17 (slug `multi-reference-image-editing-for-flux-2-ui`); the Python runner (`scripts/flux2_macos.py`) currently ignores the `--reference-images`/`--reference-strengths` args that `local.js` now passes. Wire diffusers' multi-reference API in the runner and swap `server/lib/mediaModels.js#flux2-klein-9b` `tokenizerRepo` to `FLUX.2-klein-9B-kv` (gated repo — requires the user to accept the license on HF). Validate end-to-end with 2–4 uploaded refs.

server/services/socket.js

@@ -21,6 +21,7 @@ import { loopEvents } from './loops.js';
 import { imageGenEvents } from './imageGenEvents.js';
 import { videoGenEvents } from './videoGen/events.js';
 import { aiStatusEvents } from './aiStatusEvents.js';
+import { wireProactiveTriggers } from './voice/proactiveTriggers.js';
 import * as shellService from './shell.js';
 import {
   validateSocketData,
@@ -559,6 +560,10 @@ export function initSocket(io) {
 
   // Set up AI status event forwarding (broadcast to all clients)
   setupAIStatusEventForwarding();
+
+  // Wire proactive voice (CoS speaks first on high-severity errors, new tasks,
+  // and high-priority notifications — rate-limited per source).
+  setupProactiveSpeechForwarding();
 }
 
 let aiStatusForwardingSetup = false;
@@ -570,6 +575,13 @@ function setupAIStatusEventForwarding() {
   });
 }
 
+let proactiveSpeechForwardingSetup = false;
+function setupProactiveSpeechForwarding() {
+  if (proactiveSpeechForwardingSetup) return;
+  proactiveSpeechForwardingSetup = true;
+  wireProactiveTriggers({ io: ioInstance });
+}
+
 function cleanupStream(socketId) {
   const stream = activeStreams.get(socketId);
   if (stream) {

server/services/voice/proactiveTriggers.js

@@ -0,0 +1,155 @@
+// Wire proactive CoS speech to real subsystem events.
+//
+// `speakProactive` (proactiveSpeech.js) is the delivery primitive — it knows
+// how to suppress for quiet hours / disabled voice and push a line over the
+// `voice:speak` socket event. This module decides WHEN the CoS speaks first,
+// by subscribing to three live event sources and turning select events into
+// spoken lines:
+//
+//   1. errorEvents 'error'        — only `severity: 'critical'` (the rest are
+//                                    routine 4xx/5xx the user shouldn't hear).
+//   2. cosEvents   'task:ready'   — a new task became spawnable.
+//   3. notificationEvents 'added' — only high/critical priority notifications.
+//
+// Each source has its OWN rate-limit bucket so a burst from one source can't
+// starve another, and a storm within a source can't talk over the user. The
+// rate-limit is applied BEFORE `speakProactive` (so we skip the config read +
+// synthesis cost on a throttled event) and the bucket only advances on a line
+// that actually went out — a quiet-hours/disabled suppression doesn't consume
+// the budget.
+//
+// EventEmitter does NOT await async listeners: a rejection from the synthesis
+// path inside `speakProactive` would surface as an unhandled rejection
+// (process-killing on Node ≥15). Listeners here stay synchronous and call the
+// async `dispatch` fire-and-forget; the single `.catch` on that call site
+// (`fire`) is the rejection boundary. `dispatch` itself uses only try/finally
+// (to release a rate-limit reservation), so a synthesis rejection propagates
+// out to that `.catch` rather than being swallowed.
+
+import { errorEvents } from '../../lib/errorHandler.js';
+import { cosEvents } from '../cosEvents.js';
+import { notificationEvents } from '../notifications.js';
+import { speakProactive as defaultSpeak } from './proactiveSpeech.js';
+
+// Per-source minimum interval between spoken lines (ms). Tuned for an opt-in
+// assistant: critical errors are rare so a wide spacing is fine; tasks and
+// notifications can cluster, so a one-minute floor keeps them from chattering.
+export const RATE_LIMIT_MS = {
+  error: 90_000,
+  'task:ready': 60_000,
+  notification: 60_000,
+};
+
+// Spoken lines should be short — synthesis is capped at MAX_PROACTIVE_TEXT_LEN
+// upstream, but for the ear a sentence or two is plenty. Trim long source text.
+const SPEECH_CLIP_LEN = 240;
+
+const clip = (text) => {
+  const s = (text || '').toString().trim().replace(/\s+/g, ' ');
+  return s.length > SPEECH_CLIP_LEN ? `${s.slice(0, SPEECH_CLIP_LEN - 1)}…` : s;
+};
+
+// Pure rate-limit predicate — given a source, its last-spoken timestamp, and
+// "now", may it speak? Unknown sources have no limit. Exported for unit tests.
+export const allowBySource = (source, lastSpokenAt, now, limits = RATE_LIMIT_MS) => {
+  const limit = limits[source];
+  if (!limit) return true;
+  if (lastSpokenAt == null) return true;
+  return now - lastSpokenAt >= limit;
+};
+
+// High-severity notification gate — only `high` / `critical` get spoken.
+export const isHighPriorityNotification = (priority) =>
+  priority === 'high' || priority === 'critical';
+
+// --- Pure formatters: event payload → spoken line (or '' to skip) ---
+
+export const formatErrorLine = (error) => {
+  if (error?.severity !== 'critical') return '';
+  const msg = clip(error.message);
+  return msg ? `Heads up. A critical error just occurred. ${msg}` : '';
+};
+
+export const formatTaskLine = (task) => {
+  const label = clip(task?.title || task?.description);
+  return label ? `A new task is ready: ${label}.` : '';
+};
+
+export const formatNotificationLine = (notification) => {
+  if (!isHighPriorityNotification(notification?.priority)) return '';
+  const title = clip(notification?.title);
+  if (!title) return '';
+  const description = clip(notification?.description);
+  return description ? `${title}. ${description}` : title;
+};
+
+/**
+ * Subscribe proactive speech to live event sources.
+ *
+ * @param {object}   opts
+ * @param {object}   opts.io     Socket.IO server (passed to speakProactive).
+ * @param {Function} [opts.speak] Override the delivery primitive (tests).
+ * @param {object}   [opts.limits] Override per-source rate limits (tests).
+ * @returns {Function} unwire — removes the listeners (boot wires once; tests
+ *                     and hot-reload use this to avoid double-wiring).
+ */
+export const wireProactiveTriggers = ({ io, speak = defaultSpeak, limits = RATE_LIMIT_MS } = {}) => {
+  if (!io) {
+    console.warn('🔕 voice: proactive triggers not wired (no io)');
+    return () => {};
+  }
+
+  // Per-source last-spoken timestamps live in this closure so each wiring gets
+  // isolated state and a rewire starts fresh.
+  const lastSpokenAt = new Map();
+
+  // Speak one line, advancing the source's rate-limit bucket. The slot is
+  // reserved BEFORE awaiting `speak`: synthesis is async, so a same-tick burst
+  // of same-source events (an error storm) would otherwise all read the stale
+  // timestamp, pass the gate, and start concurrent syntheses — defeating the
+  // per-source limit exactly when it matters. The reservation stands on a line
+  // that goes out; on suppression/failure/throw we roll it back (unless a later
+  // event already claimed the slot) so the budget isn't spent on a non-line.
+  // try/finally only — a synthesis rejection still propagates to the caller's
+  // catch.
+  const dispatch = async (source, text, priority) => {
+    if (!text) return;
+    const now = Date.now();
+    if (!allowBySource(source, lastSpokenAt.get(source), now, limits)) return;
+    const previous = lastSpokenAt.get(source) ?? null;
+    lastSpokenAt.set(source, now);
+    let ok = false;
+    try {
+      const result = await speak({ io, text, priority, source });
+      ok = !!result?.ok;
+    } finally {
+      if (!ok && lastSpokenAt.get(source) === now) lastSpokenAt.set(source, previous);
+    }
+  };
+
+  // EventEmitter doesn't await async listeners, so a rejected dispatch would
+  // surface as a process-killing unhandled rejection. The synchronous listeners
+  // call dispatch fire-and-forget with this single explicit catch as the error
+  // boundary — never let a TTS failure escape.
+  const fire = (source, text, priority) => {
+    dispatch(source, text, priority).catch((err) =>
+      console.error(`🔕 voice: proactive ${source} trigger failed: ${err?.message || err}`),
+    );
+  };
+
+  const onError = (error) => fire('error', formatErrorLine(error), 'high');
+  const onTaskReady = (task) => fire('task:ready', formatTaskLine(task), 'normal');
+  const onNotification = (notification) => fire('notification', formatNotificationLine(notification), 'high');
+
+  errorEvents.on('error', onError);
+  cosEvents.on('task:ready', onTaskReady);
+  notificationEvents.on('added', onNotification);
+
+  console.log('🔔 voice: proactive triggers wired (error/task:ready/notification)');
+
+  return () => {
+    errorEvents.off('error', onError);
+    cosEvents.off('task:ready', onTaskReady);
+    notificationEvents.off('added', onNotification);
+  };
+};