Skip to content

Commit f656cb5

Browse files
zahzah
committed
docs: Initial draft of the state persistence spec
1 parent 34062a8 commit f656cb5

File tree

6 files changed

+268
-66
lines changed

6 files changed

+268
-66
lines changed

‎.agents/codebase-insights.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,10 @@
3232
- Devcontainer health check: reference the procedure in `specs/devcontainer-setup.md` and `specs/devcontainer-design.md`; formalize a single entrypoint the CLI will call.
3333
- Config layering: ensure `supported-agents` can be set in system/user/project scopes and overridden by CLI; document keys and env vars.
3434
- Decide on fallback when multiple instruction sources exist and no `source-file` is provided (current behavior: error; alternative: prompt interactively).
35+
36+
## State Persistence Notes (2025-08-27)
37+
- Local state uses a canonical SQLite DB; there is no local daemon. See docs/state-persistence.md for selection rules and SQL schema.
38+
39+
## Config Mapping Notes
40+
- TOML preserves dashes in keys (e.g., `network.api-url`, `tui.default-mode`, `repo.task-runner`).
41+
- ENV/JSON use underscores for those keys (e.g., `AGENTS_WORKFLOW_NETWORK_API_URL`). See specs/configuration.md.

‎.obsidian/workspace.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,7 @@
170170
},
171171
"active": "01bf72509d2b5652",
172172
"lastOpenFiles": [
173+
"docs/state-persistence.md",
173174
"specs/configuration.md",
174175
"specs/cli-spec.md",
175176
"specs/connectivity-layer.md"

‎docs/state-persistence.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# State Persistence
2+
3+
This document specifies how Agents‑Workflow (AW) persists CLI/TUI state locally and how it aligns with a remote server. It defines selection rules, storage locations, and the canonical SQL schema used by the local state database and mirrored (logically) by the server.
4+
5+
## Overview
6+
7+
- AW operates against one of two backends:
8+
- **Local SQLite**: the CLI performs state mutations directly against a per‑user SQLite database. Multiple `aw` processes may concurrently read/write this DB.
9+
- **Remote REST**: the CLI talks to a remote server which implements the same logical schema and APIs.
10+
11+
Both backends share the same logical data model so behavior is consistent.
12+
13+
## Backend Selection
14+
15+
- If a `remote-server` is provided via the configuration system (or via an equivalent CLI flag), AW uses the REST API of that server.
16+
- Otherwise, AW uses the local SQLite database.
17+
18+
All behavior follows standard configuration layering (CLI flags > env > project‑user > project > user > system).
19+
20+
## DB Locations
21+
22+
- Local SQLite DB:
23+
- Linux: `${XDG_STATE_HOME:-~/.local/state}/agents-workflow/state.db`
24+
- macOS: `~/Library/Application Support/Agents-Workflow/state.db`
25+
- Windows: `%LOCALAPPDATA%\Agents-Workflow\state.db`
26+
27+
SQLite is opened in WAL mode. The CLI manages `PRAGMA user_version` for migrations (see Schema Versioning).
28+
29+
## Relationship to Prior Drafts
30+
31+
Earlier drafts described PID‑like JSON session records and a local daemon. These are no longer part of the design. The SQLite database is the sole local source of truth; the CLI talks directly to it.
32+
33+
## SQL Schema (SQLite dialect)
34+
35+
This schema models repositories, workspaces, tasks, sessions, runtimes, agents, events, and filesystem snapshots. It is intentionally normalized for portability to server backends.
36+
37+
```sql
38+
-- Schema versioning
39+
PRAGMA user_version = 1;
40+
41+
-- Repositories known to the system (local path and/or remote URL)
42+
CREATE TABLE IF NOT EXISTS repos (
43+
id INTEGER PRIMARY KEY AUTOINCREMENT,
44+
vcs TEXT NOT NULL, -- git|hg|pijul|...
45+
root_path TEXT, -- local filesystem root (nullable in REST)
46+
remote_url TEXT, -- canonical remote URL (nullable in local)
47+
default_branch TEXT,
48+
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
49+
UNIQUE(root_path),
50+
UNIQUE(remote_url)
51+
);
52+
53+
-- Workspaces are named logical groupings on some servers. Optional locally.
54+
CREATE TABLE IF NOT EXISTS workspaces (
55+
id INTEGER PRIMARY KEY AUTOINCREMENT,
56+
name TEXT NOT NULL,
57+
external_id TEXT, -- server-provided ID (REST)
58+
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
59+
UNIQUE(name)
60+
);
61+
62+
-- Agents catalog (type + version descriptor)
63+
CREATE TABLE IF NOT EXISTS agents (
64+
id INTEGER PRIMARY KEY AUTOINCREMENT,
65+
name TEXT NOT NULL, -- e.g., 'openhands', 'claude-code'
66+
version TEXT NOT NULL, -- 'latest' or semver-like
67+
metadata TEXT, -- JSON string for extra capabilities
68+
UNIQUE(name, version)
69+
);
70+
71+
-- Runtime definitions (devcontainer, local, unsandboxed, etc.)
72+
CREATE TABLE IF NOT EXISTS runtimes (
73+
id INTEGER PRIMARY KEY AUTOINCREMENT,
74+
type TEXT NOT NULL, -- devcontainer|local|unsandboxed
75+
devcontainer_path TEXT, -- when type=devcontainer
76+
metadata TEXT -- JSON string
77+
);
78+
79+
-- Sessions are concrete agent runs bound to a repo (and optionally a workspace)
80+
CREATE TABLE IF NOT EXISTS sessions (
81+
id TEXT PRIMARY KEY, -- stable ULID/UUID string
82+
repo_id INTEGER NOT NULL REFERENCES repos(id) ON DELETE RESTRICT,
83+
workspace_id INTEGER REFERENCES workspaces(id) ON DELETE SET NULL,
84+
agent_id INTEGER NOT NULL REFERENCES agents(id) ON DELETE RESTRICT,
85+
runtime_id INTEGER NOT NULL REFERENCES runtimes(id) ON DELETE RESTRICT,
86+
multiplexer_kind TEXT, -- tmux|zellij|screen
87+
mux_session TEXT,
88+
mux_window INTEGER,
89+
pane_left TEXT,
90+
pane_right TEXT,
91+
pid_agent INTEGER,
92+
status TEXT NOT NULL, -- created|running|failed|succeeded|cancelled
93+
log_path TEXT,
94+
workspace_path TEXT, -- per-task filesystem workspace
95+
started_at TEXT NOT NULL,
96+
ended_at TEXT
97+
);
98+
99+
-- Tasks capture user intent and parameters used to launch a session
100+
CREATE TABLE IF NOT EXISTS tasks (
101+
id INTEGER PRIMARY KEY AUTOINCREMENT,
102+
session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
103+
prompt TEXT NOT NULL,
104+
branch TEXT,
105+
delivery TEXT, -- pr|branch|patch
106+
instances INTEGER DEFAULT 1,
107+
labels TEXT, -- JSON object k=v
108+
browser_automation INTEGER NOT NULL DEFAULT 1, -- 1=true, 0=false
109+
browser_profile TEXT,
110+
chatgpt_username TEXT,
111+
codex_workspace TEXT
112+
);
113+
114+
-- Event log per session for diagnostics and incremental state
115+
CREATE TABLE IF NOT EXISTS events (
116+
id INTEGER PRIMARY KEY AUTOINCREMENT,
117+
session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
118+
ts TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
119+
type TEXT NOT NULL,
120+
data TEXT -- JSON payload
121+
);
122+
123+
CREATE INDEX IF NOT EXISTS idx_events_session_ts ON events(session_id, ts);
124+
125+
-- Filesystem snapshots associated with a session (see docs/fs-snapshots)
126+
CREATE TABLE IF NOT EXISTS fs_snapshots (
127+
id INTEGER PRIMARY KEY AUTOINCREMENT,
128+
session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
129+
ts TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
130+
provider TEXT NOT NULL, -- zfs|btrfs|overlay|copy
131+
ref TEXT, -- dataset/subvolume id, overlay dir, etc.
132+
path TEXT, -- mount or copy path
133+
parent_id INTEGER REFERENCES fs_snapshots(id) ON DELETE SET NULL,
134+
metadata TEXT -- JSON payload
135+
);
136+
137+
-- Key/value subsystem for small, fast lookups (scoped configuration, caches)
138+
CREATE TABLE IF NOT EXISTS kv (
139+
scope TEXT NOT NULL, -- user|project|repo|runtime|...
140+
k TEXT NOT NULL,
141+
v TEXT,
142+
PRIMARY KEY (scope, k)
143+
);
144+
```
145+
146+
### Schema Versioning
147+
148+
- The database uses `PRAGMA user_version` for migrations. Increment the version for any backwards‑incompatible change. A simple `migrations/` folder with `N__description.sql` files can be applied in order.
149+
150+
### Concurrency and Locking
151+
152+
- SQLite operates in WAL mode to minimize writer contention. Multiple `aw` processes can write concurrently; all writes use transactions with retry on `SQLITE_BUSY`.
153+
154+
### Security and Privacy
155+
156+
- Secrets are never stored in plaintext in this DB. Authentication with remote services uses OS‑level keychains or scoped token stores managed by the CLI and/or OS keychain helpers.
157+
158+
## Repo Detection
159+
160+
When `--repo` is not supplied, AW detects a repository by walking up from the current directory until it finds a VCS root. All supported VCS are checked (git, hg, etc.). If none is found, commands requiring a repository fail with a clear error.
161+
162+
## Workspaces
163+
164+
`--workspace` is only meaningful when speaking to a server that supports named workspaces. Local SQLite mode does not define workspaces by default. Commands that specify `--workspace` while the active backend does not support workspaces MUST fail with a clear message.

‎specs/cli-spec.md‎

Lines changed: 23 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ The CLI honors the layered configuration model in [configuration](configuration.
1010

1111
- One tool for both the TUI dashboard and automation-ready commands
1212
- First-class support for:
13-
- Local directory mode (PID-like files for discovery)
13+
- Local state mode (SQLite only)
1414
- Remote REST service mode (on-prem/private cloud), aligned with `docs/rest-service.md`
1515
- Terminal multiplexers: tmux, zellij, screen
1616
- Devcontainers and local runtimes (including unsandboxed, policy-gated)
@@ -20,33 +20,33 @@ The CLI honors the layered configuration model in [configuration](configuration.
2020

2121
- `aw` (no args): Launches the TUI dashboard (default mode described below)
2222
- Common global flags (apply to all subcommands unless noted):
23-
- `--mode <auto|local|rest>`: Discovery/operation mode. Default: `auto` (prefer local when in a repo with `.agents/`; otherwise rest if `network.apiUrl` configured)
24-
- `--repo <PATH|URL>`: Target repository (filesystem path in local mode; git URL otherwise)
25-
- `--project <NAME>`: Project/workspace name (REST mode)
23+
- `--remote-server <NAME|URL>`: If provided, use the REST API at this server (by name lookup in config or raw URL). Otherwise use local SQLite state.
24+
- `--repo <PATH|URL>`: Target repository (filesystem path in local runs; git URL may be used by some servers). If omitted, AW auto-detects a VCS root by walking parent directories and checking all supported VCS.
25+
- `--workspace <NAME>`: Named workspace (only valid on servers that support workspaces). Errors if unsupported by the selected server.
2626
- `--json`: Emit machine-readable JSON
2727
- `--quiet`: Reduce output
2828
- `--log-level <debug|info|warn|error>`
2929
- `--no-color`
3030

31-
Configuration mapping examples:
31+
Configuration mapping examples (dashes preserved in TOML; underscores in ENV/JSON):
3232

33-
- `network.apiUrl` ↔ `--network-api-url`, `AGENTS_WORKFLOW_NETWORK_API_URL`
34-
- `tui.defaultMode` ↔ `--mode`
33+
- `remote-server` ↔ `--remote-server`, `AGENTS_WORKFLOW_REMOTE_SERVER`
34+
- `network.api-url` (per-server) lives under `[[server]]` entries
3535
- `terminal.multiplexer` ↔ `--multiplexer <tmux|zellij|screen>`
3636
- `editor.default` ↔ `--editor`
3737
- `browserAutomation.enabled` ↔ `--browser-automation`, `AGENTS_WORKFLOW_BROWSER_AUTOMATION_ENABLED`
3838
- `browserAutomation.profile` ↔ `--browser-profile`, `AGENTS_WORKFLOW_BROWSER_PROFILE`
39-
- `browserAutomation.chatgptUsername` ↔ `--chatgpt-username`, `AGENTS_WORKFLOW_BROWSER_AUTOMATION_CHATGPT_USERNAME`
39+
- `browserAutomation.chatgpt-username` ↔ `--chatgpt-username`, `AGENTS_WORKFLOW_BROWSER_AUTOMATION_CHATGPT_USERNAME`
4040
- `codex.workspace` ↔ `--codex-workspace`, `AGENTS_WORKFLOW_CODEX_WORKSPACE`
4141

4242
### Subcommands
4343

4444
#### 1) TUI
4545

46-
- `aw` or `aw tui [--mode <local|rest>] [--multiplexer <tmux|zellij|screen>]`
46+
- `aw` or `aw tui [--multiplexer <tmux|zellij|screen>] [--remote-server <NAME|URL>]`
4747
- Starts the dashboard and auto-attaches to the active multiplexer session (creating one if needed).
48-
- Mode `rest`: connects to REST service to retrieve projects/repos/agents/branches and launches local (or remote via SSH) multiplexer windows for new tasks.
49-
- Mode `local`: operates in current repository (default), discovering running tasks via PID-like files in the filesystem.
48+
- With `--remote-server` (or configured `remote-server`), connects to the REST service to retrieve workspaces/repos/agents/branches and launches local (or remote via SSH) multiplexer windows for new tasks.
49+
- Without `--remote-server`, operates against the local SQLite state.
5050

5151
TUI dashboard (simplified quick-launch UI):
5252

@@ -60,13 +60,14 @@ Task launch behavior in TUI:
6060

6161
#### 2) Tasks
6262

63-
- `aw task [create] [--prompt <TEXT> | --prompt-file <FILE>] [--repo <PATH|URL>] [--branch <NAME>] [--agent <TYPE>[@VERSION]] [--instances <N>] [--runtime <devcontainer|local|unsandboxed>] [--devcontainer-path <PATH>] [--labels k=v ...] [--delivery <pr|branch|patch>] [--target-branch <NAME>] [--browser-automation <true|false>] [--browser-profile <NAME>] [--chatgpt-username <NAME>] [--codex-workspace <WORKSPACE>] [--yes]`
63+
- `aw task [create] [--prompt <TEXT> | --prompt-file <FILE>] [--repo <PATH|URL>] [--branch <NAME>] [--agent <TYPE>[@VERSION]] [--instances <N>] [--runtime <devcontainer|local|unsandboxed>] [--devcontainer-path <PATH>] [--labels k=v ...] [--delivery <pr|branch|patch>] [--target-branch <NAME>] [--browser-automation <true|false>] [--browser-profile <NAME>] [--chatgpt-username <NAME>] [--codex-workspace <WORKSPACE>] [--workspace <NAME>] [--fleet <NAME>] [--yes]`
6464

6565
Behavior:
6666

67-
- In local mode, prepares a per-task workspace using snapshot preference order (ZFS > Btrfs > Overlay > copy) and launches the agent.
68-
- In rest mode, calls `POST /api/v1/tasks` with the provided parameters.
69-
- Creates/updates a local PID-like session record when launching locally (see “Local Discovery”).
67+
- With a configured/provided `remote-server`, calls the server’s REST API to create and manage the task.
68+
- Otherwise, prepares a per-task workspace using snapshot preference order (ZFS > Btrfs > Overlay > copy) and launches the agent locally.
69+
- Persists session/task state in the local SQLite database; see `docs/state-persistence.md`.
70+
- Fleet resolution: when `--fleet` is provided (or a default fleet is defined in config), AW expands the fleet into one or more members. For local members, it applies the referenced sandbox profile; for `remote` members, it targets the specified server URL/name. Implementations may run members sequentially or in parallel depending on runtime limits.
7071
- When `--browser-automation true` (default), launches site-specific browser automation (e.g., Codex) using the selected agent browser profile. When `false`, web automation is skipped.
7172
- Codex integration: if `--browser-profile` is not specified, discovers or creates a ChatGPT profile per `docs/browser-automation/codex.md`, optionally filtered by `--chatgpt-username`. Workspace is taken from `--codex-workspace` or config; branch is taken from `--branch`.
7273
- Branch autocompletion uses standard git protocol:
@@ -79,7 +80,7 @@ Draft flow (TUI/Web parity):
7980

8081
#### 3) Sessions
8182

82-
- `aw session list [--status <...>] [--project <...>] [--repo <...>]`
83+
- `aw session list [--status <...>] [--workspace <...>] [--repo <...>] [--remote-server <NAME|URL>]`
8384
- `aw session get <SESSION_ID>`
8485
- `aw session logs <SESSION_ID> [-f] [--tail <N>]`
8586
- `aw session events <SESSION_ID> [-f]`
@@ -90,7 +91,7 @@ Draft flow (TUI/Web parity):
9091

9192
Behavior:
9293

93-
- Local mode reads session records from filesystem; `logs -f` tails the agent log.
94+
- Local mode reads session records from the state database; `logs -f` tails the agent log.
9495
- REST mode proxies to the service and uses SSE for `events -f`.
9596

9697
#### 4) Attach / Open
@@ -104,10 +105,10 @@ Remote sessions:
104105

105106
#### 5) Repositories and Projects
106107

107-
- `aw repo list` (local: from recent usage; rest: from workspace projects)
108+
- `aw repo list` (local: from recent usage; rest: from server workspaces)
108109
- `aw repo add <PATH|URL>` (local only by default)
109110
- `aw repo remove <PATH|URL>` (local; protected confirm)
110-
- `aw project list` (rest mode)
111+
- `aw workspace list` (rest mode)
111112

112113
#### 5a) Repo Init & Instructions
113114

@@ -241,37 +242,9 @@ Mirrors `docs/configuration.md` including provenance, precedence, and Windows be
241242
- `aw doctor` — Environment diagnostics (snapshot providers, multiplexer availability, docker/devcontainer, git).
242243
- `aw completion [bash|zsh|fish|pwsh]` — Shell completions.
243244

244-
### Local Discovery (PID-like Files)
245+
### Local State
245246

246-
Purpose: Enable TUI and CLI to enumerate and manage running local sessions without a server.
247-
248-
Location:
249-
250-
- Per-repo: `.agents/sessions/` within the repository root.
251-
- Optional user index: `~/.config/agents-workflow/sessions/` to aggregate across repos (TUI may use this for “recent sessions”).
252-
253-
Record format (JSON example):
254-
255-
```json
256-
{
257-
"id": "01HVZ6K9T1N8S6M3V3Q3F0X5B7",
258-
"repoPath": "/home/user/src/app",
259-
"workspacePath": "/workspaces/agent-01HVZ6K9",
260-
"agent": {"type": "claude-code", "version": "latest"},
261-
"runtime": {"type": "devcontainer", "devcontainerPath": ".devcontainer/devcontainer.json"},
262-
"multiplexer": {"kind": "tmux", "session": "aw:01HVZ6K9", "window": 3, "paneLeft": "%5", "paneRight": "%6"},
263-
"pids": {"agent": 12345},
264-
"status": "running",
265-
"startedAt": "2025-01-01T12:00:00Z",
266-
"logPath": "/workspaces/agent-01HVZ6K9/.agents/logs/agent.log"
267-
}
268-
```
269-
270-
Creation and lifecycle:
271-
272-
- `aw task` writes the record on successful launch and updates status transitions.
273-
- On normal termination, the record is finalized (endedAt, final status) and moved to an archive file.
274-
- Crash/unclean exit detection: stale PID check on startup; records marked as `failed` with reason.
247+
Local enumeration and management of running sessions is backed by the canonical state database described in `docs/state-persistence.md`. The SQLite database is authoritative; no PID files are used.
275248

276249
### Multiplexer Integration
277250

@@ -334,7 +307,7 @@ aw attach 01HVZ6K9T1N8S6M3V3Q3F0X5B7 --pane left
334307
Run the TUI against a REST service:
335308

336309
```bash
337-
aw tui --mode rest --multiplexer tmux
310+
aw tui --remote-server office-1 --multiplexer tmux
338311
```
339312

340313
Start local REST service and WebUI for a single developer:
@@ -351,4 +324,3 @@ aw webui --local --port 8080 --rest http://127.0.0.1:8081
351324
### Security Notes
352325

353326
- Honors admin-enforced config. Secrets never printed. Unsandboxed runtime gated and requires explicit opt-in.
354-

0 commit comments

Comments
 (0)