Add compressed file snapshot storage for edit history

- Store gzip-compressed file snapshots in database BLOB column
- Hook reads file content and sends base64-encoded to daemon
- Daemon decodes, compresses, and stores in file_snapshot column
- Query retrieval decompresses and populates FileContent field
- Add VCS detection (git/jj) with commit_sha and vcs_type tracking
- New internal/vcs package for VCS operations
- E2E test for file snapshot storage and retrieval
- Fix workspace filtering test for omitempty JSON behavior

Author: zach-source

HOOKS.md

@@ -1,5 +1,7 @@
 # Claude Code Hooks Setup
 
+> Real-time edit tracking for claude-mon
+
 This guide explains how to install the hooks that send Claude's edits to claude-mon for real-time tracking and persistent history.
 
 ## Prerequisites

README.md

@@ -32,16 +32,24 @@ A TUI application and daemon for watching Claude Code's file edits in real-time,
 - **Automatic injection**: Inject context into Claude prompts via hooks
 - **TUI management**: Full UI for viewing, editing, and managing context
 
+### Task Automation
+- **Ralph Loop**: Monitor and control iterative Claude loops with promise tracking
+- **And-Then Queue**: Sequential task queue that auto-advances when tasks complete
+- **Queue management**: Cancel, skip, or monitor task progress in real-time
+- **State persistence**: YAML-based state files for reliable resumption
+
 ### UI Features
 - **Two-pane layout**: List on left, content preview on right
 - **Toast notifications**: Floating feedback for all actions
 - **Mode switching**: Toggle between History, Prompts, Ralph, Plan, and Context views
 - **Auto-refresh**: Ralph page auto-refreshes every 5 seconds to track loop progress
+- **Status indicators**: Real-time daemon and socket connection status in status bar
 
 ### Daemon & Data Management
 - **Background daemon**: Tracks all edits from any Claude session
 - **Persistent storage**: SQLite database with WAL mode for reliability
 - **Query interface**: Search edits by file, session, or recency
+- **Heartbeat status**: Real-time connection and workspace activity tracking
 - **Automated cleanup**: Configurable data retention and vacuum
 - **Backup system**: Periodic compressed backups
 - **Workspace filtering**: Track or ignore specific paths
@@ -181,10 +189,17 @@ claude-mon daemon start
 ### Ralph Mode
 | Key | Action |
 |-----|--------|
+| `r` | Manual refresh |
+| `C` | Cancel Ralph loop |
+| `Q` | Cancel And-Then queue |
+| `s` | Skip current And-Then task |
+| `R` | Open Ralph chat |
 | **Auto-refresh** | State refreshes every 5 seconds automatically |
-| View loop status | Shows iteration progress and elapsed time |
-| Read prompt | Displays the current loop prompt |
-| See state path | Shows which state file is active |
+
+**Display Features:**
+- Ralph Loop: Shows iteration progress (e.g., "3/10"), promise, and elapsed time
+- And-Then Queue: Shows task progress (e.g., "2/5"), current task, and "done when" criteria
+- State path: Shows which state file is active
 
 ### Context Mode
 | Key | Action |
@@ -418,7 +433,9 @@ claude-mon-notify.sh
         │                      │                      │
         │                      ├── Cleanup Manager    │
         │                      ├── Backup Manager     │
-        │                      └── Query Interface    │
+        │                      ├── Query Interface    │
+        │                      └── Status/Heartbeat   │
+        │                           └── Workspace activity tracking
         │
         └──► Unix socket ──► claude-mon (TUI)
                                     │
@@ -430,14 +447,19 @@ claude-mon-notify.sh
                                     │   └── Version management
                                     │
                                     ├── Ralph View
-                                    │   └── Loop status monitoring
+                                    │   ├── Ralph Loop status monitoring
+                                    │   └── And-Then Queue management
                                     │
                                     ├── Plan View
                                     │   └── Plan generation
                                     │
-                                    └── Context View
-                                        ├── Project context display
-                                        └── Kubernetes/AWS/Git/Env management
+                                    ├── Context View
+                                    │   ├── Project context display
+                                    │   └── Kubernetes/AWS/Git/Env management
+                                    │
+                                    └── Status Bar
+                                        ├── D● Daemon connection indicator
+                                        └── S● Socket connection indicator
 ```
 
 **Data Flow:**
@@ -483,6 +505,9 @@ claude-mon-notify.sh
 
 ## Recent Enhancements
 
+- ✅ **And-Then Queue** sequential task automation with auto-advance
+- ✅ **Daemon heartbeat** real-time connection and workspace activity tracking
+- ✅ **Status bar indicators** showing daemon (D●) and socket (S●) connection state
 - ✅ **Working context management** with per-project context storage
 - ✅ **Context injection hook** for automatic prompt enhancement
 - ✅ **Five-tab layout**: History, Prompts, Ralph, Plan, and Context modes

hooks/claude-mon-hook.sh

@@ -0,0 +1,112 @@
+#!/bin/bash
+# claude-mon PostToolUse hook
+# Sends tool edits to both the TUI and daemon for real-time display and persistence
+
+# Get the current directory and resolve to absolute path
+CWD="$(cd "$(pwd)" && pwd)"
+
+# Resolve symlinks (macOS compatible)
+if command -v realpath &>/dev/null; then
+    CWD="$(realpath "$CWD")"
+elif [[ "$(uname)" == "Darwin" ]]; then
+    # macOS: use perl to resolve symlinks
+    CWD="$(perl -MCwd -e 'print Cwd::realpath($ARGV[0])' "$CWD")"
+else
+    CWD="$(readlink -f "$CWD")"
+fi
+
+# Hash the path (matching Go's sha256.Sum256[:12])
+HASH="$(echo -n "$CWD" | sha256sum | cut -c1-12)"
+
+# Get username
+USER="${USER:-unknown}"
+
+# Socket paths
+TUI_SOCKET="/tmp/claude-mon-${USER}-${HASH}.sock"
+DAEMON_SOCKET="/tmp/claude-mon-daemon.sock"
+
+# Send to TUI if socket exists (raw TOOL_INPUT)
+if [[ -S "$TUI_SOCKET" ]]; then
+    echo "$TOOL_INPUT" | nc -U "$TUI_SOCKET" &
+fi
+
+# Send to daemon if socket exists (formatted payload)
+if [[ -S "$DAEMON_SOCKET" ]] && command -v jq &>/dev/null; then
+    # Parse tool input
+    TOOL_NAME="${TOOL_NAME:-unknown}"
+    FILE_PATH=$(echo "$TOOL_INPUT" | jq -r '.file_path // .path // empty' 2>/dev/null)
+    OLD_STRING=$(echo "$TOOL_INPUT" | jq -r '.old_string // empty' 2>/dev/null | head -c 10000)
+    NEW_STRING=$(echo "$TOOL_INPUT" | jq -r '.new_string // .content // empty' 2>/dev/null | head -c 10000)
+
+    if [[ -n "$FILE_PATH" ]]; then
+        # Get VCS info (jj or git)
+        BRANCH=""
+        COMMIT_SHA=""
+        VCS_TYPE=""
+
+        # Check for jj first (it auto-commits every change)
+        if jj root &>/dev/null 2>&1; then
+            VCS_TYPE="jj"
+            # Get current change ID (short form)
+            COMMIT_SHA=$(jj log -r @ --no-graph -T 'change_id.short()' 2>/dev/null || echo "")
+            # jj doesn't have branches in the same way, use bookmark or description
+            BRANCH=$(jj log -r @ --no-graph -T 'bookmarks' 2>/dev/null | head -1 || echo "")
+        elif git rev-parse --git-dir &>/dev/null; then
+            VCS_TYPE="git"
+            BRANCH=$(git branch --show-current 2>/dev/null || echo "")
+            COMMIT_SHA=$(git rev-parse HEAD 2>/dev/null || echo "")
+        fi
+
+        # Calculate line count
+        LINE_COUNT=0
+        if [[ -n "$NEW_STRING" ]]; then
+            LINE_COUNT=$(echo "$NEW_STRING" | wc -l | tr -d ' ')
+        fi
+
+        # Read and base64-encode file content (max 500KB to avoid huge payloads)
+        FILE_CONTENT_B64=""
+        ABSOLUTE_PATH="$FILE_PATH"
+        if [[ ! "$FILE_PATH" = /* ]]; then
+            ABSOLUTE_PATH="$CWD/$FILE_PATH"
+        fi
+        if [[ -f "$ABSOLUTE_PATH" ]] && [[ $(stat -f%z "$ABSOLUTE_PATH" 2>/dev/null || stat -c%s "$ABSOLUTE_PATH" 2>/dev/null) -lt 512000 ]]; then
+            FILE_CONTENT_B64=$(base64 < "$ABSOLUTE_PATH" 2>/dev/null | tr -d '\n' || echo "")
+        fi
+
+        # Create daemon payload
+        PAYLOAD=$(jq -n \
+            --arg type "edit" \
+            --arg workspace "$CWD" \
+            --arg workspace_name "$(basename "$CWD")" \
+            --arg branch "$BRANCH" \
+            --arg commit_sha "$COMMIT_SHA" \
+            --arg vcs_type "$VCS_TYPE" \
+            --arg tool_name "$TOOL_NAME" \
+            --arg file_path "$FILE_PATH" \
+            --arg old_string "$OLD_STRING" \
+            --arg new_string "$NEW_STRING" \
+            --arg file_content_b64 "$FILE_CONTENT_B64" \
+            --argjson line_num 0 \
+            --argjson line_count "$LINE_COUNT" \
+            '{
+                type: $type,
+                workspace: $workspace,
+                workspace_name: $workspace_name,
+                branch: $branch,
+                commit_sha: $commit_sha,
+                vcs_type: $vcs_type,
+                tool_name: $tool_name,
+                file_path: $file_path,
+                old_string: $old_string,
+                new_string: $new_string,
+                file_content_b64: $file_content_b64,
+                line_num: $line_num,
+                line_count: $line_count
+            }')
+
+        echo "$PAYLOAD" | nc -U "$DAEMON_SOCKET" &
+    fi
+fi
+
+# Wait for background jobs to complete before exiting
+wait

hooks/e2e_test.sh

@@ -0,0 +1,119 @@
+#!/bin/bash
+# E2E Hook Reliability Test
+# Tests the claude-mon hook under various conditions
+# Usage: ./hooks/e2e_test.sh [count]
+
+set -e
+
+COUNT=${1:-100}
+TIMEOUT_MS=${2:-100}  # milliseconds
+TIMEOUT_S=$(echo "scale=3; $TIMEOUT_MS / 1000" | bc)
+
+cd "$(dirname "$0")/.."
+
+echo "=============================================="
+echo "Claude-Mon Hook E2E Test"
+echo "=============================================="
+echo "Records per test: $COUNT"
+echo "Hook timeout: ${TIMEOUT_MS}ms"
+echo ""
+
+# Verify daemon is running
+if ! claude-mon query recent 1 &>/dev/null; then
+    echo "ERROR: Daemon not running. Start with: claude-mon daemon start"
+    exit 1
+fi
+
+# Create test hooks
+TEMP_DIR=$(mktemp -d)
+trap "rm -rf $TEMP_DIR" EXIT
+
+# Old hook (no wait)
+cat > "$TEMP_DIR/old-hook.sh" << 'HOOK'
+#!/bin/bash
+CWD="$(pwd)"
+DAEMON_SOCKET="/tmp/claude-mon-daemon.sock"
+if [[ -S "$DAEMON_SOCKET" ]] && command -v jq &>/dev/null; then
+    FILE_PATH=$(echo "$TOOL_INPUT" | jq -r '.file_path // empty' 2>/dev/null)
+    NEW_STRING=$(echo "$TOOL_INPUT" | jq -r '.new_string // empty' 2>/dev/null)
+    sleep 0.01  # Simulate processing time
+    PAYLOAD=$(jq -n --arg type "edit" --arg workspace "$CWD" --arg workspace_name "test-old" \
+        --arg tool_name "$TOOL_NAME" --arg file_path "$FILE_PATH" --arg new_string "$NEW_STRING" \
+        --arg old_string "" --arg branch "" --arg commit_sha "" --argjson line_num 0 --argjson line_count 1 \
+        '{type:$type,workspace:$workspace,workspace_name:$workspace_name,tool_name:$tool_name,file_path:$file_path,old_string:$old_string,new_string:$new_string,branch:$branch,commit_sha:$commit_sha,line_num:$line_num,line_count:$line_count}')
+    echo "$PAYLOAD" | nc -U "$DAEMON_SOCKET" &
+fi
+HOOK
+chmod +x "$TEMP_DIR/old-hook.sh"
+
+# New hook (with wait)
+cat > "$TEMP_DIR/new-hook.sh" << 'HOOK'
+#!/bin/bash
+CWD="$(pwd)"
+DAEMON_SOCKET="/tmp/claude-mon-daemon.sock"
+if [[ -S "$DAEMON_SOCKET" ]] && command -v jq &>/dev/null; then
+    FILE_PATH=$(echo "$TOOL_INPUT" | jq -r '.file_path // empty' 2>/dev/null)
+    NEW_STRING=$(echo "$TOOL_INPUT" | jq -r '.new_string // empty' 2>/dev/null)
+    sleep 0.01  # Simulate processing time
+    PAYLOAD=$(jq -n --arg type "edit" --arg workspace "$CWD" --arg workspace_name "test-new" \
+        --arg tool_name "$TOOL_NAME" --arg file_path "$FILE_PATH" --arg new_string "$NEW_STRING" \
+        --arg old_string "" --arg branch "" --arg commit_sha "" --argjson line_num 0 --argjson line_count 1 \
+        '{type:$type,workspace:$workspace,workspace_name:$workspace_name,tool_name:$tool_name,file_path:$file_path,old_string:$old_string,new_string:$new_string,branch:$branch,commit_sha:$commit_sha,line_num:$line_num,line_count:$line_count}')
+    echo "$PAYLOAD" | nc -U "$DAEMON_SOCKET" &
+fi
+wait  # KEY FIX: Wait for background nc to complete
+HOOK
+chmod +x "$TEMP_DIR/new-hook.sh"
+
+# Generate unique test ID
+TEST_ID=$(date +%s)
+
+echo ">>> Test 1: OLD hook (no wait) with ${TIMEOUT_MS}ms timeout..."
+OLD_SUCCESS=0
+for i in $(seq 1 $COUNT); do
+    export TOOL_NAME="Edit"
+    export TOOL_INPUT='{"file_path":"/tmp/e2e-'$TEST_ID'-old-'$i'.txt","new_string":"old hook '$i'"}'
+    timeout $TIMEOUT_S "$TEMP_DIR/old-hook.sh" 2>/dev/null && ((OLD_SUCCESS++)) || true
+done
+sleep 2
+OLD_COUNT=$(claude-mon query recent $((COUNT * 3)) 2>/dev/null | grep -c "e2e-${TEST_ID}-old-" || true)
+OLD_COUNT=${OLD_COUNT:-0}
+echo "  Hooks completed: $OLD_SUCCESS / $COUNT"
+echo "  Records in DB:   $OLD_COUNT / $COUNT"
+OLD_LOSS=$((COUNT - OLD_COUNT))
+echo ""
+
+echo ">>> Test 2: NEW hook (with wait) with ${TIMEOUT_MS}ms timeout..."
+NEW_SUCCESS=0
+for i in $(seq 1 $COUNT); do
+    export TOOL_NAME="Edit"
+    export TOOL_INPUT='{"file_path":"/tmp/e2e-'$TEST_ID'-new-'$i'.txt","new_string":"new hook '$i'"}'
+    timeout $TIMEOUT_S "$TEMP_DIR/new-hook.sh" 2>/dev/null && ((NEW_SUCCESS++)) || true
+done
+sleep 2
+NEW_COUNT=$(claude-mon query recent $((COUNT * 3)) 2>/dev/null | grep -c "e2e-${TEST_ID}-new-" || true)
+NEW_COUNT=${NEW_COUNT:-0}
+echo "  Hooks completed: $NEW_SUCCESS / $COUNT"
+echo "  Records in DB:   $NEW_COUNT / $COUNT"
+NEW_LOSS=$((COUNT - NEW_COUNT))
+echo ""
+
+echo "=============================================="
+echo "RESULTS"
+echo "=============================================="
+printf "| %-20s | %-12s | %-12s | %-12s |\n" "Hook Version" "Completed" "Recorded" "Data Loss"
+printf "| %-20s | %-12s | %-12s | %-12s |\n" "--------------------" "------------" "------------" "------------"
+printf "| %-20s | %-12s | %-12s | %-12s |\n" "Old (no wait)" "$OLD_SUCCESS/$COUNT" "$OLD_COUNT/$COUNT" "$OLD_LOSS"
+printf "| %-20s | %-12s | %-12s | %-12s |\n" "New (with wait)" "$NEW_SUCCESS/$COUNT" "$NEW_COUNT/$COUNT" "$NEW_LOSS"
+echo ""
+
+if [[ $NEW_LOSS -eq 0 ]] && [[ $OLD_LOSS -gt 0 || $NEW_COUNT -ge $OLD_COUNT ]]; then
+    echo "✅ PASS: New hook (with wait) is more reliable"
+    exit 0
+elif [[ $NEW_LOSS -eq 0 ]] && [[ $OLD_LOSS -eq 0 ]]; then
+    echo "✅ PASS: Both hooks succeeded (try lower timeout to stress test)"
+    exit 0
+else
+    echo "❌ FAIL: New hook lost data"
+    exit 1
+fi