refactor(mcp): require daemon for search_files and remove internal/search package

Author: leefowlercu

CLAUDE.md

@@ -56,7 +56,7 @@ The following principles guide development decisions for this project:
 
 Files are processed through three distinct phases:
 
-1. **Metadata Extraction** (`internal/metadata/`) - Fast, deterministic extraction using specialized handlers for 9 file type categories
+1. **Metadata Extraction** (`internal/metadata/`) - Fast, deterministic extraction using specialized handlers for 8 file type categories (Markdown, Docx, Pptx, PDF, Image, VTT, JSON, Code)
 2. **Semantic Analysis** (`internal/semantic/`) - AI-powered content understanding via provider abstraction supporting Claude, OpenAI, and Gemini
 3. **Knowledge Graph Storage** (`internal/graph/`) - FalkorDB stores files, tags, topics, entities, and relationships for semantic search
 
@@ -138,7 +138,7 @@ Detailed technical documentation for each subsystem is available in `docs/subsys
 | [integrations](docs/subsystems/integrations/) | Framework-agnostic integration with dual-hook architecture |
 | [logging](docs/subsystems/logging/) | Structured logging with slog, rotation, and context propagation |
 | [mcp](docs/subsystems/mcp/) | Model Context Protocol with JSON-RPC 2.0 and graph-powered tools |
-| [metadata](docs/subsystems/metadata/) | Fast metadata extraction with handlers for 9 file type categories |
+| [metadata](docs/subsystems/metadata/) | Fast metadata extraction with handlers for 8 file type categories |
 | [semantic](docs/subsystems/semantic/) | Multi-provider AI content understanding with intelligent routing |
 | [tui](docs/subsystems/tui/) | Interactive setup wizard built on Bubble Tea |
 | [version](docs/subsystems/version/) | Build-time version injection with embedded fallback |
@@ -293,7 +293,9 @@ make build                    # Build binary with git version info
 make install                  # Install to ~/.local/bin
 
 # Testing
-make test                     # Run all tests
+make test                     # Run unit tests
+make test-integration         # Run integration tests
+make test-all                 # Run all non-e2e tests (unit + integration)
 make test-race                # Run tests with race detector
 make test-e2e                 # Run E2E tests
 make test-e2e-quick           # Run quick E2E smoke tests
@@ -309,6 +311,7 @@ make coverage-html            # Generate HTML coverage report
 # Cleanup
 make clean                    # Clean build artifacts
 make clean-cache              # Clean cache files
+make uninstall                # Remove installed binary
 make deps                     # Download and tidy dependencies
 
 # Daemon development

README.md

@@ -244,7 +244,7 @@ Agentic Memorizer integrates with multiple AI agent frameworks, providing automa
 - Up to 50 facts, 10-500 characters each
 - Facts injected via UserPromptSubmit (Claude) / BeforeAgent (Gemini) hooks
 
-**Semantic Search** (`internal/search/`):
+**Graph Search** (`internal/graph/`):
 - Graph-powered Cypher queries
 - Full-text search on summaries
 - Entity-based file discovery
@@ -2572,7 +2572,6 @@ agentic-memorizer/
 │   ├── metadata/             # File metadata extraction (9 category handlers)
 │   ├── semantic/             # Multi-provider semantic analysis (Claude, OpenAI, Gemini)
 │   ├── cache/                # Content-addressable analysis caching
-│   ├── search/               # Semantic search engine (graph-powered)
 │   ├── format/               # Output formatting system
 │   │   ├── formatters/       # Individual formatters (text, JSON, XML, YAML, markdown)
 │   │   └── testdata/         # Test data for formatters

docs/subsystems/README.md

@@ -409,7 +409,7 @@ Model Context Protocol implementation with JSON-RPC 2.0 messaging, stdio transpo
 - Five graph-powered tools: search_files, get_file_metadata, list_recent_files, get_related_files, search_entities
 - Three resources: file index in XML, JSON, and Markdown formats with subscription support
 - Three built-in prompts: analyze-file, search-context, explain-summary
-- Dual-source fallback to in-memory index when daemon unavailable
+- Partial fallback for two tools (get_file_metadata, list_recent_files) when daemon unavailable
 - Real-time updates via SSE client for subscribed resource notifications
 
 **Primary Components:**
@@ -569,6 +569,7 @@ Comprehensive integration testing with isolated environments, Docker-based Falko
 
 **Recent Updates:**
 
+- Updated mcp subsystem documentation - search_files now requires daemon (2025-12-29)
 - Created cli subsystem documentation (2025-12-29)
 - Created tui subsystem documentation (2025-12-29)
 - Created logging subsystem documentation (2025-12-29)

docs/subsystems/mcp/README.md

@@ -27,7 +27,7 @@ Key capabilities include:
 - **Five graph tools** - search_files, get_file_metadata, list_recent_files, get_related_files, search_entities
 - **Three resources** - File index in XML, JSON, and Markdown formats with subscription support
 - **Three prompts** - Built-in prompts for file analysis, search context, and summary explanation
-- **Fallback operation** - Graceful degradation to in-memory index when daemon unavailable
+- **Partial fallback** - Two tools (get_file_metadata, list_recent_files) degrade to in-memory index when daemon unavailable
 - **Real-time updates** - SSE client receives index changes and notifies subscribed clients
 
 ## Design Principles
@@ -42,7 +42,7 @@ Tool handlers implement a common Handler interface with Name, Execute, and ToolD
 
 ### Dual-Source Fallback Strategy
 
-Three of the five tools (search_files, get_file_metadata, list_recent_files) implement dual-source logic: they first attempt the daemon HTTP API for current graph data, then fall back to the in-memory index if the daemon is unavailable. This enables degraded operation without complete failure. The remaining tools (get_related_files, search_entities) require the daemon's graph database and cannot fall back.
+Two of the five tools (get_file_metadata, list_recent_files) implement dual-source logic: they first attempt the daemon HTTP API for current graph data, then fall back to the in-memory index if the daemon is unavailable. This enables degraded operation without complete failure. The remaining three tools (search_files, get_related_files, search_entities) require the daemon's graph database for their full functionality and return clear error messages when the daemon is unavailable.
 
 ### Thread-Safe Index Updates
 
@@ -74,7 +74,7 @@ The protocol package defines all JSON-RPC 2.0 and MCP message types. Core types
 
 Five tool handlers implement the Handler interface:
 
-**search_files** - Semantic search with query, optional categories filter, and max_results limit. Tries daemon API first, falls back to index search with fuzzy matching.
+**search_files** - Semantic search with query, optional categories filter, and max_results limit. Requires daemon for graph-powered search across filenames, tags, topics, and entities.
 
 **get_file_metadata** - Complete metadata for a file by path. Tries daemon API first, falls back to case-insensitive index lookup with substring matching.
 
@@ -150,7 +150,7 @@ The cmd/mcp/subcommands/start.go command initializes the MCP server. It loads co
 A feature supported by the MCP server or client, exchanged during initialization. Server capabilities include resources (with subscribe and listChanged), tools, and prompts.
 
 **Fallback**
-The strategy of attempting the daemon API first, then using the in-memory index if unavailable. Enables degraded operation without complete failure.
+The strategy of attempting the daemon API first, then using the in-memory index if unavailable. Two tools (get_file_metadata, list_recent_files) support fallback for degraded operation.
 
 **Handler**
 An implementation of tool logic that receives dependencies and returns results. Handlers are registered by name and invoked on tools/call requests.

internal/mcp/handlers/search_files.go

@@ -4,10 +4,8 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
-	"time"
 
 	"github.com/leefowlercu/agentic-memorizer/internal/mcp/protocol"
-	"github.com/leefowlercu/agentic-memorizer/internal/search"
 )
 
 // SearchFilesHandler handles the search_files tool
@@ -47,18 +45,12 @@ func (h *SearchFilesHandler) Execute(ctx context.Context, args json.RawMessage)
 		params.MaxResults = 10 // default
 	}
 
-	// Try daemon API first if available
-	if h.deps.HasDaemonAPI() {
-		result, err := h.executeDaemon(ctx, params)
-		if err == nil {
-			return result, nil
-		}
-		// Fall through to index-based search on error
-		h.deps.Logger.Debug("daemon search failed, falling back to index", "error", err)
+	// Check if daemon API is available
+	if !h.deps.HasDaemonAPI() {
+		return nil, fmt.Errorf("daemon API not available; search requires daemon connection")
 	}
 
-	// Fallback to index-based search
-	return h.executeIndex(params)
+	return h.executeDaemon(ctx, params)
 }
 
 func (h *SearchFilesHandler) executeDaemon(ctx context.Context, params struct {
@@ -119,52 +111,11 @@ func (h *SearchFilesHandler) executeDaemon(ctx context.Context, params struct {
 	}, nil
 }
 
-func (h *SearchFilesHandler) executeIndex(params struct {
-	Query      string   `json:"query"`
-	Categories []string `json:"categories,omitempty"`
-	MaxResults int      `json:"max_results,omitempty"`
-}) (any, error) {
-	index := h.deps.Index.GetIndex()
-	searcher := search.NewSearcher(index)
-	results := searcher.Search(search.SearchQuery{
-		Query:      params.Query,
-		Categories: params.Categories,
-		MaxResults: params.MaxResults,
-	})
-
-	// Format results
-	formattedResults := make([]map[string]any, len(results))
-	for i, result := range results {
-		formattedResults[i] = map[string]any{
-			"path":       result.File.Path,
-			"name":       result.File.Name,
-			"category":   result.File.Category,
-			"score":      result.Score,
-			"match_type": result.MatchType,
-			"size_human": result.File.SizeHuman,
-			"modified":   result.File.Modified.Format(time.RFC3339),
-		}
-
-		// Add semantic fields if available
-		if result.File.Summary != "" {
-			formattedResults[i]["summary"] = result.File.Summary
-			formattedResults[i]["tags"] = result.File.Tags
-		}
-	}
-
-	return map[string]any{
-		"query":        params.Query,
-		"result_count": len(results),
-		"source":       "index",
-		"results":      formattedResults,
-	}, nil
-}
-
 // ToolDefinition returns the MCP tool definition
 func (h *SearchFilesHandler) ToolDefinition() protocol.Tool {
 	return protocol.Tool{
 		Name:        "search_files",
-		Description: "Search for files in the memory index using semantic search. Returns ranked results based on relevance to the query.",
+		Description: "Search for files in the memory index using semantic search. Returns ranked results based on relevance to the query. Requires FalkorDB to be running.",
 		InputSchema: protocol.InputSchema{
 			Schema: "https://json-schema.org/draft/2020-12/schema",
 			Type:   "object",

internal/mcp/integration_test.go

@@ -7,6 +7,7 @@ import (
 	"context"
 	"encoding/json"
 	"log/slog"
+	"strings"
 	"testing"
 	"time"
 
@@ -399,8 +400,8 @@ func TestIntegration_FullToolsFlow(t *testing.T) {
 		}
 	})
 
-	// Step 4: Call search_files tool
-	t.Run("tools/call_search_files", func(t *testing.T) {
+	// Step 4: Call search_files tool (requires daemon, so expect error)
+	t.Run("tools/call_search_files_requires_daemon", func(t *testing.T) {
 		callReq := protocol.JSONRPCRequest{
 			JSONRPC: "2.0",
 			ID:      3,
@@ -432,22 +433,14 @@ func TestIntegration_FullToolsFlow(t *testing.T) {
 			t.Fatalf("Failed to unmarshal tools call response: %v", err)
 		}
 
-		if callResp.IsError {
-			t.Fatalf("Tool returned error: %s", callResp.Content[0].Text)
-		}
-
-		// Parse and verify result
-		var result map[string]any
-		if err := json.Unmarshal([]byte(callResp.Content[0].Text), &result); err != nil {
-			t.Fatalf("Failed to parse result JSON: %v", err)
-		}
-
-		if result["query"] != "terraform" {
-			t.Errorf("Query = %v, want terraform", result["query"])
+		// search_files requires daemon, so it should return an error
+		if !callResp.IsError {
+			t.Fatal("Expected search_files to return error when daemon is not available")
 		}
 
-		if result["result_count"].(float64) < 1 {
-			t.Error("Expected at least 1 search result for terraform")
+		// Verify error message mentions daemon
+		if !strings.Contains(callResp.Content[0].Text, "daemon") {
+			t.Errorf("Error message should mention daemon, got: %s", callResp.Content[0].Text)
 		}
 	})
 

internal/mcp/server_test.go

@@ -5,6 +5,7 @@ import (
 	"context"
 	"encoding/json"
 	"log/slog"
+	"strings"
 	"testing"
 	"time"
 
@@ -693,7 +694,7 @@ func TestServer_ToolsList(t *testing.T) {
 	}
 }
 
-func TestServer_ToolsCall_SearchFiles(t *testing.T) {
+func TestServer_ToolsCall_SearchFiles_RequiresDaemon(t *testing.T) {
 	index := &types.FileIndex{
 		Generated:  time.Now(),
 		MemoryRoot: "/test",
@@ -712,7 +713,7 @@ func TestServer_ToolsCall_SearchFiles(t *testing.T) {
 	}
 
 	logger := slog.New(slog.NewTextHandler(bytes.NewBuffer(nil), nil))
-	server := NewServer(index, logger, "")
+	server := NewServer(index, logger, "") // No daemon URL
 	server.initialized = true
 
 	writeBuf := bytes.NewBuffer(nil)
@@ -721,7 +722,7 @@ func TestServer_ToolsCall_SearchFiles(t *testing.T) {
 		writeBuf: writeBuf,
 	}
 
-	// Send tools/call request for search_files
+	// Send tools/call request for search_files without daemon
 	request := protocol.JSONRPCRequest{
 		JSONRPC: "2.0",
 		ID:      1,
@@ -746,39 +747,31 @@ func TestServer_ToolsCall_SearchFiles(t *testing.T) {
 	}
 
 	if resp.Error != nil {
-		t.Fatalf("Got error response: %s", resp.Error.Message)
+		t.Fatalf("Got protocol error response: %s", resp.Error.Message)
 	}
 
-	// Parse tool call response
+	// Parse tool call response - should indicate error
 	var callResp protocol.ToolsCallResponse
 	if err := json.Unmarshal(resp.Result, &callResp); err != nil {
 		t.Fatalf("Failed to unmarshal tool call response: %v", err)
 	}
 
-	if callResp.IsError {
-		t.Fatalf("Tool returned error: %s", callResp.Content[0].Text)
+	// search_files requires daemon, so it should return an error
+	if !callResp.IsError {
+		t.Fatal("Expected tool to return error when daemon is not available")
 	}
 
 	if len(callResp.Content) != 1 {
 		t.Errorf("Content count = %d, want 1", len(callResp.Content))
 	}
 
+	// Verify error message mentions daemon
 	if callResp.Content[0].Type != "text" {
 		t.Errorf("Content type = %s, want text", callResp.Content[0].Type)
 	}
 
-	// Verify result contains expected data
-	var result map[string]any
-	if err := json.Unmarshal([]byte(callResp.Content[0].Text), &result); err != nil {
-		t.Fatalf("Failed to unmarshal result JSON: %v", err)
-	}
-
-	if result["query"] != "terraform" {
-		t.Errorf("Query = %v, want terraform", result["query"])
-	}
-
-	if result["result_count"].(float64) != 1 {
-		t.Errorf("Result count = %v, want 1", result["result_count"])
+	if !strings.Contains(callResp.Content[0].Text, "daemon") {
+		t.Errorf("Error message should mention daemon, got: %s", callResp.Content[0].Text)
 	}
 }