Architecture Documentation

Detailed architecture documentation for the Context Engine MCP Server.

Historical note: some older sections below are archival. The current active runtime is local_native, and legacy provider SDKs are not part of the live provider path.

Overview

This project implements a local-first, agent-agnostic context engine using the Model Context Protocol (MCP) as the interface layer and a local-native retrieval runtime as the core engine.

5-Layer Architecture

Layer 1: Core Context Engine (Local-Native Runtime)

Location: src/retrieval/, src/mcp/serviceClient.ts, src/internal/retrieval/

Purpose: The brain of the system. Handles all low-level context operations.

Responsibilities:

File ingestion and scanning
Code chunking with language awareness
Embedding generation
Semantic retrieval via vector search
Metadata management

What it does NOT do:

❌ Serve HTTP
❌ Know about prompts or agents
❌ Generate LLM answers

Interface: MCP tools and internal provider boundary (index_workspace, semantic_search, codebase_retrieval)

Layer 2: Context Service Layer

Location: src/mcp/serviceClient.ts, src/mcp/services/

Purpose: Adapts raw retrieval into agent-friendly context and provides planning capabilities.

Responsibilities:

Decide how much context to return
Format snippets for readability
Deduplicate results by file
Enforce limits (max files, max tokens)
Apply heuristics (importance, recency)
Generate and manage implementation plans (v1.4.0+)
Track plan execution and versioning (v1.4.0+)

What it does NOT do:

❌ Index files
❌ Store vectors
❌ Talk to agents directly

Key Services:

ContextServiceClient (`serviceClient.ts`)

semanticSearch(query, topK): SearchResult[]
getFile(path): string
getContextForPrompt(query, maxFiles): ContextBundle

PlanningService (`services/planningService.ts`) - v1.4.0+

generatePlan(task, options): PlanResult
refinePlan(currentPlan, feedback): PlanResult
analyzeDependencies(steps): DependencyGraph

PlanPersistenceService (`services/planPersistenceService.ts`) - v1.4.0+

savePlan(plan, options): SaveResult
loadPlan(planId): EnhancedPlanOutput
listPlans(filters): PlanMetadata[]
deletePlan(planId): boolean

ExecutionTrackingService (`services/executionTrackingService.ts`) - v1.4.0+

initializeExecution(plan): PlanExecutionState
startStep(planId, stepNumber): StepExecutionState
completeStep(planId, stepNumber, options): StepExecutionState
failStep(planId, stepNumber, options): StepExecutionState
getProgress(planId): ExecutionProgress

ApprovalWorkflowService (`services/approvalWorkflowService.ts`) - v1.4.0+

createPlanApprovalRequest(plan): ApprovalRequest
createStepApprovalRequest(plan, stepNumber): ApprovalRequest
respondToApproval(requestId, action, comments): ApprovalResult

PlanHistoryService (`services/planHistoryService.ts`) - v1.4.0+

recordVersion(plan, changeType, description): void
getHistory(planId, options): PlanHistory
generateDiff(planId, fromVersion, toVersion): PlanDiff
rollback(planId, version, reason): EnhancedPlanOutput

ReactiveReviewService (`ReactiveReviewService.ts`) - v1.7.1+

createReviewSession(metadata): string
startReview(sessionId): void
getReviewStatus(sessionId): ReviewStatus
pauseReview(sessionId): void
resumeReview(sessionId): void

CodeReviewService (`services/CodeReviewService.ts`) - v1.7.0+

reviewChanges(diff, options): ReviewResult
reviewGitDiff(options): ReviewResult

Context Bundle Format:

  files: Array<{
    path: string;
    snippets: Array<{
      text: string;
      lines: string;
    }>;
  }>;
  hints: string[];
}

Layer 2.5: Internal Shared Handlers

Location: src/internal/handlers/

Purpose: Provide shared internal logic used by multiple MCP tools while preserving tool-specific formatting in Layer 3.

Responsibilities:

Wrap retrieval pipeline with consistent timing and caching hooks
Provide shared context assembly helpers (bundles + snippets)
Provide prompt enhancement helpers used by AI tools
Offer disabled-by-default performance hooks (cache/batching/embedding reuse)

What it does NOT do:

❌ Expose MCP tools directly
❌ Change tool schemas or outputs
❌ Apply tool-specific formatting

Key Handlers:

internalRetrieveCode(query, serviceClient, options): InternalRetrieveResult
internalContextBundle(query, serviceClient, options): ContextBundle
internalContextSnippet(results, maxFiles, maxChars): string | null
internalPromptEnhancer(prompt, serviceClient): string

Layer 3: MCP Interface Layer

Location: src/mcp/server.ts, src/mcp/tools/

Purpose: Protocol adapter that lets agents communicate with the service layer.

Responsibilities:

Expose tools via MCP protocol
Validate input/output
Map tool calls to service layer methods
Stay stateless

What it does NOT do:

❌ Business logic
❌ Retrieval logic
❌ Formatting decisions

Tools Exposed (38 total):

Core Context Tools

index_workspace - Index workspace files
codebase_retrieval - Semantic search (JSON output)
semantic_search - Semantic search (markdown output)
get_file - Retrieve file contents
get_context_for_prompt - Get context bundle
enhance_prompt - AI-powered prompt enhancement
tool_manifest - List available tools

Index Management Tools

index_status - View index health
reindex_workspace - Rebuild index
clear_index - Clear index state

Memory Tools (v1.4.1)

add_memory - Store persistent memory
list_memories - List stored memories

Planning Tools (v1.4.0+)

create_plan - Generate implementation plans
refine_plan - Refine existing plans
visualize_plan - Generate diagrams
execute_plan - Execute plan steps (automated)

Plan Persistence Tools (v1.4.0+)

save_plan - Save plans to storage
load_plan - Load saved plans
list_plans - List plans with filters
delete_plan - Delete plans

Approval Workflow Tools (v1.4.0+)

request_approval - Create approval requests
respond_approval - Respond to approvals

Execution Tracking Tools (v1.4.0+)

start_step - Mark step as in-progress
complete_step - Mark step as completed
fail_step - Mark step as failed
view_progress - View execution progress

History & Versioning Tools (v1.4.0+)

view_history - View plan version history
compare_plan_versions - Compare versions
rollback_plan - Rollback to previous version

Code Review Tools (v1.7.0+)

review_changes - AI-powered code review from diff
review_git_diff - Automatic git review

Reactive Review Tools (v1.7.1 - v1.8.0)

reactive_review_pr - Start reactive review session
get_review_status - Track reactive progress
pause_review - Pause active session
resume_review - Resume session
get_review_telemetry - Performance metrics
scrub_secrets - Remove secrets from content
validate_content - Multi-tier content validation

Example Tool Definitions:

semantic_search
- Input: { query: string, top_k?: number }
- Output: Formatted search results
- Use case: Find specific code patterns
get_file
- Input: { path: string }
- Output: Complete file contents
- Use case: Retrieve full file after search
create_plan (v1.4.0+)
- Input: { task: string, max_context_files?: number, generate_diagrams?: boolean }
- Output: Structured plan with steps, dependencies, diagrams
- Use case: Generate implementation plans
get_context_for_prompt
- Input: { query: string, max_files?: number }
- Output: Rich context bundle
- Use case: Primary tool for prompt enhancement

Layer 4: Agent Clients

Location: External (Codex CLI, Cursor, etc.)

Purpose: Consume context and generate responses.

Agent Responsibilities:

Decide when to call tools
Decide how to use context
Generate final answers

What the system does NOT do:

❌ Generate answers
❌ Make decisions for agents
❌ Interpret results

Layer 5: Storage Backend

Location: Local workspace state and runtime persistence files

Purpose: Persist embeddings and metadata.

Responsibilities:

Store vector embeddings
Store file metadata
Support fast vector similarity search

Storage Options (historical reference):

Qdrant (recommended)
SQLite (simple)
Hybrid (future)

Data Flow

Indexing Flow

File System
    ↓
Scanner (Layer 1)
    ↓
Chunker (Layer 1)
    ↓
Embedder (Layer 1)
    ↓
Vector Store (Layer 5)

Prompt Enhancement Flow

Agent Prompt
    ↓
MCP Tool Call (Layer 3)
    ↓
Internal Handlers (Layer 2.5)
    ↓
Context Service (Layer 2)
    ↓
Engine Retrieval (Layer 1)
    ↓
Context Bundle (Layer 2)
    ↓
Agent Final Prompt (Layer 4)

Design Principles

1. Separation of Concerns

Each layer has one responsibility only. Never collapse layers.

2. Clean Contracts

Interfaces between layers are well-defined and stable:

Layer 1 ↔ Layer 2: CLI commands and JSON output
Layer 2 ↔ Layer 3: TypeScript interfaces
Layer 3 ↔ Layer 4: MCP protocol

3. Stateless MCP Layer

Layer 3 maintains no state. Each tool call is independent.

4. Agent-Agnostic

No LLM-specific logic anywhere in the stack. Works with any MCP client.

5. Local-First

No cloud dependencies
No exposed network ports
No data leaves the machine
All processing happens locally

Planning Mode Architecture (v1.4.0+)

The planning system adds a complete workflow for AI-assisted software planning and execution tracking.

Planning Services Architecture

┌─────────────────────────────────────────────────────────────┐
│                    MCP Tools Layer                          │
│  create_plan | refine_plan | visualize_plan                 │
│  save_plan | load_plan | list_plans | delete_plan           │
│  request_approval | respond_approval                        │
│  start_step | complete_step | fail_step | view_progress     │
│  view_history | compare_plan_versions | rollback_plan       │
└────────────────────┬────────────────────────────────────────┘
                     │
┌────────────────────┴────────────────────────────────────────┐
│                 Planning Services Layer                     │
│                                                              │
│  ┌──────────────────┐  ┌──────────────────────┐            │
│  │ PlanningService  │  │ PlanPersistenceService│            │
│  │ - generatePlan() │  │ - savePlan()          │            │
│  │ - refinePlan()   │  │ - loadPlan()          │            │
│  │ - analyzeDeps()  │  │ - listPlans()         │            │
│  └──────────────────┘  └──────────────────────┘            │
│                                                              │
│  ┌──────────────────┐  ┌──────────────────────┐            │
│  │ExecutionTracking │  │ApprovalWorkflow      │            │
│  │ - startStep()    │  │ - createRequest()    │            │
│  │ - completeStep() │  │ - respondApproval()  │            │
│  │ - getProgress()  │  │ - trackHistory()     │            │
│  └──────────────────┘  └──────────────────────┘            │
│                                                              │
│  ┌──────────────────┐                                       │
│  │ PlanHistory      │                                       │
│  │ - recordVersion()│                                       │
│  │ - generateDiff() │                                       │
│  │ - rollback()     │                                       │
│  └──────────────────┘                                       │
└─────────────────────────────────────────────────────────────┘
                     │
┌────────────────────┴────────────────────────────────────────┐
│                  Storage Layer                              │
│  .context-engine/plans/                                     │
│  ├── index.json (plan metadata)                             │
│  ├── plan_abc123.json (plan data)                           │
│  └── history/                                               │
│      └── plan_abc123.json (version history)                 │
└─────────────────────────────────────────────────────────────┘

Planning Workflow

Plan Generation
- User provides task description
- PlanningService retrieves relevant codebase context
- AI generates structured plan with steps, dependencies, diagrams
- Plan includes DAG analysis (topological sort, critical path, parallel groups)
Plan Persistence
- Plans saved to .context-engine/plans/ directory
- Metadata tracked in index.json
- Each plan has unique ID and version number
Approval Workflow (Optional)
- Create approval requests for full plans or specific steps
- Track approval status and comments
- Support approve/reject/request_changes actions
Execution Tracking
- Initialize execution state from saved plan
- Track step states: pending → ready → in_progress → completed/failed/skipped
- Automatically unlock dependent steps when prerequisites complete
- Calculate real-time progress percentage
Version History
- Every plan modification creates new version
- Track changes with timestamps and descriptions
- Generate diffs between versions
- Support rollback to previous versions

Key Data Structures

EnhancedPlanOutput:

{
  id: string;
  version: number;
  goal: string;
  scope: { included, excluded, assumptions, constraints };
  mvp_features: string[];
  nice_to_have_features: string[];
  architecture: { notes, patterns_used, diagrams };
  risks: Array<{ issue, mitigation, likelihood }>;
  milestones: Array<{ name, steps_included, estimated_time }>;
  steps: Array<{
    step_number: number;
    title: string;
    description: string;
    files_to_modify: string[];
    files_to_create: string[];
    depends_on: number[];
    blocks: number[];
    can_parallel_with: number[];
    priority: 'high' | 'medium' | 'low';
    estimated_effort: string;
    acceptance_criteria: string[];
  }>;
  dependency_graph: {
    nodes, edges, critical_path, parallel_groups, execution_order
  };
  testing_strategy: { unit, integration, coverage_target };
  confidence_score: number;
  questions_for_clarification: string[];
}

PlanExecutionState:

{
  plan_id: string;
  status: 'ready' | 'executing' | 'completed' | 'failed';
  steps: Array<{
    step_number: number;
    status: 'pending' | 'ready' | 'in_progress' | 'completed' | 'failed' | 'skipped';
    started_at?: string;
    completed_at?: string;
    duration_ms?: number;
    error?: string;
    notes?: string;
  }>;
  current_steps: number[];
  ready_steps: number[];
  blocked_steps: number[];
}

Extension Points

Adding New Tools

Create tool handler in src/mcp/tools/
Define input schema
Implement handler function
Register in src/mcp/server.ts

Example:

// src/mcp/tools/myTool.ts
export const myTool = {
  name: 'my_tool',
  description: 'Does something useful',
  inputSchema: { /* ... */ }
};

export async function handleMyTool(args, serviceClient) {
  // Implementation
}

Adding Service Methods

Add methods to ContextServiceClient in src/mcp/serviceClient.ts:

async myServiceMethod(params): Promise<Result> {
  // Call retrieval/runtime layer or process data
  // Apply Layer 2 logic (formatting, deduplication, etc.)
  return result;
}

Future Enhancements

These can be added without architectural changes:

File watchers (Layer 1)
Incremental indexing (Layer 1)
Multi-repo support (Layer 2)
Role-based filtering (Layer 2)
Hybrid search (Layer 1)
Caching (Layer 2)
Custom context strategies (Layer 2)

Security Considerations

Authentication

Uses configured provider session/environment when AI-assisted flows are enabled
No credentials stored in code
Session file: ~/.augment/session.json

Data Privacy

All data stays local
No legacy-provider network dependency in active local-native retrieval paths
No telemetry or tracking

Input Validation

All tool inputs validated in Layer 3
Path traversal prevention in file access
Query sanitization before CLI execution

Performance Considerations

Indexing

Initial indexing can be slow for large codebases
Incremental updates are faster
Respects .gitignore and .augmentignore

Search

Vector search is fast (< 100ms typically)
Results limited by top_k parameter
Deduplication adds minimal overhead

Context Bundling

Limited by max_files parameter
Snippet extraction is fast
Formatting is lightweight

Monitoring and Debugging

Logs

Server logs to stderr
Codex CLI: Check ~/.codex/config.toml and use codex mcp list
Provider logs: check the configured client/runtime documentation

Debugging Tools

MCP Inspector for interactive testing
Direct runtime testing with node dist/index.js --workspace ... --index
TypeScript source maps for stack traces

Testing Strategy

See docs/archive/TESTING.md for comprehensive testing guide.

References

docs/archive/plan.md - Original architecture plan
MCP Documentation
Context Engine docs index

FilesExpand file tree

ARCHITECTURE.md

Latest commit

History

ARCHITECTURE.md

File metadata and controls

Architecture Documentation

Overview

5-Layer Architecture

Layer 1: Core Context Engine (Local-Native Runtime)

Layer 2: Context Service Layer

ContextServiceClient (serviceClient.ts)

PlanningService (services/planningService.ts) - v1.4.0+

PlanPersistenceService (services/planPersistenceService.ts) - v1.4.0+

ExecutionTrackingService (services/executionTrackingService.ts) - v1.4.0+

ApprovalWorkflowService (services/approvalWorkflowService.ts) - v1.4.0+

PlanHistoryService (services/planHistoryService.ts) - v1.4.0+

ReactiveReviewService (ReactiveReviewService.ts) - v1.7.1+

CodeReviewService (services/CodeReviewService.ts) - v1.7.0+

Layer 2.5: Internal Shared Handlers

Layer 3: MCP Interface Layer

Core Context Tools

Index Management Tools

Memory Tools (v1.4.1)

Planning Tools (v1.4.0+)

Plan Persistence Tools (v1.4.0+)

Approval Workflow Tools (v1.4.0+)

Execution Tracking Tools (v1.4.0+)

History & Versioning Tools (v1.4.0+)

Code Review Tools (v1.7.0+)

Reactive Review Tools (v1.7.1 - v1.8.0)

Layer 4: Agent Clients

Layer 5: Storage Backend

Data Flow

Indexing Flow

Prompt Enhancement Flow

Design Principles

1. Separation of Concerns

2. Clean Contracts

3. Stateless MCP Layer

4. Agent-Agnostic

5. Local-First

Planning Mode Architecture (v1.4.0+)

Planning Services Architecture

Planning Workflow

Key Data Structures

Extension Points

Adding New Tools

Adding Service Methods

Future Enhancements

Security Considerations

Authentication

Data Privacy

Input Validation

Performance Considerations

Indexing

Search

Context Bundling

Monitoring and Debugging

Logs

Debugging Tools

Testing Strategy

References

ContextServiceClient (`serviceClient.ts`)

PlanningService (`services/planningService.ts`) - v1.4.0+

PlanPersistenceService (`services/planPersistenceService.ts`) - v1.4.0+

ExecutionTrackingService (`services/executionTrackingService.ts`) - v1.4.0+

ApprovalWorkflowService (`services/approvalWorkflowService.ts`) - v1.4.0+

PlanHistoryService (`services/planHistoryService.ts`) - v1.4.0+

ReactiveReviewService (`ReactiveReviewService.ts`) - v1.7.1+

CodeReviewService (`services/CodeReviewService.ts`) - v1.7.0+