Agent Architecture Design Document

[zerob13]

Overview

This document outlines the proposed architecture for transforming the current LLM provider system into a comprehensive multi-agent framework. The goal is to create a flexible, scalable agent system that can handle multiple concurrent agents with different capabilities, input queuing, and both synchronous and asynchronous tool execution.

Current Architecture Analysis

Current Structure

ThreadPresenter as Basic Master Loop Agent (src/main/presenter/threadPresenter/index.ts)

• Conversation Management: Manages conversation state and message flow
• LLM Coordination: Coordinates with LLMProviderPresenter for agent execution
• Session Context: Maintains conversation context and history
• Basic Input Handling: Handles user input and message sequencing

LLMProviderPresenter as Single Action Agent (src/main/presenter/llmProviderPresenter/index.ts)

• Single Agent Loop: Basic agent loop in startStreamCompletion method (lines 694-2033)
• Synchronous Tool Execution: Tools are executed synchronously within the agent loop
• No Input Queue: User input cannot be processed during streaming
• Limited Concurrency: Basic rate limiting but no true multi-agent support
• Provider-Centric: Focused on LLM providers rather than agent capabilities

Key Limitations

1. Blocking During Stream: Once streaming starts, no new inputs can be processed
2. Single Agent: Only one agent can operate per conversation/thread
3. Synchronous Tools: All tool execution blocks the agent loop
4. No Agent Persistence: Agents are ephemeral and tied to specific conversations
5. Limited Tool Integration: Primarily MCP tools with synchronous execution

Proposed Architecture

Core Concepts

1. Agent Hierarchy and Management

• Master Loop Agent: Central coordinator that manages all sub-agents, input queue, and session state
• Action Agents: Specialized agents with specific purposes, tools, and well-defined input/output contracts
• Agent Orchestration: Master agent delegates tasks to appropriate action agents based on context

2. Agent Definition

An agent is defined by:

• Agent Type: Master, Action, or Specialized
• Provider Configuration: Configurable LLM provider and model selection
• System Prompt: Clear description of agent purpose, capabilities, and expected outputs
• Tool Registry: Available capabilities (MCP, async, custom) with access control
• Input/Output Contract: Well-defined input requirements and output specifications
• State Management: Persistent state across interactions with isolation boundaries

3. Action Agent Specifications

Action agents have:

• Single Responsibility: Focused on specific task types (research, coding, analysis, etc.)
• Deterministic Output: Clear final output format without intermediate streaming
• Process Isolation: Internal reasoning and tool usage are encapsulated
• Resource Limits: Configurable timeouts, token limits, and tool usage constraints
• Result Caching: Intelligent caching of completed agent executions

4. Enhanced Tool System

• Synchronous Tools: Immediate execution (current MCP tools)
• Asynchronous Tools: Background execution with result queuing and callback system
• Tool Categories: MCP, CLI, API, Custom with proper sandboxing
• Tool Lifecycle: Registration, validation, execution, monitoring, cleanup
• Provider Integration: Tools can leverage different LLM providers based on agent configuration

Architecture Components

1. Master Loop Agent Management Strategy

The Master Loop Agent serves as the central coordinator that manages:

Input Queue Management

• Priority-based Input Queue: User inputs are queued with configurable priority levels
• Context-aware Processing: Queue processing considers conversation context and agent availability
• Non-blocking Input: Users can provide input during agent processing without blocking
• Queue Statistics: Real-time monitoring of queue length, wait times, and processing rates

Sub-agent Management

• Dynamic Agent Registration: Action agents register with the master agent at runtime
• Agent Discovery: Automatic discovery of available action agents with capability matching
• Resource Allocation: Intelligent allocation of system resources to action agents
• Health Monitoring: Continuous monitoring of agent health and performance

Session State Management

• Conversation Context: Maintains shared context across all action agents
• Agent State Persistence: Saves and restores agent states across sessions
• Tool Execution History: Tracks tool usage and results for intelligent reuse
• User Preferences: Stores user-specific agent configuration preferences

Delegation and Routing

• Specification-based Routing: Routes inputs to appropriate agents based on agent specifications
• Fallback Strategies: Automatic fallback to alternative agents when primary agents fail
• Collaboration Patterns: Orchestrates multi-agent collaboration for complex tasks
• Progress Tracking: Monitors agent execution and provides real-time status updates

2. Agent Manager

interface AgentManager {
  createAgent(type: string, config: AgentConfig): Agent
  getAgent(agentId: string): Agent | null
  listAgents(): Agent[]
  removeAgent(agentId: string): void
  registerAgentType(type: string, factory: AgentFactory): void
}

2. Agent Interface

interface Agent {
  id: string
  type: 'master' | 'action' | 'specialized'
  state: AgentState
  
  // Core methods
  processInput(input: AgentInput): Promise<AgentOutput>
  queueInput(input: AgentInput): void
  getQueuedInputs(): AgentInput[]
  
  // Tool management
  registerTool(tool: ToolDefinition): void
  executeTool(toolName: string, params: any): Promise<ToolResult>
  
  // Provider configuration
  getProviderConfig(): ProviderConfig
  setProviderConfig(config: Partial<ProviderConfig>): void
  
  // Agent specification
  getSpecification(): AgentSpecification
  updateSpecification(spec: Partial<AgentSpecification>): void
  
  // State management
  getState(): AgentState
  setState(state: Partial<AgentState>): void
  persistState(): Promise<void>
  
  // Lifecycle
  start(): Promise<void>
  stop(): Promise<void>
  isRunning(): boolean
}

interface AgentSpecification {
  name: string
  description: string
  purpose: string
  capabilities: string[]
  expectedOutputFormat: string
  systemPrompt: string
  inputRequirements: string
  outputSpecification: string
  constraints: string[]
  timeoutMs: number
  maxTokenUsage: number
  maxIterations: number
  allowedTools: string[]
  providerPreferences: ProviderPreference[]
  
  // Enhanced specification for action agents
  inputContract: {
    requiredFields: string[]
    optionalFields: string[]
    validationRules: ValidationRule[]
    contextRequirements: ContextRequirement[]
  }
  
  outputContract: {
    requiredFields: string[]
    format: 'text' | 'json' | 'markdown' | 'html' | 'structured'
    schema?: JSONSchema
    qualityStandards: QualityStandard[]
  }
  
  // Performance and resource constraints
  resourceLimits: {
    maxMemoryMB: number
    maxExecutionTimeMs: number
    maxConcurrentTools: number
    rateLimits: RateLimitConfig[]
  }
  
  // Monitoring and observability
  metricsToTrack: string[]
  successCriteria: SuccessCriterion[]
  failureConditions: FailureCondition[]
}

interface ProviderConfig {
  providerId: string
  modelId: string
  temperature: number
  maxTokens: number
  contextLength: number
  fallbackProviders: string[]
  
  // Enhanced provider configuration
  providerSpecificConfig: {
    [key: string]: any
  }
  
  // Performance optimization
  cachingEnabled: boolean
  cacheTTL: number
  retryConfig: {
    maxRetries: number
    retryDelayMs: number
    backoffStrategy: 'linear' | 'exponential'
  }
  
  // Cost and usage tracking
  costTracking: {
    costPerToken: number
    costPerRequest: number
    budgetLimit?: number
  }
  
  // Model capabilities
  supportsToolCalling: boolean
  supportsStreaming: boolean
  supportsVision: boolean
  maxParallelRequests: number
}

3. Enhanced Tool System

interface ToolSystem {
  // Tool registration
  registerTool(tool: ToolDefinition): void
  unregisterTool(toolName: string): void
  
  // Tool execution
  executeSync(toolName: string, params: any): ToolResult
  executeAsync(toolName: string, params: any): Promise<string> // returns taskId
  
  // Task management
  getTaskStatus(taskId: string): TaskStatus
  cancelTask(taskId: string): boolean
  
  // Result retrieval
  getTaskResult(taskId: string): ToolResult | null
  subscribeToTask(taskId: string, callback: TaskCallback): void
}

interface ToolDefinition {
  name: string
  description: string
  category: 'mcp' | 'cli' | 'api' | 'custom'
  async: boolean
  timeout?: number
  execute: (params: any) => Promise<any> | any
  validate?: (params: any) => ValidationResult
}

4. Input Queue System

interface InputQueue {
  // Queue management
  enqueue(input: AgentInput, priority?: number): string // returns queueId
  dequeue(): AgentInput | null
  peek(): AgentInput | null
  clear(): void
  
  // Priority handling
  setPriority(queueId: string, priority: number): boolean
  
  // Event system
  onInputAdded(callback: QueueCallback): void
  onInputProcessed(callback: QueueCallback): void
  
  // Statistics
  getQueueLength(): number
  getAverageWaitTime(): number
}

5. Master Loop Agent

interface MasterLoopAgent extends Agent {
  // Sub-agent management
  registerSubAgent(agent: Agent): void
  unregisterSubAgent(agentId: string): void
  getSubAgents(): Agent[]
  findSuitableAgent(input: AgentInput): Agent | null
  
  // Input queue management
  processUserInput(input: UserInput): Promise<AgentOutput>
  getInputQueueStatus(): QueueStatus
  prioritizeInput(queueId: string, priority: number): boolean
  
  // Session management
  getSessionState(): SessionState
  updateSessionState(updates: Partial<SessionState>): void
  persistSession(): Promise<void>
  
  // Agent orchestration
  delegateToActionAgent(actionType: string, input: AgentInput): Promise<AgentOutput>
  monitorAgentProgress(agentId: string): AgentProgress
  handleAgentCompletion(agentId: string, result: AgentOutput): void
  handleAgentFailure(agentId: string, error: Error): void
}

interface SessionState {
  conversationHistory: ChatMessage[]
  activeAgents: string[]
  completedTasks: CompletedTask[]
  pendingInputs: QueuedInput[]
  resourceUsage: ResourceMetrics
  userPreferences: UserPreferences
}

interface CompletedTask {
  taskId: string
  agentId: string
  input: AgentInput
  output: AgentOutput
  startTime: number
  endTime: number
  resourceUsage: ResourceUsage
  success: boolean
  error?: string
}

6. Action Agent Execution Model

Action agents follow a strict execution model with clear input/output contracts:

Input Processing Pipeline

interface AgentInput {
  // Required core fields
  task: string
  context: ConversationContext
  
  // Optional enhancement fields
  constraints?: string[]
  qualityRequirements?: QualityRequirement[]
  formatPreferences?: FormatPreference[]
  
  // Execution context
  priority: number
  timeoutMs?: number
  metadata?: Record<string, any>
}

interface ConversationContext {
  history: ChatMessage[]
  sharedState: Record<string, any>
  toolResults: ToolExecutionResult[]
  userPreferences: UserAgentPreferences
}

Internal Execution Isolation

Action agents encapsulate all internal reasoning and tool execution:

• Internal Loop: Multiple iterations with tool usage and reasoning
• State Isolation: Internal state not exposed to master agent
• Tool Execution: Full tool access within agent boundaries
• Result Processing: Only final formatted output is returned

Output Contract Enforcement

interface AgentOutput {
  // Final content according to specification
  content: string | object
  
  // Execution metadata
  metadata: {
    agentId: string
    executionTimeMs: number
    tokenUsage: number
    toolsUsed: string[]
    iterationCount: number
    success: boolean
    error?: string
  }
  
  // Quality assurance
  validation: {
    meetsSpecification: boolean
    qualityScore: number
    warnings: string[]
  }
}

7. Action Agent Implementation

class ActionAgent extends BaseAgent {
  private specification: AgentSpecification
  private providerConfig: ProviderConfig
  private toolAccess: Set<string>
  
  constructor(config: ActionAgentConfig) {
    super(config)
    this.specification = config.specification
    this.providerConfig = config.providerConfig
    this.toolAccess = new Set(config.specification.allowedTools)
  }

  async processInput(input: AgentInput): Promise<AgentOutput> {
    // Validate input against specification
    this.validateInput(input)
    
    // Execute agent logic with proper isolation
    const result = await this.executeWithIsolation(input)
    
    // Return only final output, internal reasoning is encapsulated
    return this.formatFinalOutput(result)
  }

  private async executeWithIsolation(input: AgentInput): Promise<InternalResult> {
    // Internal execution with full tool access and reasoning
    // This is encapsulated and not exposed to master agent
    const context = this.buildExecutionContext(input)
    
    let iteration = 0
    let finalResult: InternalResult | null = null
    
    while (iteration < this.specification.maxIterations && !finalResult) {
      const iterationResult = await this.executeSingleIteration(context)
      
      if (this.isFinalResult(iterationResult)) {
        finalResult = iterationResult
      } else {
        // Update context and continue
        context.update(iterationResult)
        iteration++
      }
    }
    
    return finalResult || this.createTimeoutResult()
  }

  private formatFinalOutput(internalResult: InternalResult): AgentOutput {
    // Extract only the final output according to specification
    return {
      content: internalResult.finalContent,
      metadata: {
        agentId: this.id,
        executionTime: internalResult.executionTime,
        tokenUsage: internalResult.tokenUsage,
        toolsUsed: internalResult.toolsUsed
      }
    }
  }
}

8. Provider Configuration Management System

The provider configuration system enables dynamic provider and model selection:

Provider Registry

interface ProviderRegistry {
  // Provider discovery and registration
  registerProvider(provider: ProviderDefinition): void
  unregisterProvider(providerId: string): void
  getAvailableProviders(): ProviderDefinition[]
  
  // Model management
  getModelsForProvider(providerId: string): ModelDefinition[]
  registerModel(providerId: string, model: ModelDefinition): void
  
  // Capability-based selection
  findProvidersWithCapability(capability: string): ProviderDefinition[]
  findBestProviderForTask(taskType: string, constraints: ProviderConstraints): ProviderDefinition | null
}

interface ProviderDefinition {
  id: string
  name: string
  type: 'cloud' | 'local' | 'hybrid'
  capabilities: string[]
  authentication: AuthenticationConfig
  rateLimits: RateLimitConfig
  costStructure: CostConfig
  supportedModels: ModelDefinition[]
}

interface ModelDefinition {
  id: string
  name: string
  providerId: string
  contextLength: number
  capabilities: {
    toolCalling: boolean
    streaming: boolean
    vision: boolean
    reasoning: 'basic' | 'advanced' | 'expert'
  }
  performance: {
    tokensPerSecond: number
    latencyMs: number
    reliability: number
  }
  cost: {
    inputTokensPerMillion: number
    outputTokensPerMillion: number
  }
}

Dynamic Provider Selection

• Capability Matching: Select providers based on task requirements
• Cost Optimization: Choose most cost-effective provider for task
• Performance-Based: Select providers based on performance metrics
• Fallback Chains: Automatic fallback to alternative providers
• Load Balancing: Distribute requests across available providers

Provider-Aware Tool Execution

interface ProviderAwareToolSystem extends ToolSystem {
  executeToolWithProvider(
    toolName: string, 
    params: any, 
    providerConfig: ProviderConfig
  ): Promise<ToolResult>
  
  getBestProviderForTool(toolName: string): ProviderDefinition | null
  
  // Provider-specific tool adaptations
  adaptToolForProvider(
    tool: ToolDefinition, 
    provider: ProviderDefinition
  ): AdaptedToolDefinition
}

9. Event System Enhancement

interface EnhancedEventSystem {
  // Standard events
  AGENT_CREATED: 'agent:created'
  AGENT_STARTED: 'agent:started'
  AGENT_STOPPED: 'agent:stopped'
  INPUT_QUEUED: 'input:queued'
  INPUT_PROCESSED: 'input:processed'
  TOOL_EXECUTED: 'tool:executed'
  TOOL_FAILED: 'tool:failed'
  ASYNC_TASK_STARTED: 'async:task:started'
  ASYNC_TASK_COMPLETED: 'async:task:completed'
  
  // Agent-specific events
  ACTION_AGENT_STARTED: 'action:agent:started'
  ACTION_AGENT_COMPLETED: 'action:agent:completed'
  ACTION_AGENT_FAILED: 'action:agent:failed'
  MASTER_DELEGATION: 'master:delegation'
  SESSION_UPDATED: 'session:updated'
  
  // Enhanced event payloads
  interface AgentEvent {
    agentId: string
    timestamp: number
    type: string
    data: any
  }

  interface ActionAgentEvent extends AgentEvent {
    actionType: string
    inputSummary: string
    outputSummary: string
    executionMetrics: ExecutionMetrics
  }
}

Implementation Plan

Phase 1: Foundation (2-3 weeks)

1. Extract Core Agent Logic from current startStreamCompletion

   • Create BaseAgent class with loop functionality
   • Implement input queue system with priority handling
   • Standardize event interface for agent communication

2. Create Master Loop Agent Framework

   • Master agent with sub-agent management
   • Session state management and persistence
   • Input queue orchestration and delegation logic

3. Implement Action Agent Base Class

   • Agent specification system with provider configuration
   • Input validation and output formatting
   • Internal execution isolation with proper encapsulation

4. Enhanced Tool System

   • Tool registration with access control
   • Async tool execution framework with callback system
   • Provider-aware tool execution based on agent configuration

Phase 2: Multi-Agent Support (2-3 weeks)

1. Agent Coordination and Orchestration

   • Master agent delegation and routing logic
   • Agent selection based on specification matching
   • Resource sharing with conflict resolution
   • Agent collaboration patterns and task chaining

2. Advanced Tool Integration

   • CLI tool execution with proper output capture and parsing
   • Long-running async task management with progress tracking
   • Tool result caching and intelligent reuse strategies
   • Provider-specific tool optimizations

3. State Persistence and Session Management

   • Agent state storage with proper isolation boundaries
   • Conversation context management across agent boundaries
   • Tool execution history with performance analytics
   • Session recovery and continuity management

Phase 3: Advanced Features (3-4 weeks)

1. Agent Specialization and Configuration

   • Pre-defined agent types (ResearchAgent, CodeAgent, AnalysisAgent, CreativeAgent)
   • Custom agent configuration with template system
   • Skill-based agent routing and capability matching
   • Dynamic agent specification updates and hot-reloading

2. Performance Optimization and Resource Management

   • Intelligent input queue prioritization based on context
   • Resource pooling with provider-specific optimizations
   • Efficient tool execution with result caching and reuse
   • Adaptive resource allocation based on agent requirements

3. Comprehensive Monitoring and Analytics

   • Agent performance metrics with provider-specific benchmarks
   • Tool usage statistics and cost optimization
   • System health monitoring with predictive analytics
   • User experience metrics and quality of service tracking

Migration Strategy

Step 1: Backward Compatibility with Master Agent Wrapper

• Keep existing LLMProviderPresenter interface unchanged
• Create MasterLoopAgent that wraps legacy functionality
• Route calls through master agent with proper input/output conversion
• Maintain current event structure with additional agent-specific events

Step 2: Gradual Agent Migration

• Implement new action agents alongside existing code
• Feature flag system for gradual agent adoption
• Parallel execution with legacy system for comparison
• Progressive migration of tool execution to new framework

Step 3: Full Transition to Multi-Agent System

• Deprecate old single-agent implementation
• Migrate all functionality to master+action agent architecture
• Remove legacy code after comprehensive testing
• Optimize performance based on multi-agent metrics

Key Technical Challenges

1. Master Agent Orchestration

• Challenge: Efficient delegation and routing between multiple action agents
• Solution: Specification-based agent matching with context awareness
• Implementation: Intelligent agent selection algorithm with fallback strategies

2. Action Agent Isolation

• Challenge: Encapsulating internal reasoning while exposing only final outputs
• Solution: Strict input/output contracts with validation layers
• Implementation: Execution context isolation with controlled tool access

3. Provider Configuration Management

• Challenge: Dynamic provider and model selection per agent
• Solution: Provider preference system with fallback mechanisms
• Implementation: Provider-aware tool execution with automatic failover

4. Async Tool Integration with Agent Context

• Challenge: Managing long-running tasks across agent boundaries
• Solution: Task registry with agent context preservation
• Implementation: Context-aware callback system with state recovery

5. Resource Contention and Optimization

• Challenge: Multiple agents competing for limited provider resources
• Solution: Provider-aware resource pooling with priority scheduling
• Implementation: Adaptive resource allocation based on agent requirements

Benefits of New Architecture

1. True Multi-Agent Support: Multiple agents can operate concurrently
2. Non-Blocking Input: Users can provide input during agent processing
3. Enhanced Tool Capabilities: Both sync and async tool execution
4. Better Resource Management: Efficient use of system resources
5. Improved Scalability: Horizontal scaling of agent instances
6. Enhanced Monitoring: Comprehensive observability of agent behavior
7. Flexible Deployment: Agents can be distributed across processes/nodes

Next Steps

1. Review and Feedback: Discuss this design with the team
2. Prototype Implementation: Build basic agent framework
3. Testing Strategy: Develop comprehensive test plan
4. Performance Benchmarks: Establish baseline metrics
5. Rollout Plan: Phased deployment strategy

Architecture Diagrams

Current Architecture Flow

┌─────────────────┐    ┌─────────────────────┐    ┌──────────────────────┐
│   User Input    │───▶│   ThreadPresenter    │───▶│  LLMProviderPresenter │
│                 │    │ (Basic Master Agent) │    │  (Single Action Agent)│
└─────────────────┘    └─────────────────────┘    └──────────────────────┘
                                 │                           │
                                 ▼                           ▼
                         ┌───────────────┐           ┌─────────────────┐
                         │ Conversation  │           │ Tool Execution  │
                         │   State       │           │   & LLM Calls   │
                         └───────────────┘           └─────────────────┘

Proposed Multi-Agent Architecture Flow

┌─────────────────┐    ┌─────────────────────────────────────────────────┐
│   User Input    │───▶│               Master Loop Agent                 │
│                 │    │           (Enhanced ThreadPresenter)            │
└─────────────────┘    ├─────────────────────────────────────────────────┤
                       │  • Input Queue with Priority Handling           │
                       │  • Agent Discovery & Routing                   │
                       │  • Session State Management                    │
                       │  • Resource Allocation & Monitoring            │
                       └───────────────┬─────────────────────────────────┘
                                       │
                                       │ Delegates to appropriate agent
                                       ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                      Action Agent Registry                              │
│                                                                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐    │
│  │ Research    │  │   Code      │  │  Analysis   │  │  Creative   │    │
│  │   Agent     │  │   Agent     │  │   Agent     │  │   Agent     │    │
│  │             │  │             │  │             │  │             │    │
│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘    │
│                                                                         │
│  Each agent:                                                            │
│  • Has specific capabilities & tools                                   │
│  • Configurable provider preferences                                   │
│  • Internal execution isolation                                        │
│  • Well-defined input/output contracts                                 │
└─────────────────────────────────────────────────────────────────────────┘

Agent Execution Sequence Diagram

┌───────┐    ┌───────────────┐    ┌─────────────┐    ┌─────────────┐    ┌───────┐
│ User  │    │ Master Agent  │    │ Action Agent│    │ Tool System │    │ LLM   │
│       │    │ (ThreadPres)  │    │ Registry    │    │             │    │Provider│
└───┬───┘    └───────┬───────┘    └──────┬──────┘    └──────┬──────┘    └───┬───┘
    │     Input       │                   │                  │               │
    │───────────────▶│                   │                  │               │
    │                │  Find suitable    │                  │               │
    │                │─────agent───────▶│                  │               │
    │                │                   │                  │               │
    │                │    Return agent   │                  │               │
    │                │◀─────handle───────│                  │               │
    │                │                   │                  │               │
    │                │  Delegate task    │                  │               │
    │                │─────────────────▶│                  │               │
    │                │                   │  Execute with    │               │
    │                │                   │───isolation────▶│               │
    │                │                   │                  │  Call tools   │
    │                │                   │                  │─────────────▶│
    │                │                   │                  │               │
    │                │                   │                  │  Tool results │
    │                │                   │                  │◀──────────────│
    │                │                   │  Return final    │               │
    │                │◀───output─────────│                  │               │
    │    Response    │                   │                  │               │
    │◀───────────────│                   │                  │               │
└───┴───┘    └───────┴───────┘    └──────┴──────┘    └──────┴──────┘    └───┴───┘

Provider Configuration Flow

┌─────────────────────────────────────────────────────────────────────────┐
│                      Provider Selection Algorithm                        │
│                                                                         │
│  Input: Task requirements, constraints, agent preferences               │
│                                                                         │
│  1. Filter providers by capabilities                                    │
│  2. Check provider health & availability                                │
│  3. Apply selection strategy:                                           │
│     • Cost optimization                                                 │
│     • Performance optimization                                          │
│     • Reliability optimization                                          │
│  4. Select best provider                                                │
│  5. Configure provider-specific tool adaptations                         │
│                                                                         │
│  Output: Configured provider with fallback chain                        │
└─────────────────────────────────────────────────────────────────────────┘

This architecture provides a solid foundation for building a sophisticated multi-agent system that addresses the limitations of the current implementation while maintaining backward compatibility and enabling future enhancements.

[zerob13]

代理架构设计文档

概述

本文档概述了将当前LLM提供商系统转换为综合多代理框架的建议架构。目标是创建一个灵活、可扩展的代理系统，能够处理多个具有不同能力、输入队列以及同步和异步工具执行的并发代理。

当前架构分析

当前结构

ThreadPresenter作为基本主循环代理 (src/main/presenter/threadPresenter/index.ts)
• 对话管理：管理对话状态和消息流
• LLM协调：与LLMProviderPresenter协调进行代理执行
• 会话上下文：维护对话上下文和历史记录
• 基本输入处理：处理用户输入和消息序列

LLMProviderPresenter作为单动作代理 (src/main/presenter/llmProviderPresenter/index.ts)
• 单代理循环：在startStreamCompletion方法中的基本代理循环（第694-2033行）
• 同步工具执行：工具在代理循环内同步执行
• 无输入队列：流式传输期间无法处理用户输入
• 有限并发性：基本速率限制但无真正的多代理支持
• 以提供商为中心：专注于LLM提供商而非代理能力

主要限制

1. 流式传输期间阻塞：一旦开始流式传输，无法处理新输入
2. 单代理：每个对话/线程只能操作一个代理
3. 同步工具：所有工具执行都会阻塞代理循环
4. 无代理持久性：代理是临时的，与特定对话绑定
5. 有限工具集成：主要是MCP工具，同步执行

建议的架构

核心概念

1. 代理层次结构和管理
   • 主循环代理：管理所有子代理、输入队列和会话状态的中央协调器
   • 动作代理：具有特定用途、工具和明确定义输入/输出合约的专用代理
   • 代理编排：主代理根据上下文将任务委托给适当的动作代理

2. 代理定义
   代理由以下内容定义：
   • 代理类型：主代理、动作代理或专用代理
   • 提供商配置：可配置的LLM提供商和模型选择
   • 系统提示：代理目的、能力和预期输出的清晰描述
   • 工具注册表：可用能力（MCP、异步、自定义）与访问控制
   • 输入/输出合约：明确定义的输入要求和输出规范
   • 状态管理：跨交互的持久状态与隔离边界

3. 动作代理规范
   动作代理具有：
   • 单一职责：专注于特定任务类型（研究、编码、分析等）
   • 确定性输出：清晰的最终输出格式，无中间流式传输
   • 进程隔离：内部推理和工具使用是封装的
   • 资源限制：可配置的超时、令牌限制和工具使用约束
   • 结果缓存：已完成代理执行的智能缓存

4. 增强的工具系统
   • 同步工具：立即执行（当前MCP工具）
   • 异步工具：后台执行，结果队列和回调系统
   • 工具类别：MCP、CLI、API、自定义，具有适当的沙盒
   • 工具生命周期：注册、验证、执行、监控、清理
   • 提供商集成：工具可以基于代理配置利用不同的LLM提供商

架构组件

1. 主循环代理管理策略
   主循环代理作为中央协调器管理：

输入队列管理
• 基于优先级的输入队列：用户输入按可配置优先级排队
• 上下文感知处理：队列处理考虑对话上下文和代理可用性
• 非阻塞输入：用户可在代理处理期间提供输入而不阻塞
• 队列统计：队列长度、等待时间和处理速率的实时监控

子代理管理
• 动态代理注册：动作代理在运行时向主代理注册
• 代理发现：自动发现具有能力匹配的可用动作代理
• 资源分配：智能分配系统资源给动作代理
• 健康监控：持续监控代理健康状态和性能

会话状态管理
• 对话上下文：维护所有动作代理间的共享上下文
• 代理状态持久性：跨会话保存和恢复代理状态
• 工具执行历史：跟踪工具使用和结果以便智能重用
• 用户偏好：存储用户特定的代理配置偏好

委托和路由
• 基于规范的路由：根据代理规范将输入路由到适当的代理
• 回退策略：主代理失败时自动回退到备用代理
• 协作模式：为复杂任务编排多代理协作
• 进度跟踪：监控代理执行并提供实时状态更新

[继续输入接口定义和实现细节...]

实现计划

第一阶段：基础（2-3周）

1. 从当前startStreamCompletion提取核心代理逻辑
   • 创建具有循环功能的BaseAgent类
   • 实现具有优先级处理的输入队列系统
   • 标准化代理通信的事件接口

2. 创建主循环代理框架
   • 具有子代理管理的主代理
   • 会话状态管理和持久性
   • 输入队列编排和委托逻辑

3. 实现动作代理基类
   • 具有提供商配置的代理规范系统
   • 输入验证和输出格式化
   • 具有适当封装的内部执行隔离

4. 增强的工具系统
   • 具有访问控制的工具注册
   • 具有回调系统的异步工具执行框架
   • 基于代理配置的提供商感知工具执行

第二阶段：多代理支持（2-3周）

1. 代理协调和编排
   • 主代理委托和路由逻辑
   • 基于规范匹配的代理选择
   • 具有冲突解决的资源共享
   • 代理协作模式和任务链

2. 高级工具集成
   • 具有适当输出捕获和解析的CLI工具执行
   • 具有进度跟踪的长时间运行异步任务管理
   • 工具结果缓存和智能重用策略
   • 提供商特定的工具优化

3. 状态持久性和会话管理
   • 具有适当隔离边界的代理状态存储
   • 跨代理边界的对话上下文管理
   • 具有性能分析的工具执行历史
   • 会话恢复和连续性管理

第三阶段：高级功能（3-4周）

1. 代理专业化和配置
   • 预定义代理类型（研究代理、代码代理、分析代理、创意代理）
   • 具有模板系统的自定义代理配置
   • 基于技能的代理路由和能力匹配
   • 动态代理规范更新和热重载

2. 性能优化和资源管理
   • 基于上下文的智能输入队列优先级
   • 具有提供商特定优化的资源池
   • 具有结果缓存和重用的高效工具执行
   • 基于代理要求的自适应资源分配

3. 综合监控和分析
   • 具有提供商特定基准的代理性能指标
   • 工具使用统计和成本优化
   • 具有预测分析的系统健康监控
   • 用户体验指标和服务质量跟踪

迁移策略

步骤1：使用主代理包装器的向后兼容性
• 保持现有LLMProviderPresenter接口不变
• 创建包装遗留功能的MasterLoopAgent
• 通过主代理路由调用，进行适当的输入/输出转换
• 维护当前事件结构，增加代理特定事件

步骤2：渐进式代理迁移
• 在现有代码基础上实现新的动作代理
• 功能标志系统用于渐进式代理采用
• 与遗留系统并行执行以进行比较
• 将工具执行逐步迁移到新框架

步骤3：完全过渡到多代理系统
• 弃用旧的单代理实现
• 将所有功能迁移到主代理+动作代理架构
• 经过全面测试后删除遗留代码
• 基于多代理指标优化性能

主要技术挑战

1. 主代理编排
   • 挑战：多个动作代理间的高效委托和路由
   • 解决方案：基于规范的代理匹配与上下文感知
   • 实现：具有回退策略的智能代理选择算法

2. 动作代理隔离
   • 挑战：封装内部推理同时仅公开最终输出
   • 解决方案：具有验证层的严格输入/输出合约
   • 实现：具有受控工具访问的执行上下文隔离

3. 提供商配置管理
   • 挑战：每个代理的动态提供商和模型选择
   • 解决方案：具有回退机制的提供商偏好系统
   • 实现：具有自动故障转移的提供商感知工具执行

4. 异步工具与代理上下文集成
   • 挑战：管理跨代理边界的长时间运行任务
   • 解决方案：具有代理上下文保护的任务注册表
   • 实现：具有状态恢复的上下文感知回调系统

5. 资源争用和优化
   • 挑战：多个代理竞争有限的提供商资源
   • 解决方案：具有优先级调度的提供商感知资源池
   • 实现：基于代理要求的自适应资源分配

新架构的好处

1. 真正的多代理支持：多个代理可以并发操作
2. 非阻塞输入：用户可在代理处理期间提供输入
3. 增强的工具能力：同步和异步工具执行
4. 更好的资源管理：系统资源的有效使用
5. 改进的可扩展性：代理实例的水平扩展
6. 增强的监控：代理行为的综合可观察性
7. 灵活的部署：代理可在进程/节点间分布

下一步

1. 审查和反馈：与团队讨论此设计
2. 原型实现：构建基本代理框架
3. 测试策略：开发综合测试计划
4. 性能基准：建立基线指标
5. 推出计划：分阶段部署策略