Skip to content

04-Multi-Agent-Orchestration.md

Latest commit

 

History

History
135 lines (100 loc) · 5.57 KB

04-Multi-Agent-Orchestration.md

File metadata and controls

135 lines (100 loc) · 5.57 KB

Multi-Agent Orchestration: Patterns and Best Practices

Purpose

This document describes how to coordinate multiple Claude agents and commands for highly efficient, scalable, and reliable workflows. It covers orchestration architecture, communication, planning documents, and anti-patterns to avoid.

When Multi-Agent is Appropriate

Important: Anthropic's latest research shows that for most workflows, a single general agent with skills is more efficient than multiple specialized agents. This guide focuses on the specific scenarios where multi-agent orchestration provides value.

Use Multi-Agent When:

  1. Breadth-First Parallelization

    • Research across independent sources
    • Exploring multiple solution approaches
    • Multi-environment deployments (dev/staging/prod)

  2. Scale Requires Concurrency

    • Large codebases needing parallel analysis
    • High-volume data processing
    • Time-sensitive deliverables

  3. Comparison Through Diversity

    • Want multiple implementations to compare
    • Leveraging stochastic variation in LLM outputs
    • A/B testing different approaches

Don't Use Multi-Agent For:

❌ Sequential Workflows: Use single agent + skills

  • Feature implementation (depth-first)
  • Code refactoring with context dependencies
  • Documentation generation
  • Standard testing and validation

❌ Context-Heavy Tasks: Use single agent + progressive skill loading

  • Complex debugging requiring full codebase understanding
  • Architecture design decisions
  • API integration (sequential setup steps)

Modern Best Practice: Hybrid Approach

Use orchestrator-worker pattern, but equip each worker with dynamically-loaded skills:

orchestrator:
  role: orchestrator-lead
  skills: [task-decomposition, dependency-management]

workers:
  - role: general-agent
    skills: [dynamically-loaded-per-task]
    isolation: git-worktree

See: Agent Skills vs. Multi-Agent Guide for detailed comparison and migration strategies.

Context Efficiency Improvements (January 2026)

Recent optimizations have significantly reduced the "context tax" that previously constrained multi-agent architectures:

MCP Tool Search (Lazy Loading):

  • Token consumption reduced from ~134k to ~5k (85% reduction)
  • Accuracy improved: Opus 4.5 went from 79.5% to 88.1% on MCP evaluations
  • Agents can now access thousands of tools without startup penalty
  • Focus shifts from limiting tools to optimizing discoverability

Implications for Multi-Agent:

  • Each worker agent benefits from reduced context overhead
  • More context available for actual task execution
  • Virtual MCP servers become more valuable (bundle tools with focused instructions)
  • Workers can be equipped with richer toolsets without context bloat

See: MCP Registry Best Practices - Section 12 and LLM Production Optimization for implementation details.

Orchestrator-Worker Architecture

Lead Agent (Orchestrator)

  • Receives user or project requests
  • Decomposes requirements into actionable subtasks
  • Assigns subtasks to specialized subagents
  • Monitors subtask progress/completion
  • Synthesizes results
  • Handles error recovery and quality control

Subagents (Workers)

  • Execute focused, well-defined tasks
  • Return structured outputs and metadata
  • Operate in parallel with isolated context as needed

Planning and Task Distribution

Shared Planning Document

  • Use a standard file, e.g., MULTI_AGENT_PLAN.md, to outline:
    • Overall project goal
    • Task breakdown and dependencies
    • Assignment of agents to tasks
    • Status of each subtask (not started / in progress / done)
  • The architect agent maintains the plan, and all agents read/write task status as they work

Task Handoffs and Collaboration

  • Clearly define natural handoff points (e.g., after builder agent finishes code, validator agent takes over)
  • Specify expected outputs and success criteria for each phase
  • Enable agents to communicate status or blockers using update comments or plan fields

Communication Protocols

  • Use structured message formats when agents interact directly
  • Include task IDs, agent roles, objectives, context, expected output, and metadata
  • Trace all exchanges via an audit log or a shared message file

Parallelization and Independence

  • Where possible, assign non-dependent tasks to multiple agents simultaneously
  • Each agent maintains independent context (buffers/window) to avoid information bleed
  • Compare, synthesize, or select the best result among outputs if duplicate tasks are run in parallel (e.g., two independent implementations for risk mitigation)

Anti-Patterns to Avoid

  • Conflicting assignments: No single task should be assigned ambiguously to more than one agent (unless by design)
  • Lost context: Always re-read planning documents and recent changes if the agent context expires or is inconsistent
  • Over-coordination: Avoid deep hierarchies that introduce latency without benefit
  • Poor error handling: Lead agent should always detect and recover from agent failures

Monitoring and Iteration

  • Agents should record the outcome of their step in the plan
  • Periodically run static checks and status audits across planning documents
  • The orchestrator agent or a designated reviewer should periodically summarize completed, ongoing, and blocked work

See Document 5 for agent testing, validation, and quality assurance workflows.

Document Version: 1.1.0 Last Updated: January 17, 2026 Maintained By: Claude Command and Control Project