Prioritize and Expose the Experimental Agent Framework

[NAME0x0]

Hello Gemini CLI Team,

First, I want to say this is an incredible tool. I've been digging into the repository and am extremely excited by the experimental work I've found in the packages/core/src/agents/ and packages/a2a-server/ directories.

It's clear a very sophisticated agent architecture is already being built, complete with a core AgentExecutor, AgentDefinition types, and even a concrete codebase_investigator agent. This is exactly the kind of functionality that would be a game-changer for the CLI.

My idea is to advocate for prioritizing, stabilizing, and officially exposing this agent framework to all users.

The benefits you're already building toward are immense:

• Complex, Multi-Step Tasks: An agentic framework is the perfect solution for handling complex goals that are difficult for a standard chat prompt. Agents can run in a loop, use tools, and work autonomously to solve a problem, which is ideal for overcoming context limitations in longer, more complex tasks.
• Rivaling Other Tools: Exposing agents like the codebase_investigator would allow users to perform deep, autonomous analysis on their repositories. This would provide capabilities that rival the advanced, multi-step coding and analysis features seen in tools like Claude or GitHub Copilot, but directly in our terminals.
• Productivity: This would make users significantly more productive by offloading complex analysis and investigation tasks to an autonomous agent that can report back with structured findings.

What are the team's current plans for this agent framework? Is there a roadmap for moving it from experimental to stable?

I believe the community would be very excited to help test, provide feedback, and even contribute to this feature if it's a priority. Thank you for all the great work!

[NAME0x0]

Proposal: The Gemini CLI Agentic Framework

The goal of this architecture is to evolve the Gemini CLI from a conversational tool into a fully autonomous, multi-agent system capable of handling complex, multi-step developer tasks.

This proposal builds directly on the excellent experimental foundation in packages/core/src/agents/. It leverages the existing AgentExecutor as the core runtime and the AgentDefinition as the blueprint for specialized agents.

---

Core Philosophy: The "Dispatcher & Specialists" Model

The architecture is modeled after a software development team. A user's prompt is not handled by a single, generalist AI, but by a "Dispatcher" (like a team lead) that breaks the problem down and assigns work to a team of "Specialist Agents" (like individual developers).

• The User provides a high-level goal (e.g., "Add a new API endpoint and write tests for it").
• The Dispatcher Agent (the "Team Lead") analyzes the goal, creates a plan, and routes sub-tasks to the correct specialist.
• The Specialist Agents (the "Team Members") are non-interactive agents (like codebase_investigator) that execute their specific task (e.g., "find relevant files" or "write new code") and report back.

This model allows for greater power, specialization, and reliability.

---

The Four-Tier Architecture

This entire system can be visualized in four distinct layers that build upon each other.

Tier 1: The Dispatcher Agent (The "Team Lead")

This is the new "brain" of the agentic system. It is a long-running, stateful agent that orchestrates the entire workflow. When a user starts an agentic session (e.g., gemini --agent "my goal..."), the Dispatcher takes over.

Its primary responsibilities are:

1. Task Analysis & Planning: It analyzes the user's goal and breaks it down into a sequence of concrete, logical steps.
2. Agent Selection: For each step, it queries the AgentRegistry (Tier 4) to select the best AgentDefinition for the job.
3. Orchestration: It invokes the selected Specialist Agent (Tier 2) using the AgentExecutor (Tier 3), sequentially.
4. State Management: It maintains a "Shared Context" or "Scratchpad." It takes the structured output from one agent's run (e.g., the JSON report from CodebaseInvestigatorAgent) and feeds it as input to the next agent (e.g., CodeGenerationAgent).
5. User Reporting: It provides continuous, real-time feedback to the user (e.g., "Running Codebase Investigator...", "Generating Code...") and presents the final, synthesized result.

Tier 2: Specialist Agents (The "Team Members")

These are the "workhorses" of the system. A Specialist Agent is simply an AgentDefinition that is non-interactive, specialized, and designed to accomplish one task perfectly.

• Existing Example: The CodebaseInvestigatorAgent is the perfect prototype for this tier.
• Interface: Each agent receives its task via AgentInputs and must return its findings via the complete_task tool with a structured OutputObject.
• Tools: Each agent definition specifies the exact tools it needs from the ToolRegistry (Tier 4).

Tier 3: The Agent Executor (The "Engine")

This component already exists. The AgentExecutor is the runtime environment that knows how to take an AgentDefinition, provide it with inputs and tools, and run its execution loop until it terminates.

The Dispatcher (Tier 1) simply calls AgentExecutor.create(...).run(...) for each sub-task it needs to delegate. No major changes are needed here; it's the stable foundation we build upon.

Tier 4: The Central Agent & Tool Registry

This is a new, central service responsible for managing all available agents and tools.

• ToolRegistry: This (mostly existing) registry holds all available tools (e.g., ls, readFile, grep, web_search). An AgentDefinition's toolConfig grants it a subset of these tools.
• AgentRegistry: This new registry holds all available AgentDefinitions. The Dispatcher queries this registry to find agents by name or description (e.g., "find agent for code generation"). This makes the system extensible, allowing new agents to be added easily.

---

Proposed List of Core Specialist Agents

To make this system functional, we would complement the existing codebase_investigator with a few other core agents:

1. CodeGenerationAgent

   • Description: A specialist agent that writes new code or modifies existing files based on a detailed plan.
   • Inputs: objective (the high-level goal), relevant_files (a list of files from the investigator), instructions (a detailed plan from the Dispatcher).
   • Tools: readFile, writeFile, ls.
   • Output: A JSON report of files created or modified.

2. CodeRefactorAgent

   • Description: A specialized version of the generator that focuses on modifying existing code. It is given a specific task (e.g., "rename this function," "optimize this loop") and a set of target files.
   • Inputs: objective (e.g., "Rename function foo to bar"), target_files (list of files to apply changes to).
   • Tools: readFile, writeFile.
   • Output: A diff or a report of changes made.

3. TestGenerationAgent

   • Description: An agent specialized in writing unit or integration tests for a given set of source files.
   • Inputs: source_files (list of files to write tests for), test_framework_preference (e.g., "jest", "vitest").
   • Tools: readFile, writeFile, ls, grep (to find existing test files).
   • Output: A list of new test files created.

4. ShellExecutionAgent

   • Description: A secure agent that runs shell commands (like linters, builds, or test runners) and reports the stdout, stderr, and exitCode.
   • Inputs: command (the shell command to run).
   • Tools: run_shell_command.
   • Output: A structured object: { stdout: string, stderr: string, exitCode: number }.

5. DocumentationAgent

   • Description: An agent that reads code files and writes or updates documentation (e.g., README.md, or code comments).
   • Inputs: source_files (list of files to document), documentation_style (e.g., "JSDoc", "README").
   • Tools: readFile, writeFile.
   • Output: A list of documentation files created or updated.

---

Example Workflow: "Add a new endpoint"

This flow demonstrates how all the tiers work together.

1. User Prompt: gemini --agent "Add a POST /api/users endpoint that creates a new user, and add a test for it."

2. Tier 1: Dispatcher Agent

   • Plan: "OK, I need to add a new endpoint.
     1. First, I need to understand the codebase structure.
     2. Second, I need to write the new endpoint code.
     3. Third, I need to write a test for the new code.
     4. Fourth, I should run the tests to make sure I didn't break anything."
   • User Feedback: > Planning complete. Starting investigation...

3. Step 1: Investigation

   • Dispatcher: Selects CodebaseInvestigatorAgent.
   • Tier 3 (Executor): Runs the agent.
   • Tier 2 (Specialist): CodebaseInvestigatorAgent runs. It uses ls and grep to find relevant files. It calls complete_task with its report:

     {
       "SummaryOfFindings": "The main router is in `src/routes/index.ts`. User models are in `src/models/user.ts`...",
       "RelevantLocations": [{"FilePath": "src/routes/index.ts"}, {"FilePath": "src/models/user.ts"}]
     }

4. Step 2: Code Generation

   • Dispatcher: Receives the report. Selects CodeGenerationAgent. Passes it the original objective and the investigator's report.
   • User Feedback: > Investigation complete. Generating code for new endpoint...
   • Tier 3 (Executor): Runs the agent.
   • Tier 2 (Specialist): CodeGenerationAgent runs. It reads src/routes/index.ts, adds the new POST /api/users endpoint, and calls complete_task.

5. Step 3: Test Generation

   • Dispatcher: Receives the code gen report. Selects TestGenerationAgent.
   • User Feedback: > Code generation complete. Writing new tests...
   • Tier 3 (Executor): Runs the agent.
   • Tier 2 (Specialist): TestGenerationAgent runs. It finds src/tests/api.test.ts, adds a new test block for the POST /api/users endpoint, and calls complete_task.

6. Step 4: Verification

   • Dispatcher: Receives the test gen report. Selects ShellExecutionAgent.
   • User Feedback: > Test generation complete. Running 'npm test'...
   • Tier 3 (Executor): Runs the agent.
   • Tier 2 (Specialist): ShellExecutionAgent runs npm test and reports the output.

7. Tier 1: Final Report

   • Dispatcher: All steps are complete. It synthesizes the final report for the user.
   • User Feedback:

     > Task complete.
     - Modified `src/routes/index.ts` to add the new endpoint.
     - Modified `src/tests/api.test.ts` to add a new test.
     - All tests passed.
     

[NAME0x0]

Technical Proposal: A Dispatcher-Based Agent Architecture

This proposal outlines a concrete implementation for the 4-tier architecture. It is designed to be layered, testable, and extensible, leveraging the existing AgentExecutor as the core runtime for all specialist tasks.

---

Tier 1: The OrchestrationService (The "Dispatcher")

This is the new top-level service that manages the end-to-end agentic workflow. It would likely live in packages/a2a-server/ or a new high-level packages/core/src/orchestration/ directory. It is not an agent itself; it is the director of agents.

Suggested Implementation:

// In a new file, e.g., packages/core/src/orchestration/OrchestrationService.ts

import { AgentExecutor } from '../agents/executor';
import { AgentDefinition, AgentInputs, OutputObject } from '../agents/types';
import { AgentRegistry } from './AgentRegistry'; // Tier 4
import { TaskContext } from './TaskContext'; // New state object
import { GeminiChat } from '../core/geminiChat';
import { Config } from '../config/config';
import { SubagentActivityEvent } from '../agents/types';

export class OrchestrationService {
  private agentRegistry: AgentRegistry;
  private runtimeContext: Config;
  private planningChat: GeminiChat;

  constructor(runtimeContext: Config, agentRegistry: AgentRegistry) {
    this.runtimeContext = runtimeContext;
    this.agentRegistry = agentRegistry;
    // This chat is used *only* for planning
    this.planningChat = new GeminiChat(runtimeContext, { /* ... */ });
  }

  /**
   * The main entry point for an agentic task.
   * @param userGoal The high-level goal from the user.
   * @param onActivity A callback to stream all sub-agent activity to the UI.
   */
  public async runTask(
    userGoal: string,
    onActivity: (activity: SubagentActivityEvent) => void,
  ): Promise<string> {

    // 1. Create a persistent state object for this task
    const taskContext = new TaskContext(userGoal);

    // 2. Step 1: Planning
    // The planner's job is to create a structured list of steps.
    const plan = await this.generatePlan(taskContext);
    taskContext.setPlan(plan);

    // 3. Step 2: Sequential Execution
    for (const step of plan) {
      if (taskContext.isCancelled()) break;

      // 3a. Find the right agent from the registry
      const agentDef = this.agentRegistry.getAgent(step.agentName);
      if (!agentDef) {
        throw new Error(`No agent found with name: ${step.agentName}`);
      }

      // 3b. Prepare inputs for this agent
      // This maps data from the shared context to the agent's expected inputs
      const agentInputs = this.prepareAgentInputs(agentDef, taskContext, step);

      // 3c. Create and run the specialist executor
      const executor = await AgentExecutor.create(
        agentDef,
        this.runtimeContext,
        // *** This is the key UI link ***
        // Pass the main onActivity callback down to the sub-agent
        onActivity,
      );

      const output: OutputObject = await executor.run(
        agentInputs,
        taskContext.abortSignal,
      );

      // 3d. Update shared context with the results
      taskContext.update(output);
      if (output.terminate_reason !== 'GOAL') {
        // Handle sub-agent failure
        throw new Error(`Agent ${agentDef.name} failed: ${output.result}`);
      }
    }

    // 4. Step 3: Final Synthesis
    // (Optional) A final call to synthesize all results into a
    // friendly report for the user.
    return this.synthesizeFinalReport(taskContext);
  }

  private async generatePlan(context: TaskContext): Promise<TaskStep[]> {
    // ... logic to call this.planningChat
    // The prompt instructs the model to act as a project manager
    // and return a JSON array of steps, like:
    // [
    //   { "step": 1, "agentName": "codebase_investigator", "task": "Find relevant files for..." },
    //   { "step": 2, "agentName": "code_generation", "task": "Implement the new endpoint in files..." }
    // ]
    return []; // Placeholder
  }

  private prepareAgentInputs(
    agentDef: AgentDefinition<any>,
    context: TaskContext,
    step: TaskStep,
  ): AgentInputs {
    // This function is critical. It reads agentDef.inputConfig
    // and pulls the required data from the shared TaskContext.
    // e.g., if agentDef.inputConfig needs 'relevant_files',
    // it pulls `context.get('relevant_files')`.
    // It also adds the specific task:
    // return { ...mappedInputs, objective: step.task };
    return { objective: step.task }; // Placeholder
  }

  private async synthesizeFinalReport(context: TaskContext): Promise<string> {
    // ... final chat call to summarize the entire run
    return 'Task completed successfully.';
  }
}

---

Tier 2: Specialist Agents & Contracts

This tier is implemented simply by creating new AgentDefinition files, following the pattern of codebase-investigator.ts.

Key Technical Requirement: The inputConfig and outputConfig for each agent must be well-defined with Zod schemas. This is the API contract that the OrchestrationService uses to pass data between agents.

Example: CodeGenerationAgent Definition

// In a new file, e.g., packages/core/src/agents/specialists/codeGenerationAgent.ts

import { z } from 'zod';
import { AgentDefinition } from '../types';

// 1. Define the Zod schema for the *structured output*
const CodeGenerationReportSchema = z.object({
  files_modified: z.array(z.string()).describe('List of files modified.'),
  files_created: z.array(z.string()).describe('List of files newly created.'),
  summary: z.string().describe('Summary of changes made.'),
});

// 2. Define the Agent
export const CodeGenerationAgent: AgentDefinition<
  typeof CodeGenerationReportSchema
> = {
  name: 'code_generation',
  description: 'Writes or modifies code based on a specific objective.',
  
  // 3. Define the *inputs* this agent expects
  inputConfig: {
    inputs: {
      objective: {
        description: 'The specific coding task to perform.',
        type: 'string',
        required: true,
      },
      relevant_files: {
        description: 'A list of relevant files provided by the investigator.',
        type: 'string[]', // Corresponds to z.array(z.string())
        required: false,
      },
    },
  },

  // 4. Define the *output* this agent *must* provide
  outputConfig: {
    outputName: 'report',
    description: 'The structured report of code changes.',
    schema: CodeGenerationReportSchema,
  },

  // 5. This agent needs tools to write files
  toolConfig: {
    tools: ['readFile', 'writeFile', 'ls', 'glob'],
  },
  
  // ... (modelConfig, runConfig) ...
  
  promptConfig: {
    systemPrompt: `You are a specialist code generation agent. Your task is to execute the objective given.
      You MUST use the 'writeFile' tool to save your changes.
      When finished, you MUST call 'complete_task' with the 'report' object.`,
    query: `
      Objective: ${'${objective}'}
      Relevant Files: ${'${relevant_files}'}
      
      Begin execution.
    `,
  },
};

---

Tier 3: The AgentExecutor (Existing)

No new implementation is needed. The OrchestrationService (Tier 1) will call AgentExecutor.create() for each specialist agent it needs to run.

Key Technical Requirement: The onActivity callback must be passed from the OrchestrationService to the AgentExecutor.create method. This allows the main CLI UI to show a unified stream of events, even from sub-agents.

Example UI Feedback:

> [Dispatcher] Planning task...
> [Dispatcher] Plan: 1. Investigate, 2. Generate Code
> [Dispatcher] Starting agent 'codebase_investigator'...
> [codebase_investigator] (thought) I will start by using 'ls -R'.
> [codebase_investigator] (tool call) ls -R
> [Dispatcher] Agent 'codebase_investigator' finished.
> [Dispatcher] Starting agent 'code_generation'...
> [code_generation] (thought) I will modify 'src/routes/index.ts'.
> [code_generation] (tool call) writeFile(...)
> [Dispatcher] Agent 'code_generation' finished.
> [Dispatcher] Task complete.

---

Tier 4: The Registries

ToolRegistry (Existing)

This is already implemented in packages/core/src/tools/tool-registry.ts. The OrchestrationService would be initialized with the global ToolRegistry and use it to provision each AgentExecutor with its sandboxed set of tools.

AgentRegistry (New Class)

This new class is a simple service for mapping agent names to their definitions.

Suggested Implementation:

// In a new file, e.g., packages/core/src/orchestration/AgentRegistry.ts

import { AgentDefinition } from '../agents/types';

export class AgentRegistry {
  private agents = new Map<string, AgentDefinition<any>>();

  /**
   * Registers a new agent definition.
   */
  public register(agentDef: AgentDefinition<any>): void {
    if (this.agents.has(agentDef.name)) {
      console.warn(`Overwriting agent: ${agentDef.name}`);
    }
    this.agents.set(agentDef.name, agentDef);
  }

  /**
   * Retrieves an agent definition by its unique name.
   */
  public getAgent(name: string): AgentDefinition<any> | undefined {
    return this.agents.get(name);
  }

  /**
   * (For the planner) Gets a list of agents and their descriptions
   * so the planner model knows what its "team" can do.
   */
  public getAgentManifest(): { name: string; description: string }[] {
    return Array.from(this.agents.values()).map((agent) => ({
      name: agent.name,
      description: agent.description,
    }));
  }
}

// This registry would be initialized at startup:
//
// import { CodebaseInvestigatorAgent } from '../agents/codebase-investigator';
// import { CodeGenerationAgent } from '../agents/specialists/codeGenerationAgent';
//
// const agentRegistry = new AgentRegistry();
// agentRegistry.register(CodebaseInvestigatorAgent);
// agentRegistry.register(CodeGenerationAgent);
//
// const orchestrator = new OrchestrationService(config, agentRegistry);

---

Further Technical Considerations for the Agentic Framework

1. Advanced State Management (The TaskContext Object)

The TaskContext I mentioned earlier is the "blood" of this system. It needs to be more than a simple Map.

• Recommendation: Define it as a class that serializes cleanly. It should be passed to the OrchestrationService and updated by each agent's execution.
• Proposed Structure:

  // In a new file, e.g., packages/core/src/orchestration/TaskContext.ts
  export class TaskContext {
    public userGoal: string;
    public plan: TaskStep[];
    public history: Array<{ agentName: string; input: AgentInputs; output: OutputObject }>;
    public sharedScratchpad: Record<string, any>; // For agents to pass data
    public abortController: AbortController;
  
    constructor(userGoal: string) {
      // ... init
    }
  
    // Method to get data for the next agent's inputs
    public get(key: string): any {
      return this.sharedScratchpad[key];
    }
  
    // Method to update the context after an agent run
    public update(agentName: string, input: AgentInputs, output: OutputObject) {
      this.history.push({ agentName, input, output });
  
      // Example: If the agent's output was a JSON report,
      // merge it into the shared scratchpad for the next agent.
      if (output.terminate_reason === 'GOAL' && output.result) {
        try {
          const resultData = JSON.parse(output.result);
          // This is how the CodeGenerator gets the Investigator's data
          Object.assign(this.sharedScratchpad, resultData);
        } catch (e) {
          // It was just a string, add it as 'last_result'
          this.sharedScratchpad['last_result'] = output.result;
        }
      }
    }
  }

2. Orchestrator-Level Error Handling & Self-Correction

This is the most critical part of making the system "agentic." What happens when Step 3 (Test Generation) fails? The orchestrator shouldn't just give up.

• Recommendation: The OrchestrationService (Tier 1) must treat sub-agent failures as new information to be fed back into its planning model. This creates a self-correction loop.

• Proposed Logic (in OrchestrationService.runTask):

  // Inside the execution loop...
  try {
    const output: OutputObject = await executor.run(...);
    taskContext.update(agentDef.name, agentInputs, output);
  
    if (output.terminate_reason !== 'GOAL') {
      // *** SELF-CORRECTION HOOK ***
      // The sub-agent failed.
      // 1. Create a new "recovery task" for the planner.
      const recoveryGoal = `
        The previous step failed.
        Agent: ${agentDef.name}
        Task: ${step.task}
        Error: ${output.result}
        
        My original plan was: ${JSON.stringify(taskContext.plan)}
        My current context is: ${JSON.stringify(taskContext.sharedScratchpad)}
  
        Please generate a new, corrected plan to recover from this error and
        still achieve the original goal: ${taskContext.userGoal}
      `;
      
      // 2. Get a new plan
      const newPlan = await this.generatePlan(recoveryGoal); // Pass the error
      taskContext.setPlan(newPlan); // Overwrite the old plan
      
      // 3. Continue the loop with the new, corrected plan
      continue;
    }
  
  } catch (error) {
    // Handle executor-level (e.g., timeout) failures
    // ... similar self-correction logic ...
  }

3. Granular Security via the Policy Engine

You cannot give every agent the same permissions. The CodebaseInvestigatorAgent should never be allowed to write files.

• Recommendation: The OrchestrationService should create a new, specific Config object for each specialist agent it's about to run, loading a different policy for each. This leverages the existing policy-engine.ts.

• Example (in OrchestrationService.runTask):

  // Inside the execution loop, before creating the executor...
  
  // 1. Get the agent definition
  const agentDef = this.agentRegistry.getAgent(step.agentName);
  
  // 2. Select the correct policy for this *specific agent*
  let policyName = 'read-only'; // Default
  if (agentDef.name === 'code_generation' || agentDef.name === 'code_refactor') {
    policyName = 'write'; // From write.toml
  } else if (agentDef.name === 'shell_execution') {
    policyName = 'yolo'; // Or a new 'shell-only' policy
  }
  
  // 3. Create a new, temporary config for this agent's run
  const agentConfig = await this.runtimeContext.loadPolicy(policyName);
  
  // 4. Create the executor with the sandboxed config
  const executor = await AgentExecutor.create(
    agentDef,
    agentConfig, // Pass the sandboxed config
    onActivity,
  );
  
  const output = await executor.run(...);

4. Handling User Interaction (The "Confirmation" Step)

My proposal was non-interactive, but the best systems allow for user confirmation. The existing AgentExecutor is non-interactive by design, which is correct. The Dispatcher is what should handle interaction.

• Recommendation: The OrchestrationService's plan should include steps that require confirmation. It can use the onActivity callback to send a special event to the UI and await a response.
• Example:
  1. The generatePlan method produces a step:
     { "step": 3, "agentName": "shell_execution", "task": "npm install", "requires_confirmation": true }
  2. The OrchestrationService sees requires_confirmation.
  3. It emits an event: onActivity({ type: 'CONFIRMATION_REQUIRED', data: { task: 'npm install' } }).
  4. The CLI's UI (in gemini.tsx) sees this event and renders a [Y/n] prompt.
  5. The OrchestrationService awaits a response from the UI before proceeding.

5. CLI Entrypoint (The agent Command)

This entire workflow needs a new "mode" to be triggered from the CLI.

• Recommendation: Add a new command to packages/cli/src/nonInteractiveCliCommands.ts. This command, e.g., gemini agent "<prompt>", would be the entrypoint.
• Implementation:
  1. This command would not start the main App.tsx REPL.
  2. Instead, it would initialize the AgentRegistry, ToolRegistry, and OrchestrationService.
  3. It would call orchestrationService.runTask(prompt, ...) directly.
  4. It would need a new, simple renderer (or reuse nonInteractiveUi.ts) that just listens to the onActivity stream and prints the status updates (e.g., [codebase_investigator] (thought) ...) directly to stdout.

These additions would make the proposed architecture secure, resilient, interactive, and clearly wired into the existing CLI application.

[verdverm]

Check out the Google ADK project, it's got some solid building blocks for agents and is already available in Python, Go, and Java