Feature Request: Intelligent Agent Routing in Agent Chains

[ogipogi]

Updated - 19.07.2025

Updated Feature Request: Agent Modularization with Tool-Based Architecture

Current Situation

We currently have a monolithic approach with one large prompt containing many endpoints (code search, web search, dev docs, confluence, etc.), making it difficult to maintain and extend.

Problem Statement

• Large prompts become unwieldy and error-prone
• Adding new capabilities increases complexity exponentially
• Difficult to maintain specialized knowledge in separate domains

Proposed Solution

Implement a modular agent architecture where:

1. Main Agent: Acts as a coordinator/router
2. Specialized Agents: Function as tools with focused capabilities
3. Direct Access Pattern: Main agent can access specialized knowledge and endpoints directly

Key Requirements

• Main agent should analyze queries and determine which specialized capabilities to use
• Specialized knowledge/tools should remain modular and maintainable separately
• System should produce unified, coherent responses

Example Implementation

Our fork demonstrates one approach with:

• Query Router Agent as coordinator
• Specialized agents (Web Search, Code Search, Dev Docs) as tools

This tool-based architecture maintains modularity while providing a seamless user experience.

[bhagwad]

Agreed. The web search use-case is particularly relevant. Right now, I have to manually choose which model I want to query based on the query characteristics, which is annoying.

Ideally, I would create my agent choice workflow, and then use just one point for all queries. No more wondering if I should use Perplexity Sonar because it's a web search. The model should be able to detect that a web search is necessary and call it appropriately.

Another use case is when I want to rope in a more advanced model for some queries. Ideally, the main agent should be able to detect that this is a complicated question, and bring in a more powerful model for that query. If we can give the main agent its rules, that would automate all future choices.

[ogipogi]

Current Situation

We currently have a very large prompt with many endpoints (code search, web search, dev doc search, confluence search, etc.) as OpenAPI actions and tools in our main agent. Based on the system prompt and user prompt, the agent decides which endpoint to call.

Problem Statement

The current problem with this setup is:

• Every additional endpoint must be added to the system prompt, leading to a very large prompt that becomes error-prone and difficult to maintain.

Proposed Solution

We wanted to use the Agent Chain feature to split this up (modularize it). This means transitioning from one very large prompt with many endpoints to specialized LibreChat agents, each with specific prompts and endpoints.

Current Issue with Agent Chains

We discovered that the current implementation goes through all agents that are created in the chain, regardless of whether the call is relevant to the user prompt or not.

Desired Behavior

Current State: One large LibreChat agent that calls the correct endpoint based on the user query

Desired State: A LibreChat main agent that decides which specialized agent to call based on the user query

Example

Example Chain

• Web Search Agent
• Code Search Agent

Example Use Case User Query: "How will the weather be tomorrow?"

Expected Behavior:

• Main agent analyzes the query and determines it needs a web search
• Main agent recognizes that other agents (like code search) are not needed
• Only the web search agent is called

Current Behavior with Agent Chain:

• All agents in the chain are called, including irrelevant ones like code search

Feature Request

Implement intelligent agent routing in Agent Chains where:

1. A main/coordinator agent analyzes the user query
2. Based on the analysis, it selectively calls only the relevant specialized agents
3. Irrelevant agents in the chain are skipped entirely

This would enable true modularization while maintaining efficiency and reducing unnecessary API calls.

Hi @danny-avila

Could this be an option for you?

Here's an example of how this could look:

• Query Router Agent acts as the main coordinator
• Specialized agents as tools:
  • Web Search Agent
  • Code Search Agent
  • Developer Documentation Agent

demo.mp4

[danny-avila]

See this discussion: #8372

[ogipogi]

Oh nice, goes exactly in that direction 👍

[ogipogi]

Hi @danny-avila

We have now implemented intelligent agent chaining in our fork.

• The main agent decides which specialized agent(s) can best assist with a query
• It receives the chosen agent's prompt and available tools dynamically
• It calls the necessary specialized tools and uses their knowledge to generate comprehensive output

Key modular advantages:

• Prevents prompt overloading by distributing knowledge across specialized agents
• Allows each agent to maintain focused expertise without context pollution
• Makes it much easier to extend the system with new capabilities

First, we want to express our gratitude for LibreChat - the work done is truly amazing and has been an excellent foundation to build upon.

Looking forward to seeing your approach to implementing similar functionality in upstream.

demo-modular-agents.mp4

[danny-avila]

Looks cool, thanks for sharing. Official implementation will likely differ, as it will more closely align to what I described here:
#8372 (comment)

[rtamang21]

@ogipogi We were also thinking of implementing something like this, can you provide code change you did for above feature to use agent as tools?

[ogipogi]

Hi @rtamang21 , yes, I can - I will make the change public and let you know when it's available.

[mrgizmo212]

I too would be very interested in this

[danny-avila]

I'm prototyping multi-agent architecture here for LibreChat: danny-avila/agents#18

This is working really well so far, and is highly composable, allowing for almost any multi-agent pattern you can think of.

The challenge is how to develop a UI for this. Maybe just "handoffs" via simple form to start, to allow for "intelligent routing" but we can do much more like parallel execution, message transformation between agents, etc.

I'm thinking a simple node-based interface makes sense longer term.

[mrgizmo212]

Amazing! Where do you find the inspiration for such ideas :)

[ogipogi]

Cool @danny-avila 💪