Can I attach Query engine to a agent and call with other agents in 'OpenAIAgent.from_tools' function?

[mraguth]

I have developed a group of agents capable of linking our databases, and retrieving and preparing data for analysis. Currently, I need to channel all my input through the 'query engine'. The main query is whether it is feasible to transform the query engine into an agent that can be called like the other agents. I have included sample agent code along with my query engine code for reference.

def multiply(a: int, b: int) -> int:
    """Multiple two integers and returns the result integer"""
    return a * b

multiply_tool = FunctionTool.from_defaults(fn=multiply)

query_engine = RAGStringQueryEngine(
    retriever=recursive_retriever,
    response_synthesizer=response_synthesizer,
    pandas_prompt= pandas_prompt,
    pandas_output_parser= pandas_output_parser,
    llm=llm,
    qa_prompt=qa_prompt,
    synthesize_response=True,
    response_synthesis_prompt=response_synthesis_prompt,
)

[dosubot[bot]]

Yes, it is possible to attach the query engine to an agent and call it with other agents using the OpenAIAgent.from_tools function. You can achieve this by wrapping your RAGStringQueryEngine in a QueryEngineTool and then using the OpenAIAgent.from_tools function to create an agent that includes this tool.

Here is how you can do it:

1. Wrap the Query Engine in a Tool: Use the QueryEngineTool class to wrap your RAGStringQueryEngine.

2. Create the Agent: Use the OpenAIAgent.from_tools function to create an agent that includes your query engine tool along with any other tools you have.

Here is the relevant code to achieve this:

from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.agent.openai import OpenAIAgent

# Assuming your query engine is already defined as:
query_engine = RAGStringQueryEngine(
    retriever=recursive_retriever,
    response_synthesizer=response_synthesizer,
    pandas_prompt=pandas_prompt,
    pandas_output_parser=pandas_output_parser,
    llm=llm,
    qa_prompt=qa_prompt,
    synthesize_response=True,
    response_synthesis_prompt=response_synthesis_prompt,
)

# Create a QueryEngineTool from your query engine
query_engine_tool = QueryEngineTool.from_defaults(
    query_engine=query_engine,
    name="query_engine_tool",
    description="Tool for querying the RAGStringQueryEngine."
)

# Define other tools if you have any
multiply_tool = FunctionTool.from_defaults(
    fn=multiply,
    name="multiply_tool",
    description="Multiply two integers and return the result integer."
)

# Create the agent with the tools
agent = OpenAIAgent.from_tools(
    tools=[query_engine_tool, multiply_tool],
    verbose=True
)

# Start the agent's chat REPL or use it programmatically
agent.chat_repl()

This setup allows you to channel all your input through the query engine and call it like any other agent using the OpenAIAgent.from_tools function.

Details

Sources

• docs/docs/examples/agent/openai_agent_with_query_engine.ipynb
• llama-index-core/llama_index/core/tools/query_engine.py
• llama-index-legacy/llama_index/legacy/tools/query_engine.py
• llama-index-legacy/llama_index/legacy/agent/openai/base.py
• llama-index-integrations/agent/llama-index-agent-openai-legacy/llama_index/agent/openai_legacy/context_retriever_agent.py
• llama-index-cli/llama_index/cli/rag/base.py
• llama-index-packs/llama-index-packs-self-rag/llama_index/packs/self_rag/base.py
• llama-index-legacy/llama_index/legacy/command_line/rag.py
• llama-index-networks/examples/demo/contributor-2/contributor_2/app_query_engine.py
• llama-index-core/llama_index/core/response_synthesizers/factory.py
• llama-index-legacy/llama_index/legacy/response_synthesizers/factory.py
• llama-index-core/llama_index/core/base/base_query_engine.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.