Tool is calling multiple times for a single query using Agent

[rups20097]

I am calling the agent to get the current date or news, and it's calling the "date" tool or "news" or "calculator" tool several times like > 10.

llm = ChatOpenAI(temperature=1, model="gpt-4-1106-preview", openai_api_key=os.getenv('OPENAI_API_KEY'))
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)
memory = ConversationBufferMemory(memory_key="history", return_messages=True, max_token_limit=50)


def get_weather(self):
    return get_realtime_data('weather', "")


def get_news(self):
    return get_realtime_data('news', "")


def get_date(self):
    return get_realtime_data('date')


tools = [
    Tool(
        name="Weather",
        func=get_weather,
        description="useful for when you need to answer weather report, sunrise or sunset time, windspeed, weather forcast or any other weather forcast related information."
    ),
    Tool(
        name="Calculator",
        func=llm_math_chain.invoke,
        description="useful for when you need to answer questions about math"
    ),
    Tool(
        name="News",
        func=get_news,
        description="useful for when you need to answer about latest news headlines."
    ),
    Tool(
        name="Date",
        func=get_date,
        description="useful for getting the today's date for user query or any other execution"
    )
]


def agent_chat(message):
    # while True:

    memory.load_memory_variables({})
    template_model = f"{training_prompt}" + """
    {agent_scratchpad}
    Do not call or invoke same tool again and again.
    
    Current conversation with Rupam and you:
    {history}
    
    Rupam: {input}
    Friday: """
    prompt = PromptTemplate(input_variables=["history", "input", "agent_scratchpad"], template=template_model)
    # print(prompt)

    agent = create_openai_functions_agent(llm, tools, prompt)
    openai_agent = AgentExecutor(
        tools=tools,
        agent=agent,
        verbose=True,
        memory=memory,
        early_stopping="force"
    )

    result = openai_agent.invoke({"input": message})
    # result = openai_agent.invoke({"input": input("Ask : ")})
    print(result)
    # return result['output']


agent_chat("What is the current date?")

as of now I hard coded the return date string like as below in get_realtime_data, but even after this, it is getting called.

def get_realtime_data(data_category, query=""):
    def get_realtime_data(data_category, query=""):
    if data_category == 'date':
        # talk("Checking date and time")
        # datetime_json = json.loads(datetime.datetime.now().strftime("{\"date\": \"%Y-%m-%d\", \"time\": \"%H:%M\"}"))
        # # time.sleep(2)
        # return datetime_json
        return "24th Feb 2024"
        # return datetime.datetime.now()

    elif data_category == 'news':
        api_url = (f"https://newsapi.org/v2/top-headlines?country=in&pageSize=3&page=1&apiKey={os.getenv('NEWSAPI_APIKEY')}")
        response = requests.get(f"{api_url}")
        if response.status_code == 200:
            return response.json()
        else:
            return "weather data is not available right now"

In the debug console it is as below

> Entering new AgentExecutor chain...

Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`


24th Feb 2024
Invoking: `Date` with `today`

How to resolve the issue? I am not sure this is due to any kind of parsing issue or else.

[dosubot[bot]]

🤖

It seems like the issue you're experiencing is related to the tool being invoked multiple times for a single query. To better assist you, I need a bit more information:

1. Could you please provide the error message or log output you're seeing when the tool is invoked multiple times?
2. Are you experiencing this issue with all the tools or just specific ones?
3. Could you provide more details about the get_realtime_data function? Specifically, how it's being used in the context of the Tool class and how it's interacting with the invoke method of the AgentExecutor class?

This additional information will help me understand the problem more clearly and provide a more accurate solution.

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[rups20097]

Every details been asked above is mentioned in my query

[jongelderloos]

@rups20097 I had a similar issue with langchainjs. I think you need to create you prompt from messages as the tool needs to inject its response into the prompt as a message.

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are world class technical documentation writer."),
    ("user", "{input}")
])

Related issues in js version:
langchain-ai/langchainjs#4876
langchain-ai/langchainjs#4890

[wl102]

i got the similar problem