Tool called multiple times and observation is not used by the agent #4890
-
Checked other resources
Commit to Help
Example Codeimport { createOpenAIFunctionsAgent, AgentExecutor } from 'langchain/agents';
import { ChatOpenAI } from '@langchain/openai';
import { DynamicStructuredTool } from '@langchain/core/tools';
import { z } from 'zod';
import {
SystemMessagePromptTemplate,
ChatPromptTemplate,
} from '@langchain/core/prompts';
const doLangChainAssistantRun = async ({
app,
userMessage,
}: {
app: Application;
userMessage: string;
}) => {
const findClients = new DynamicStructuredTool({
verbose: true,
name: 'findClients',
description:
'Find a client by name. Returns a JSON string of the client object.',
schema: z.object({
name: z
.string({
description:
'The name of the client. This could be their first name, last name, both, or something else. For example, this name could be a company name, or a couple (e.g. "Mr. and Mrs. Doe").',
})
.optional(),
}),
func: async ({ name }) => {
/* code to find clint from DB */
return `here is the client JSON data: ${JSON.stringify(
clients.data[0],
)}`;
},
});
const message = SystemMessagePromptTemplate.fromTemplate(
'{agent_scratchpad}\n{systemMessage}',
);
const agentPrompt = ChatPromptTemplate.fromMessages([
message,
['user', '{userInput}'],
]);
const agentModel = new ChatOpenAI({
modelName: 'gpt-4',
temperature: 0,
openAIApiKey: app.get('llmJobsConfig').openApiKey,
});
const tools = [findClients];
const agent = await createOpenAIFunctionsAgent({
llm: agentModel,
tools,
prompt: agentPrompt,
});
const agentExecutor = new AgentExecutor({
agent,
tools,
maxIterations: 4,
returnIntermediateSteps: true,
verbose: true,
});
const defaultSystemMessage = `You are a virtual assistant for a construction and renovation contracting company. A contractor will ask you to complete a task. You should return the result to the contractor.`;
const agentResult = await agentExecutor.invoke(
{
userInput: userMessage,
systemMessage: defaultSystemMessage,
},
);
console.log('result:', JSON.stringify(agentResult, null, 2));
};
await doLangChainAssistantRun({
app,
userMessage: "Find the client AI Guy",
});DescriptionI am trying to use an agent with tool calling. The agent calls the tool correctly and the observation has the correct data. However, the agent continues to call this same tool and never uses it for the final response. This continues until the max iterations is hit. This seems like a related issue in the python version Result of the run: System Infomac |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Based on the information you've provided and the context from similar issues in the LangChain JS repository, it seems like the agent is not correctly incorporating the observations from the tool calls into its decision-making process. This could be causing the agent to repeatedly call the same tool without using the observation for the final response. From the Agent stuck in a loop because agent_scratchpad is not serialized properly when using createOpenAIToolsAgent issue, it was suggested to ensure that any object that needs to be converted to a string for logging, concatenation, or any other purpose is properly serialized using From the Generative agents don't seem to actually add the conversation to memory as it's happening issue, it was suggested to update the conversation with each new contribution. This issue was closed by a pull request: fix dialogue to update conversation with each new contribution. In your case, you might want to check how your agent's planning logic is implemented, particularly in how it handles the observations and decides on actions based on them. Ensure that the Also, check the stream handling logic for agents that use streaming, the method I hope this helps! If you have any more questions or need further clarification, feel free to ask. Sources
|
Beta Was this translation helpful? Give feedback.
-
|
Updated the prompt to be created as follows and its working now. const agentPrompt = ChatPromptTemplate.fromMessages([
new SystemMessage('{systemMessage}'),
new HumanMessage('{userInput}'),
new MessagesPlaceholder('agent_scratchpad'),
]); |
Beta Was this translation helpful? Give feedback.
Based on the information you've provided and the context from similar issues in the LangChain JS repository, it seems like the agent is not correctly incorporating the observations from the tool calls into its decision-making process. This could be causing the agent to repeatedly call the same tool without using the observation for the final response.
From the Agent stuck in a loop because agent_scratchpad is not serialized properly when using createOpenAIToolsAgent issue, it was suggested to ensure that any object that needs to be converted to a string for logging, concatenation, or any other purpose is properly serialized using
JSON.stringifyor handled in a way that correctly represen…