ChatBedrock calls incorrect tool in a multi-agent system #73
Description
When using ChatBedrock in the context of a collaborative multi-agent system and a proposed fix to the tool_calls attribute being set, the stated node and expected tool call do not align. The entry node is apparently honored but the attempted function call is incorrect. As seen in the output below, the "Researcher" node is apparently calling the "python_repl" tool which should not be the case, "tavily_tool" is the expected function name.
The fact this is the case for two different ways of attempting to handle tool calling leads me to believe there is a separate issue here.
https://github.com/bigbernnn/langchain-aws/pull/1
#50
[0:tasks] Starting step 0 with 1 task:
- __start__ -> {'messages': [HumanMessage(content="Fetch the UK's GDP over the past 5 years, then draw a line graph of it. Once you code it up, finish.")]}
[0:writes] Finished step 0 with writes to 1 channel:
- messages -> [HumanMessage(content="Fetch the UK's GDP over the past 5 years, then draw a line graph of it. Once you code it up, finish.")]
[1:tasks] Starting step 1 with 1 task:
- Researcher -> {'messages': [HumanMessage(content="Fetch the UK's GDP over the past 5 years, then draw a line graph of it. Once you code it up, finish.")],
'sender': None}
{'Researcher': {'messages': [AIMessage(content='Okay, let\'s get started on fetching the UK\'s GDP data for the past 5 years and drawing a line graph.\n\n<function_calls>\n<invoke>\n<tool_name>python_repl</tool_name>\n<parameters>\n<code>\nimport requests\nimport json\nfrom datetime import datetime, timedelta\n\n# Function to fetch GDP data for the UK from the past 5 years\ndef fetch_uk_gdp_data():\n url = "https://ec.europa.eu/eurostat/wdds/rest/data/v2.1/json/en/naida_10_gdp"\n params = {\n "na_item": "B1GQ",\n "s_adj": "SNA_SAM",\n "precision": "1",\n "unit": "CP_MEUR",\n "geo": "UK",\n "startPeriod": (datetime.now() - timedelta(days=365*5)).strftime("%Y"),\n "endPeriod": datetime.now().year\n }\n response = requests.get(url, params=params)\n data = json.loads(response.text)\n \n gdp_data = []\n for value in data["value"]["orderedValue"]:\n year = int(value["period"])\n gdp = float(value["value"])\n gdp_data.append((year, gdp))\n \n return gdp_data\n\n# Fetch and print the GDP data\nuk_gdp_data = fetch_uk_gdp_data()\nprint("UK GDP data for the past 5 years:")\nfor year, gdp in uk_gdp_data:\n print(f"{year}: {gdp:.2f} billion Euros")\n</code>\n</parameters>\n</invoke>\n</function_calls>\n\nThe code above fetches the UK\'s GDP data for the past 5 years from the Eurostat API and prints it out.\n\nTo draw a line graph, we can use a plotting library like matplotlib:\n\n<function_calls>\n<invoke>\n<tool_name>python_repl</tool_name>\n<parameters>\n<code>\nimport matplotlib.pyplot as plt\n\n# Fetch GDP data if not already fetched\nif \'uk_gdp_data\' not in locals():\n uk_gdp_data = fetch_uk_gdp_data()\n\n# Extract years and GDP values from data\nyears, gdp_values = zip(*uk_gdp_data)\n\n# Create line plot\nplt.figure(figsize=(8, 6))\nplt.plot(years, gdp_values)\nplt.title("UK GDP for the Past 5 Years")\nplt.xlabel("Year")\nplt.ylabel("GDP (billion Euros)")\nplt.xticks(years)\nplt.show()\n</code>\n</parameters>\n</invoke>\n</function_calls>\n\nThis code will generate a line graph showing the UK\'s GDP over the past 5 years, with the years on the x-axis and the GDP values on the y-axis.\n\nFINAL ANSWER: The code above fetches the UK\'s GDP data for the past 5 years from the Eurostat API, prints out the data, and generates a line graph visualizing the GDP values over time using matplotlib.', additional_kwargs={'usage': {'prompt_tokens': 381, 'completion_tokens': 775, 'total_tokens': 1156}, 'stop_reason': 'tool_use', 'tool_calls': [{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}], 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, response_metadata={'usage': {'prompt_tokens': 381, 'completion_tokens': 775, 'total_tokens': 1156}, 'stop_reason': 'tool_use', 'tool_calls': [{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}], 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, name='Researcher', id='run-6b95de2b-a61e-4bfe-a5f1-7bc11405fe02-0', tool_calls=[{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}])], 'sender': 'Researcher'}}
----
[1:writes] Finished step 1 with writes to 2 channels:
- messages -> [AIMessage(content='Okay, let\'s get started on fetching the UK\'s GDP data for the past 5 years and drawing a line graph.\n\n<function_calls>\n<invoke>\n<tool_name>python_repl</tool_name>\n<parameters>\n<code>\nimport requests\nimport json\nfrom datetime import datetime, timedelta\n\n# Function to fetch GDP data for the UK from the past 5 years\ndef fetch_uk_gdp_data():\n url = "https://ec.europa.eu/eurostat/wdds/rest/data/v2.1/json/en/naida_10_gdp"\n params = {\n "na_item": "B1GQ",\n "s_adj": "SNA_SAM",\n "precision": "1",\n "unit": "CP_MEUR",\n "geo": "UK",\n "startPeriod": (datetime.now() - timedelta(days=365*5)).strftime("%Y"),\n "endPeriod": datetime.now().year\n }\n response = requests.get(url, params=params)\n data = json.loads(response.text)\n \n gdp_data = []\n for value in data["value"]["orderedValue"]:\n year = int(value["period"])\n gdp = float(value["value"])\n gdp_data.append((year, gdp))\n \n return gdp_data\n\n# Fetch and print the GDP data\nuk_gdp_data = fetch_uk_gdp_data()\nprint("UK GDP data for the past 5 years:")\nfor year, gdp in uk_gdp_data:\n print(f"{year}: {gdp:.2f} billion Euros")\n</code>\n</parameters>\n</invoke>\n</function_calls>\n\nThe code above fetches the UK\'s GDP data for the past 5 years from the Eurostat API and prints it out.\n\nTo draw a line graph, we can use a plotting library like matplotlib:\n\n<function_calls>\n<invoke>\n<tool_name>python_repl</tool_name>\n<parameters>\n<code>\nimport matplotlib.pyplot as plt\n\n# Fetch GDP data if not already fetched\nif \'uk_gdp_data\' not in locals():\n uk_gdp_data = fetch_uk_gdp_data()\n\n# Extract years and GDP values from data\nyears, gdp_values = zip(*uk_gdp_data)\n\n# Create line plot\nplt.figure(figsize=(8, 6))\nplt.plot(years, gdp_values)\nplt.title("UK GDP for the Past 5 Years")\nplt.xlabel("Year")\nplt.ylabel("GDP (billion Euros)")\nplt.xticks(years)\nplt.show()\n</code>\n</parameters>\n</invoke>\n</function_calls>\n\nThis code will generate a line graph showing the UK\'s GDP over the past 5 years, with the years on the x-axis and the GDP values on the y-axis.\n\nFINAL ANSWER: The code above fetches the UK\'s GDP data for the past 5 years from the Eurostat API, prints out the data, and generates a line graph visualizing the GDP values over time using matplotlib.', additional_kwargs={'usage': {'prompt_tokens': 381, 'completion_tokens': 775, 'total_tokens': 1156}, 'stop_reason': 'tool_use', 'tool_calls': [{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}], 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, response_metadata={'usage': {'prompt_tokens': 381, 'completion_tokens': 775, 'total_tokens': 1156}, 'stop_reason': 'tool_use', 'tool_calls': [{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}], 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, name='Researcher', id='run-6b95de2b-a61e-4bfe-a5f1-7bc11405fe02-0', tool_calls=[{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}])]
- sender -> 'Researcher'
[2:tasks] Starting step 2 with 1 task:
- call_tool -> {'messages': [HumanMessage(content="Fetch the UK's GDP over the past 5 years, then draw a line graph of it. Once you code it up, finish."),
AIMessage(content='Okay, let\'s get started on fetching the UK\'s GDP data for the past 5 years and drawing a line graph.\n\n<function_calls>\n<invoke>\n<tool_name>python_repl</tool_name>\n<parameters>\n<code>\nimport requests\nimport json\nfrom datetime import datetime, timedelta\n\n# Function to fetch GDP data for the UK from the past 5 years\ndef fetch_uk_gdp_data():\n url = "https://ec.europa.eu/eurostat/wdds/rest/data/v2.1/json/en/naida_10_gdp"\n params = {\n "na_item": "B1GQ",\n "s_adj": "SNA_SAM",\n "precision": "1",\n "unit": "CP_MEUR",\n "geo": "UK",\n "startPeriod": (datetime.now() - timedelta(days=365*5)).strftime("%Y"),\n "endPeriod": datetime.now().year\n }\n response = requests.get(url, params=params)\n data = json.loads(response.text)\n \n gdp_data = []\n for value in data["value"]["orderedValue"]:\n year = int(value["period"])\n gdp = float(value["value"])\n gdp_data.append((year, gdp))\n \n return gdp_data\n\n# Fetch and print the GDP data\nuk_gdp_data = fetch_uk_gdp_data()\nprint("UK GDP data for the past 5 years:")\nfor year, gdp in uk_gdp_data:\n print(f"{year}: {gdp:.2f} billion Euros")\n</code>\n</parameters>\n</invoke>\n</function_calls>\n\nThe code above fetches the UK\'s GDP data for the past 5 years from the Eurostat API and prints it out.\n\nTo draw a line graph, we can use a plotting library like matplotlib:\n\n<function_calls>\n<invoke>\n<tool_name>python_repl</tool_name>\n<parameters>\n<code>\nimport matplotlib.pyplot as plt\n\n# Fetch GDP data if not already fetched\nif \'uk_gdp_data\' not in locals():\n uk_gdp_data = fetch_uk_gdp_data()\n\n# Extract years and GDP values from data\nyears, gdp_values = zip(*uk_gdp_data)\n\n# Create line plot\nplt.figure(figsize=(8, 6))\nplt.plot(years, gdp_values)\nplt.title("UK GDP for the Past 5 Years")\nplt.xlabel("Year")\nplt.ylabel("GDP (billion Euros)")\nplt.xticks(years)\nplt.show()\n</code>\n</parameters>\n</invoke>\n</function_calls>\n\nThis code will generate a line graph showing the UK\'s GDP over the past 5 years, with the years on the x-axis and the GDP values on the y-axis.\n\nFINAL ANSWER: The code above fetches the UK\'s GDP data for the past 5 years from the Eurostat API, prints out the data, and generates a line graph visualizing the GDP values over time using matplotlib.', additional_kwargs={'usage': {'prompt_tokens': 381, 'completion_tokens': 775, 'total_tokens': 1156}, 'stop_reason': 'tool_use', 'tool_calls': [{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}], 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, response_metadata={'usage': {'prompt_tokens': 381, 'completion_tokens': 775, 'total_tokens': 1156}, 'stop_reason': 'tool_use', 'tool_calls': [{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}], 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, name='Researcher', id='run-6b95de2b-a61e-4bfe-a5f1-7bc11405fe02-0', tool_calls=[{'name': 'python_repl', 'args': {}, 'id': 'call_MEMWS5G7T1iOyTFHj5S6gg'}, {'name': 'python_repl', 'args': {}, 'id': 'call_jevlNPPIT_2oxzHFx8U_oA'}])],
'sender': 'Researcher'}
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[22], line 14
1 events = graph.stream(
2 {
3 "messages": [
(...)
12 {"recursion_limit": 150},
13 )
---> 14 for s in events:
15 print(s)
16 print("----")
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langgraph/pregel/__init__.py:963, in Pregel.stream(self, input, config, stream_mode, output_keys, input_keys, interrupt_before, interrupt_after, debug)
960 del fut, task
962 # panic on failure or timeout
--> 963 _panic_or_proceed(done, inflight, step)
964 # don't keep futures around in memory longer than needed
965 del done, inflight, futures
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langgraph/pregel/__init__.py:1489, in _panic_or_proceed(done, inflight, step)
1487 inflight.pop().cancel()
1488 # raise the exception
-> 1489 raise exc
1491 if inflight:
1492 # if we got here means we timed out
1493 while inflight:
1494 # cancel all pending tasks
File [/usr/lib/python3.10/concurrent/futures/thread.py:58](http://localhost:8888/usr/lib/python3.10/concurrent/futures/thread.py#line=57), in _WorkItem.run(self)
55 return
57 try:
---> 58 result = self.fn(*self.args, **self.kwargs)
59 except BaseException as exc:
60 self.future.set_exception(exc)
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langgraph/pregel/retry.py:66, in run_with_retry(task, retry_policy)
64 task.writes.clear()
65 # run the task
---> 66 task.proc.invoke(task.input, task.config)
67 # if successful, end
68 break
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langchain_core/runnables/base.py:2493, in RunnableSequence.invoke(self, input, config, **kwargs)
2489 config = patch_config(
2490 config, callbacks=run_manager.get_child(f"seq:step:{i+1}")
2491 )
2492 if i == 0:
-> 2493 input = step.invoke(input, config, **kwargs)
2494 else:
2495 input = step.invoke(input, config)
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langgraph/utils.py:95, in RunnableCallable.invoke(self, input, config, **kwargs)
93 if accepts_config(self.func):
94 kwargs["config"] = config
---> 95 ret = context.run(self.func, input, **kwargs)
96 if isinstance(ret, Runnable) and self.recurse:
97 return ret.invoke(input, config)
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langgraph/prebuilt/tool_node.py:68, in ToolNode._func(self, input, config)
63 return ToolMessage(
64 content=str_output(output), name=call["name"], tool_call_id=call["id"]
65 )
67 with get_executor_for_config(config) as executor:
---> 68 outputs = [*executor.map(run_one, message.tool_calls)]
69 if output_type == "list":
70 return outputs
File [/usr/lib/python3.10/concurrent/futures/_base.py:621](http://localhost:8888/usr/lib/python3.10/concurrent/futures/_base.py#line=620), in Executor.map.<locals>.result_iterator()
618 while fs:
619 # Careful not to keep a reference to the popped future
620 if timeout is None:
--> 621 yield _result_or_cancel(fs.pop())
622 else:
623 yield _result_or_cancel(fs.pop(), end_time - time.monotonic())
File [/usr/lib/python3.10/concurrent/futures/_base.py:319](http://localhost:8888/usr/lib/python3.10/concurrent/futures/_base.py#line=318), in _result_or_cancel(***failed resolving arguments***)
317 try:
318 try:
--> 319 return fut.result(timeout)
320 finally:
321 fut.cancel()
File [/usr/lib/python3.10/concurrent/futures/_base.py:451](http://localhost:8888/usr/lib/python3.10/concurrent/futures/_base.py#line=450), in Future.result(self, timeout)
449 raise CancelledError()
450 elif self._state == FINISHED:
--> 451 return self.__get_result()
453 self._condition.wait(timeout)
455 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:
File [/usr/lib/python3.10/concurrent/futures/_base.py:403](http://localhost:8888/usr/lib/python3.10/concurrent/futures/_base.py#line=402), in Future.__get_result(self)
401 if self._exception:
402 try:
--> 403 raise self._exception
404 finally:
405 # Break a reference cycle with the exception in self._exception
406 self = None
File [/usr/lib/python3.10/concurrent/futures/thread.py:58](http://localhost:8888/usr/lib/python3.10/concurrent/futures/thread.py#line=57), in _WorkItem.run(self)
55 return
57 try:
---> 58 result = self.fn(*self.args, **self.kwargs)
59 except BaseException as exc:
60 self.future.set_exception(exc)
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langchain_core/runnables/config.py:499, in ContextThreadPoolExecutor.map.<locals>._wrapped_fn(*args)
498 def _wrapped_fn(*args: Any) -> T:
--> 499 return contexts.pop().run(fn, *args)
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langgraph/prebuilt/tool_node.py:62, in ToolNode._func.<locals>.run_one(call)
61 def run_one(call: ToolCall):
---> 62 output = self.tools_by_name[call["name"]].invoke(call["args"], config)
63 return ToolMessage(
64 content=str_output(output), name=call["name"], tool_call_id=call["id"]
65 )
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langchain_core/tools.py:260, in BaseTool.invoke(self, input, config, **kwargs)
253 def invoke(
254 self,
255 input: Union[str, Dict],
256 config: Optional[RunnableConfig] = None,
257 **kwargs: Any,
258 ) -> Any:
259 config = ensure_config(config)
--> 260 return self.run(
261 input,
262 callbacks=config.get("callbacks"),
263 tags=config.get("tags"),
264 metadata=config.get("metadata"),
265 run_name=config.get("run_name"),
266 run_id=config.pop("run_id", None),
267 config=config,
268 **kwargs,
269 )
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langchain_core/tools.py:417, in BaseTool.run(self, tool_input, verbose, start_color, color, callbacks, tags, metadata, run_name, run_id, config, **kwargs)
415 except ValidationError as e:
416 if not self.handle_validation_error:
--> 417 raise e
418 elif isinstance(self.handle_validation_error, bool):
419 observation = "Tool input validation error"
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langchain_core/tools.py:406, in BaseTool.run(self, tool_input, verbose, start_color, color, callbacks, tags, metadata, run_name, run_id, config, **kwargs)
404 context = copy_context()
405 context.run(_set_config_context, child_config)
--> 406 parsed_input = self._parse_input(tool_input)
407 tool_args, tool_kwargs = self._to_args_and_kwargs(parsed_input)
408 observation = (
409 context.run(
410 self._run, *tool_args, run_manager=run_manager, **tool_kwargs
(...)
413 else context.run(self._run, *tool_args, **tool_kwargs)
414 )
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/langchain_core/tools.py:304, in BaseTool._parse_input(self, tool_input)
302 else:
303 if input_args is not None:
--> 304 result = input_args.parse_obj(tool_input)
305 return {
306 k: getattr(result, k)
307 for k, v in result.dict().items()
308 if k in tool_input
309 }
310 return tool_input
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/pydantic/v1/main.py:526, in BaseModel.parse_obj(cls, obj)
524 exc = TypeError(f'{cls.__name__} expected dict not {obj.__class__.__name__}')
525 raise ValidationError([ErrorWrapper(exc, loc=ROOT_KEY)], cls) from e
--> 526 return cls(**obj)
File ~/Documents/Projects/jupyter/.venv/lib/python3.10/site-packages/pydantic/v1/main.py:341, in BaseModel.__init__(__pydantic_self__, **data)
339 values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
340 if validation_error:
--> 341 raise validation_error
342 try:
343 object_setattr(__pydantic_self__, '__dict__', values)
ValidationError: 1 validation error for python_replSchema
code
field required (type=value_error.missing)