Skip to content

fix: prevent silent KeyError in ReActAgent after update_prompts()#20863

Open
roli-lpci wants to merge 2 commits intorun-llama:mainfrom
roli-lpci:fix/react-agent-formatter-keyerror
Open

fix: prevent silent KeyError in ReActAgent after update_prompts()#20863
roli-lpci wants to merge 2 commits intorun-llama:mainfrom
roli-lpci:fix/react-agent-formatter-keyerror

Conversation

@roli-lpci
Copy link

Summary

Fixes #20416ReActAgent silently stops working after updating the system prompt via update_prompts().

Root cause: The formatter at formatter.py:81 uses Python's str.format() on the stored system header, which crashes with KeyError when the string contains literal braces ({, }) from pre-formatted template escape sequences ({{, }}). The exception propagates through the workflow runtime and produces empty agent output with no traceback.

Fix (2 files, ~15 lines):

  • formatter.py: Replace str.format() with format_string() (the existing SafeFormatter utility) followed by brace escape processing. This safely substitutes {tool_desc} and {tool_names} without crashing on literal braces, while preserving the default template's JSON examples for the LLM.
  • react_agent.py: Add UserWarning when update_prompts() receives a prompt missing required {tool_desc} or {tool_names} placeholders, giving users immediate feedback instead of silent runtime failure.

Test plan

  • 6 new tests in test_prompt_customization.py (formatter with literal braces, default template JSON examples, pre-formatted template update, warning on missing placeholders, no false warnings)
  • 1 existing test_react_agent_prompts passes
  • 1 existing test_inheritance_react_chat_formatter passes
  • 14 existing test_react_output_parser tests pass
  • Zero regressions — default template output is identical before and after the fix

🤖 Generated with Claude Code

…re-formatted prompts

When users update a ReActAgent's system prompt via update_prompts() after
pre-formatting the template with str.format(), the agent silently fails at
query time with an unhandled KeyError. This happens because the template's
double-escaped braces (e.g. {{"input": "hello"}}) collapse to literal braces
during pre-formatting, which then crash Python's str.format() at runtime.

Changes:
- formatter.py: Replace str.format() with format_string() (SafeFormatter)
  followed by brace escape processing, handling both the default template
  and user pre-formatted templates correctly
- react_agent.py: Add validation warnings when required template variables
  ({tool_desc}, {tool_names}) are missing after prompt update
- test_prompt_customization.py: Add 5 new tests covering literal braces,
  default template JSON examples, pre-formatted templates, and warnings

Fixes run-llama#20416

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 3, 2026
Copy link
Member

@AstraBert AstraBert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand the fix:

  • We use format_string to avoid that the builtin .format( fails on formatting double curly brackets
  • format_string produces an output with single curly brackets (hello {{person}} -> hello {formatted}), and then we perform replacement in case there are still double curly brackets, but no problems with single curly brackets(?)
    Could you help me understand the reasoning here?

Comment on lines +82 to +83
fmt_sys_header = format_string(self.system_header, **format_args)
fmt_sys_header = fmt_sys_header.replace("{{", "{").replace("}}", "}")
Copy link
Member

@AstraBert AstraBert Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If self.system_header contains {{ or }}, format_string will do:

>>> format_string("hello {{person}}", person="ciao")
'hello {ciao}'

So, to sanitize the output, you will have to remove single curly brackets as well

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good observation — you're technically correct that format_string("hello {{person}}", person="ciao") produces hello {ciao}. However, this edge case is unreachable in the formatter because the substitution keys are constrained to a fixed set:

format_args = {
    "tool_desc": ...,
    "tool_names": ...,
}
if self.context:
    format_args["context"] = self.context

The only keys format_string will ever substitute are tool_desc, tool_names, and context. A template like {{person}} would never match any of those, so format_string leaves {{person}} untouched, and the subsequent .replace("{{", "{") correctly produces {person}.

The double-brace {{...}} escape sequences in the default template contain JSON examples like {{"input": "hello world", "num_beams": 5}} — none of whose inner content matches the three known keys, so they resolve cleanly to {"input": "hello world", "num_beams": 5} in the output.

I've pushed a test (test_formatter_substitutes_tool_args_with_literal_braces) that exercises this exact scenario: literal single-brace JSON alongside {tool_desc} and {tool_names} placeholders, verifying both that the tool values are substituted and the JSON survives intact.

Comment on lines +40 to +44
mock_tool = MagicMock()
mock_tool.metadata.name = "test_tool"
mock_tool.metadata.get_name.return_value = "test_tool"
mock_tool.metadata.description = "A test tool"
mock_tool.metadata.fn_schema_str = '{"type": "object"}'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about testing tool name and tool description?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call — I've pushed a new test that does exactly this. test_formatter_substitutes_tool_args_with_literal_braces verifies that {tool_desc} and {tool_names} are substituted with actual tool metadata while literal JSON braces in the same template survive intact:

assert "Search the web" in content      # tool description substituted
assert "> Tool Name: search" in content  # tool name substituted
assert '{"query": "search term"}' in content  # literal JSON preserved

The test uses single-brace JSON (not {{...}}), so it would have raised KeyError under the old str.format() code — making it a regression guard as well.

The test now uses literal single-brace JSON ({"query": "search term"})
instead of escaped double braces ({{...}}). This makes the test a true
regression guard: str.format() raises KeyError on this input, while
format_string() correctly tolerates it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Updating ReActAgent's system_prompt does not function correctly

2 participants