fix: prevent silent KeyError in ReActAgent after update_prompts()#20863
fix: prevent silent KeyError in ReActAgent after update_prompts()#20863roli-lpci wants to merge 2 commits intorun-llama:mainfrom
Conversation
…re-formatted prompts
When users update a ReActAgent's system prompt via update_prompts() after
pre-formatting the template with str.format(), the agent silently fails at
query time with an unhandled KeyError. This happens because the template's
double-escaped braces (e.g. {{"input": "hello"}}) collapse to literal braces
during pre-formatting, which then crash Python's str.format() at runtime.
Changes:
- formatter.py: Replace str.format() with format_string() (SafeFormatter)
followed by brace escape processing, handling both the default template
and user pre-formatted templates correctly
- react_agent.py: Add validation warnings when required template variables
({tool_desc}, {tool_names}) are missing after prompt update
- test_prompt_customization.py: Add 5 new tests covering literal braces,
default template JSON examples, pre-formatted templates, and warnings
Fixes run-llama#20416
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AstraBert
left a comment
There was a problem hiding this comment.
I am not sure I understand the fix:
- We use
format_stringto avoid that the builtin.format(fails on formatting double curly brackets format_stringproduces an output with single curly brackets (hello {{person}}->hello {formatted}), and then we perform replacement in case there are still double curly brackets, but no problems with single curly brackets(?)
Could you help me understand the reasoning here?
| fmt_sys_header = format_string(self.system_header, **format_args) | ||
| fmt_sys_header = fmt_sys_header.replace("{{", "{").replace("}}", "}") |
There was a problem hiding this comment.
If self.system_header contains {{ or }}, format_string will do:
>>> format_string("hello {{person}}", person="ciao")
'hello {ciao}'So, to sanitize the output, you will have to remove single curly brackets as well
There was a problem hiding this comment.
Good observation — you're technically correct that format_string("hello {{person}}", person="ciao") produces hello {ciao}. However, this edge case is unreachable in the formatter because the substitution keys are constrained to a fixed set:
format_args = {
"tool_desc": ...,
"tool_names": ...,
}
if self.context:
format_args["context"] = self.contextThe only keys format_string will ever substitute are tool_desc, tool_names, and context. A template like {{person}} would never match any of those, so format_string leaves {{person}} untouched, and the subsequent .replace("{{", "{") correctly produces {person}.
The double-brace {{...}} escape sequences in the default template contain JSON examples like {{"input": "hello world", "num_beams": 5}} — none of whose inner content matches the three known keys, so they resolve cleanly to {"input": "hello world", "num_beams": 5} in the output.
I've pushed a test (test_formatter_substitutes_tool_args_with_literal_braces) that exercises this exact scenario: literal single-brace JSON alongside {tool_desc} and {tool_names} placeholders, verifying both that the tool values are substituted and the JSON survives intact.
| mock_tool = MagicMock() | ||
| mock_tool.metadata.name = "test_tool" | ||
| mock_tool.metadata.get_name.return_value = "test_tool" | ||
| mock_tool.metadata.description = "A test tool" | ||
| mock_tool.metadata.fn_schema_str = '{"type": "object"}' |
There was a problem hiding this comment.
What about testing tool name and tool description?
There was a problem hiding this comment.
Good call — I've pushed a new test that does exactly this. test_formatter_substitutes_tool_args_with_literal_braces verifies that {tool_desc} and {tool_names} are substituted with actual tool metadata while literal JSON braces in the same template survive intact:
assert "Search the web" in content # tool description substituted
assert "> Tool Name: search" in content # tool name substituted
assert '{"query": "search term"}' in content # literal JSON preservedThe test uses single-brace JSON (not {{...}}), so it would have raised KeyError under the old str.format() code — making it a regression guard as well.
The test now uses literal single-brace JSON ({"query": "search term"})
instead of escaped double braces ({{...}}). This makes the test a true
regression guard: str.format() raises KeyError on this input, while
format_string() correctly tolerates it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
Fixes #20416 —
ReActAgentsilently stops working after updating the system prompt viaupdate_prompts().Root cause: The formatter at
formatter.py:81uses Python'sstr.format()on the stored system header, which crashes withKeyErrorwhen the string contains literal braces ({,}) from pre-formatted template escape sequences ({{,}}). The exception propagates through the workflow runtime and produces empty agent output with no traceback.Fix (2 files, ~15 lines):
formatter.py: Replacestr.format()withformat_string()(the existingSafeFormatterutility) followed by brace escape processing. This safely substitutes{tool_desc}and{tool_names}without crashing on literal braces, while preserving the default template's JSON examples for the LLM.react_agent.py: AddUserWarningwhenupdate_prompts()receives a prompt missing required{tool_desc}or{tool_names}placeholders, giving users immediate feedback instead of silent runtime failure.Test plan
test_prompt_customization.py(formatter with literal braces, default template JSON examples, pre-formatted template update, warning on missing placeholders, no false warnings)test_react_agent_promptspassestest_inheritance_react_chat_formatterpassestest_react_output_parsertests pass🤖 Generated with Claude Code