Skip to content

Commit c61b5b1

Browse files
committed
✨ Agent duplicate name handling logic improvement #1622
[Specification Details] 1.Add test cases.
1 parent 552b78f commit c61b5b1

File tree

4 files changed

+369
-10
lines changed

4 files changed

+369
-10
lines changed

backend/prompts/utils/prompt_generate.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -52,16 +52,16 @@ FEW_SHOTS_SYSTEM_PROMPT: |-
5252
- 用简单的Python编写代码
5353
- 遵循python代码规范和python语法
5454
- 根据格式规范正确调用工具/助手
55-
- 考虑到代码执行与展示用户代码的区别,使用'代码:\n```<RUN>\n'开头,并以'```<END_CODE>'表达运行代码,使用'代码:\n```<DISPLAY:语言类型>\n'开头,并以'```<END_CODE>'表达展示代码
56-
- 注意运行的代码不会被用户看到,所以如果用户需要看到代码,你需要使用'代码:\n```<DISPLAY:语言类型>\n'开头,并以'```<END_CODE>'表达展示代码。
55+
- 考虑到代码执行与展示用户代码的区别,使用'代码:\n```<RUN>\n'开头,并以'```<END_CODE>'表达运行代码,使用'代码:\n```<DISPLAY:语言类型>\n'开头,并以'```<END_DISPLAY_CODE>'表达展示代码
56+
- 注意运行的代码不会被用户看到,所以如果用户需要看到代码,你需要使用'代码:\n```<DISPLAY:语言类型>\n'开头,并以'```<END_DISPLAY_CODE>'表达展示代码。
5757
5858
3. 观察结果:
5959
- 查看代码执行结果
6060
6161
在思考结束后,当Agent认为可以回答用户问题,那么可以不生成代码,直接生成最终回答给到用户并停止循环。
6262
6363
### python代码规范
64-
1. 如果认为是需要执行的代码,代码内容以'代码:\n```<RUN>\n'开头,并以'```<END_CODE>'标识符结尾。如果是不需要执行仅用于展示的代码,代码内容以'代码:\n```<DISPLAY:语言类型>\n'开头,并以'```<END_CODE>'标识符结尾,其中语言类型例如python、java、javascript等;
64+
1. 如果认为是需要执行的代码,代码内容以'代码:\n```<RUN>\n'开头,并以'```<END_CODE>'标识符结尾。如果是不需要执行仅用于展示的代码,代码内容以'代码:\n```<DISPLAY:语言类型>\n'开头,并以'```<END_DISPLAY_CODE>'标识符结尾,其中语言类型例如python、java、javascript等;
6565
2. 只使用已定义的变量,变量将在多次调用之间持续保持;
6666
3. 使用“print()”函数让下一次的模型调用看到对应变量信息;
6767
4. 正确使用工具/助手的入参,使用关键字参数,不要用字典形式;
@@ -160,7 +160,7 @@ FEW_SHOTS_SYSTEM_PROMPT: |-
160160
middle = [x for x in arr if x == pivot]
161161
right = [x for x in arr if x > pivot]
162162
return quick_sort(left) + middle + quick_sort(right)
163-
```<END_CODE>
163+
```<END_DISPLAY_CODE>
164164
观察结果:快速排序的python代码。
165165
166166
思考:我已经获得了快速排序的python代码,现在我将生成最终回答。
@@ -174,7 +174,7 @@ FEW_SHOTS_SYSTEM_PROMPT: |-
174174
middle = [x for x in arr if x == pivot]
175175
right = [x for x in arr if x > pivot]
176176
return quick_sort(left) + middle + quick_sort(right)
177-
```<END_CODE>
177+
```<END_DISPLAY_CODE>
178178
179179
---
180180

backend/prompts/utils/prompt_generate_en.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -53,16 +53,16 @@ FEW_SHOTS_SYSTEM_PROMPT: |-
5353
- Write code in simple Python
5454
- Follow Python coding standards and Python syntax
5555
- Call tools/assistants correctly according to format specifications
56-
- To distinguish between code execution and displaying user code, use 'Code: \n```<RUN>\n' to start executing code and '```<END_CODE>' to indicate its completion. Use 'Code: \n```<DISPLAY:language_type>\n' to start displaying code and '```<END_CODE>' to indicate its completion.
57-
- Note that executed code is not visible to users. If users need to see the code, use 'Code: \n```<DISPLAY:language_type>\n' as the start and '```<END_CODE>' to denote displayed code.
56+
- To distinguish between code execution and displaying user code, use 'Code: \n```<RUN>\n' to start executing code and '```<END_CODE>' to indicate its completion. Use 'Code: \n```<DISPLAY:language_type>\n' to start displaying code and '```<END_DISPLAY_CODE>' to indicate its completion.
57+
- Note that executed code is not visible to users. If users need to see the code, use 'Code: \n```<DISPLAY:language_type>\n' as the start and '```<END_DISPLAY_CODE>' to denote displayed code.
5858
5959
3. Observe Results:
6060
- View code execution results
6161
6262
After thinking, when you believe you can answer the user's question, you can generate a final answer directly to the user without generating code and stop the loop.
6363
6464
### Python Code Specifications
65-
1. If it is considered to be code that needs to be executed, the code content begins with 'Code:\n```<RUN>\n' and ends with '```<END_CODE>'. If the code does not need to be executed for display only, the code content begins with 'Code:\n```<DISPLAY:language_type>\n', and ends with '```<END_CODE>', where language_type can be python, java, javascript, etc.;
65+
1. If it is considered to be code that needs to be executed, the code content begins with 'Code:\n```<RUN>\n' and ends with '```<END_CODE>'. If the code does not need to be executed for display only, the code content begins with 'Code:\n```<DISPLAY:language_type>\n', and ends with '```<END_DISPLAY_CODE>', where language_type can be python, java, javascript, etc.;
6666
2. Only use defined variables, variables will persist between multiple calls;
6767
3. Use "print()" function to let the next model call see corresponding variable information;
6868
4. Use tool/assistant input parameters correctly, use keyword arguments, not dictionary format;
@@ -158,7 +158,7 @@ FEW_SHOTS_SYSTEM_PROMPT: |-
158158
middle = [x for x in arr if x == pivot]
159159
right = [x for x in arr if x > pivot]
160160
return quick_sort(left) + middle + quick_sort(right)
161-
```<END_CODE>
161+
```<END_DISPLAY_CODE>
162162
Observe Results: The Python quick sort code.
163163
164164
Think: I have obtained the Python quick sort code, now I will generate the final answer.
@@ -172,7 +172,7 @@ FEW_SHOTS_SYSTEM_PROMPT: |-
172172
middle = [x for x in arr if x == pivot]
173173
right = [x for x in arr if x > pivot]
174174
return quick_sort(left) + middle + quick_sort(right)
175-
```<END_CODE>
175+
```<END_DISPLAY_CODE>
176176
177177
---
178178

test/backend/services/test_agent_service.py

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5302,6 +5302,49 @@ def test_check_agent_value_duplicate_with_and_without_exclude():
53025302
)
53035303

53045304

5305+
@patch('backend.services.agent_service.query_all_agent_info_by_tenant_id')
5306+
def test_check_agent_value_duplicate_empty_value(mock_query_all):
5307+
"""_check_agent_value_duplicate should return False when value is empty."""
5308+
# Test empty string
5309+
assert not agent_service._check_agent_value_duplicate(
5310+
"name", "", tenant_id="t", agents_cache=[]
5311+
)
5312+
# Test None value
5313+
assert not agent_service._check_agent_value_duplicate(
5314+
"name", None, tenant_id="t", agents_cache=[]
5315+
)
5316+
# Should not call query_all_agent_info_by_tenant_id when value is empty
5317+
mock_query_all.assert_not_called()
5318+
5319+
5320+
@patch('backend.services.agent_service.query_all_agent_info_by_tenant_id')
5321+
def test_check_agent_value_duplicate_cache_none(mock_query_all):
5322+
"""_check_agent_value_duplicate should query database when agents_cache is None."""
5323+
mock_query_all.return_value = [
5324+
{"agent_id": 1, "name": "agent_one"},
5325+
{"agent_id": 2, "name": "agent_two"},
5326+
]
5327+
5328+
# Should query database when cache is None
5329+
assert agent_service._check_agent_value_duplicate(
5330+
"name", "agent_one", tenant_id="t", agents_cache=None
5331+
)
5332+
mock_query_all.assert_called_once_with("t")
5333+
5334+
# Reset mock
5335+
mock_query_all.reset_mock()
5336+
mock_query_all.return_value = [
5337+
{"agent_id": 1, "name": "agent_one"},
5338+
{"agent_id": 2, "name": "agent_two"},
5339+
]
5340+
5341+
# Should query database when cache is None and no duplicate found
5342+
assert not agent_service._check_agent_value_duplicate(
5343+
"name", "agent_three", tenant_id="t", agents_cache=None
5344+
)
5345+
mock_query_all.assert_called_once_with("t")
5346+
5347+
53055348
def test_generate_unique_value_with_suffix_success():
53065349
"""_generate_unique_value_with_suffix should find first available suffix."""
53075350

@@ -5448,6 +5491,203 @@ def fallback(base):
54485491
assert used.get("called") is True
54495492

54505493

5494+
def test_regenerate_agent_value_with_llm_empty_system_prompt(monkeypatch):
5495+
"""_regenerate_agent_value_with_llm should use default_system_prompt when system_prompt is empty."""
5496+
5497+
monkeypatch.setattr(
5498+
agent_service,
5499+
"get_prompt_generate_prompt_template",
5500+
lambda lang: {},
5501+
raising=False,
5502+
)
5503+
monkeypatch.setattr(
5504+
agent_service,
5505+
"_render_prompt_template",
5506+
lambda template_str, **kwargs: "", # Return empty string
5507+
raising=False,
5508+
)
5509+
5510+
def fake_call_llm(model_id, user_prompt, system_prompt, callback, tenant_id):
5511+
# Verify that default_system_prompt was used
5512+
assert system_prompt == "default_system"
5513+
return "new_name"
5514+
5515+
fake_prompt_module = MagicMock()
5516+
fake_prompt_module.call_llm_for_system_prompt = fake_call_llm
5517+
sys.modules["services.prompt_service"] = fake_prompt_module
5518+
5519+
result = _regenerate_agent_value_with_llm(
5520+
original_value="old",
5521+
existing_values=["existing"],
5522+
task_description="task",
5523+
model_id=1,
5524+
tenant_id="tenant",
5525+
language="en",
5526+
system_prompt_key="SYS_KEY",
5527+
user_prompt_key="USER_KEY",
5528+
default_system_prompt="default_system",
5529+
default_user_prompt_builder=lambda ctx: "user",
5530+
fallback_fn=lambda base: f"fallback_{base}",
5531+
)
5532+
assert result == "new_name"
5533+
5534+
5535+
def test_regenerate_agent_value_with_llm_empty_user_prompt(monkeypatch):
5536+
"""_regenerate_agent_value_with_llm should use default_user_prompt_builder when user_prompt is empty."""
5537+
5538+
monkeypatch.setattr(
5539+
agent_service,
5540+
"get_prompt_generate_prompt_template",
5541+
lambda lang: {},
5542+
raising=False,
5543+
)
5544+
5545+
call_count = {"render_count": 0}
5546+
5547+
def mock_render(template_str, **kwargs):
5548+
call_count["render_count"] += 1
5549+
# First call is for system_prompt, return non-empty
5550+
if call_count["render_count"] == 1:
5551+
return "system_prompt"
5552+
# Second call is for user_prompt, return empty
5553+
return ""
5554+
5555+
monkeypatch.setattr(
5556+
agent_service,
5557+
"_render_prompt_template",
5558+
mock_render,
5559+
raising=False,
5560+
)
5561+
5562+
def fake_call_llm(model_id, user_prompt, system_prompt, callback, tenant_id):
5563+
# Verify that default_user_prompt_builder was used
5564+
assert user_prompt == "default_user"
5565+
return "new_name"
5566+
5567+
fake_prompt_module = MagicMock()
5568+
fake_prompt_module.call_llm_for_system_prompt = fake_call_llm
5569+
sys.modules["services.prompt_service"] = fake_prompt_module
5570+
5571+
result = _regenerate_agent_value_with_llm(
5572+
original_value="old",
5573+
existing_values=["existing"],
5574+
task_description="task",
5575+
model_id=1,
5576+
tenant_id="tenant",
5577+
language="en",
5578+
system_prompt_key="SYS_KEY",
5579+
user_prompt_key="USER_KEY",
5580+
default_system_prompt="system_prompt",
5581+
default_user_prompt_builder=lambda ctx: "default_user",
5582+
fallback_fn=lambda base: f"fallback_{base}",
5583+
)
5584+
assert result == "new_name"
5585+
5586+
5587+
def test_regenerate_agent_value_with_llm_duplicate_candidate(monkeypatch):
5588+
"""_regenerate_agent_value_with_llm should raise ValueError when generated candidate is duplicate."""
5589+
5590+
monkeypatch.setattr(
5591+
agent_service,
5592+
"get_prompt_generate_prompt_template",
5593+
lambda lang: {},
5594+
raising=False,
5595+
)
5596+
5597+
attempt_count = {"count": 0}
5598+
5599+
def fake_call_llm(model_id, user_prompt, system_prompt, callback, tenant_id):
5600+
attempt_count["count"] += 1
5601+
# Return a value that exists in existing_values
5602+
if attempt_count["count"] == 1:
5603+
return "existing" # This is a duplicate
5604+
# On retry, return a unique value
5605+
return "new_unique_name"
5606+
5607+
fake_prompt_module = MagicMock()
5608+
fake_prompt_module.call_llm_for_system_prompt = fake_call_llm
5609+
sys.modules["services.prompt_service"] = fake_prompt_module
5610+
5611+
result = _regenerate_agent_value_with_llm(
5612+
original_value="old",
5613+
existing_values=["existing", "another"],
5614+
task_description="task",
5615+
model_id=1,
5616+
tenant_id="tenant",
5617+
language="en",
5618+
system_prompt_key="SYS_KEY",
5619+
user_prompt_key="USER_KEY",
5620+
default_system_prompt="sys",
5621+
default_user_prompt_builder=lambda ctx: "user",
5622+
fallback_fn=lambda base: f"fallback_{base}",
5623+
)
5624+
# Should retry and eventually return a unique value
5625+
assert result == "new_unique_name"
5626+
assert attempt_count["count"] == 2
5627+
5628+
5629+
def test_regenerate_agent_name_with_llm(monkeypatch):
5630+
"""_regenerate_agent_name_with_llm should call _regenerate_agent_value_with_llm with correct parameters."""
5631+
5632+
monkeypatch.setattr(
5633+
agent_service,
5634+
"get_prompt_generate_prompt_template",
5635+
lambda lang: {},
5636+
raising=False,
5637+
)
5638+
5639+
def fake_call_llm(model_id, user_prompt, system_prompt, callback, tenant_id):
5640+
return "new_agent_name"
5641+
5642+
fake_prompt_module = MagicMock()
5643+
fake_prompt_module.call_llm_for_system_prompt = fake_call_llm
5644+
sys.modules["services.prompt_service"] = fake_prompt_module
5645+
5646+
result = agent_service._regenerate_agent_name_with_llm(
5647+
original_name="old_name",
5648+
existing_names=["existing1", "existing2"],
5649+
task_description="task desc",
5650+
model_id=1,
5651+
tenant_id="tenant",
5652+
language="en",
5653+
agents_cache=[],
5654+
exclude_agent_id=None
5655+
)
5656+
5657+
assert result == "new_agent_name"
5658+
5659+
5660+
def test_regenerate_agent_display_name_with_llm(monkeypatch):
5661+
"""_regenerate_agent_display_name_with_llm should call _regenerate_agent_value_with_llm with correct parameters."""
5662+
5663+
monkeypatch.setattr(
5664+
agent_service,
5665+
"get_prompt_generate_prompt_template",
5666+
lambda lang: {},
5667+
raising=False,
5668+
)
5669+
5670+
def fake_call_llm(model_id, user_prompt, system_prompt, callback, tenant_id):
5671+
return "New Display Name"
5672+
5673+
fake_prompt_module = MagicMock()
5674+
fake_prompt_module.call_llm_for_system_prompt = fake_call_llm
5675+
sys.modules["services.prompt_service"] = fake_prompt_module
5676+
5677+
result = agent_service._regenerate_agent_display_name_with_llm(
5678+
original_display_name="Old Display Name",
5679+
existing_display_names=["Display1", "Display2"],
5680+
task_description="task desc",
5681+
model_id=1,
5682+
tenant_id="tenant",
5683+
language="en",
5684+
agents_cache=[],
5685+
exclude_agent_id=None
5686+
)
5687+
5688+
assert result == "New Display Name"
5689+
5690+
54515691
@pytest.mark.asyncio
54525692
async def test_import_agent_impl_dfs_import_order(monkeypatch):
54535693
"""

0 commit comments

Comments
 (0)