Model Provider Issue - novita - glm 4.7 - hugging face router #1135
Description
Error Details
Repository Error
Type: REPOSITORY
Message: CodebaseWorkflowError: 2 codebase(s) failed during agent execution
Stack Trace
Traceback (most recent call last):
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/interceptors/agent_workflow_interceptor.py", line 225, in _handle_repository_workflow
result = await self.next.execute_workflow(input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/temporalio/worker/_workflow_instance.py", line 2776, in execute_workflow
return await input.run_fn(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py", line 475, in run
raise ApplicationError(
...<4 lines>...
)
temporalio.exceptions.ApplicationError: CodebaseWorkflowError: 2 codebase(s) failed during agent execution
Metadata
{
"type": "CodebaseWorkflowError",
"details": [
[
{
"error": "Child Workflow execution failed",
"codebase": "backend"
},
{
"error": "Child Workflow execution failed",
"codebase": "frontend"
}
]
],
"non_retryable": true,
"next_retry_delay": null
}Codebase Errors (2 failed)
backend
Message: AgentExecutionError: Agent execution failed for 2 agent(s) in codebase 'backend'
Traceback (most recent call last):
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/interceptors/agent_workflow_interceptor.py", line 357, in _handle_codebase_workflow
result = await self.next.execute_workflow(input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/temporalio/worker/_workflow_instance.py", line 2776, in execute_workflow
return await input.run_fn(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py", line 315, in run
raise ApplicationError(
...<4 lines>...
)
temporalio.exceptions.ApplicationError: AgentExecutionError: Agent execution failed for 2 agent(s) in codebase 'backend'
Metadata
{
"type": "AgentExecutionError",
"details": [
[
{
"agent": "project_configuration_agent",
"error": "Exceeded maximum retries (1) for output validation",
"codebase": "backend",
"traceback": "Traceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py\", line 132, in run\n config_result = await temporal_agents[\n ^^^^^^^^^^^^^^^^^^^^^^\n ...<5 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 392, in run\n return await super().run(\n ^^^^^^^^^^^^^^^^^^\n ...<15 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/agent/abstract.py\", line 270, in run\n await event_stream_handler(_agent_graph.build_run_context(agent_run.ctx), stream)\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 233, in _call_event_stream_handler_activity\n async for event in stream:\n ...<10 lines>...\n )\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 714, in _run_stream\n async for event in self._events_iterator:\n yield event\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 637, in _run_stream\n ctx.state.increment_retries(ctx.deps.max_result_retries, model_settings=ctx.deps.model_settings)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 123, in increment_retries\n raise exceptions.UnexpectedModelBehavior(message)\npydantic_ai.exceptions.UnexpectedModelBehavior: Exceeded maximum retries (1) for output validation\n"
},
{
"agent": "development_workflow_agent",
"error": "Exceeded maximum retries (1) for output validation",
"codebase": "backend",
"traceback": "Traceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_output.py\", line 625, in process\n output = self.validate(data, allow_partial=allow_partial, validation_context=run_context.validation_context)\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_output.py\", line 648, in validate\n return self.validator.validate_json(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n data or '{}', allow_partial=pyd_allow_partial, context=validation_context\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\npydantic_core._pydantic_core.ValidationError: 1 validation error for DevelopmentWorkflow\n Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<tool_call>get_directory_tree', input_type=str]\n For further information visit https://errors.pydantic.dev/2.11/v/json_invalid\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 691, in _run_stream\n self._next_node = await self._handle_text_response(ctx, text, text_processor)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 762, in _handle_text_response\n result_data = await text_processor.process(text, run_context=run_context)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_output.py\", line 631, in process\n raise ToolRetryError(m) from e\npydantic_ai.exceptions.ToolRetryError: 1 validation error\n__root__\n Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<tool_call>get_directory_tree']\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py\", line 196, in run\n workflow_result = await temporal_agents[\n ^^^^^^^^^^^^^^^^^^^^^^\n ...<4 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 392, in run\n return await super().run(\n ^^^^^^^^^^^^^^^^^^\n ...<15 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/agent/abstract.py\", line 270, in run\n await event_stream_handler(_agent_graph.build_run_context(agent_run.ctx), stream)\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 233, in _call_event_stream_handler_activity\n async for event in stream:\n ...<10 lines>...\n )\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 714, in _run_stream\n async for event in self._events_iterator:\n yield event\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 703, in _run_stream\n ctx.state.increment_retries(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n ctx.deps.max_result_retries, error=e, model_settings=ctx.deps.model_settings\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 121, in increment_retries\n raise exceptions.UnexpectedModelBehavior(message) from error\npydantic_ai.exceptions.UnexpectedModelBehavior: Exceeded maximum retries (1) for output validation\n"
}
]
],
"non_retryable": true,
"next_retry_delay": null
}frontend
Message: AgentExecutionError: Agent execution failed for 3 agent(s) in codebase 'frontend'
Traceback (most recent call last):
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/interceptors/agent_workflow_interceptor.py", line 357, in _handle_codebase_workflow
result = await self.next.execute_workflow(input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/temporalio/worker/_workflow_instance.py", line 2776, in execute_workflow
return await input.run_fn(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py", line 315, in run
raise ApplicationError(
...<4 lines>...
)
temporalio.exceptions.ApplicationError: AgentExecutionError: Agent execution failed for 3 agent(s) in codebase 'frontend'
Metadata
{
"type": "AgentExecutionError",
"details": [
[
{
"agent": "project_configuration_agent",
"error": "Exceeded maximum retries (1) for output validation",
"codebase": "frontend",
"traceback": "Traceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_output.py\", line 625, in process\n output = self.validate(data, allow_partial=allow_partial, validation_context=run_context.validation_context)\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_output.py\", line 648, in validate\n return self.validator.validate_json(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n data or '{}', allow_partial=pyd_allow_partial, context=validation_context\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\npydantic_core._pydantic_core.ValidationError: 1 validation error for ProjectConfiguration\n Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<tool_call>get_directory_tree', input_type=str]\n For further information visit https://errors.pydantic.dev/2.11/v/json_invalid\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 691, in _run_stream\n self._next_node = await self._handle_text_response(ctx, text, text_processor)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 762, in _handle_text_response\n result_data = await text_processor.process(text, run_context=run_context)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_output.py\", line 631, in process\n raise ToolRetryError(m) from e\npydantic_ai.exceptions.ToolRetryError: 1 validation error\n__root__\n Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<tool_call>get_directory_tree']\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py\", line 132, in run\n config_result = await temporal_agents[\n ^^^^^^^^^^^^^^^^^^^^^^\n ...<5 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 392, in run\n return await super().run(\n ^^^^^^^^^^^^^^^^^^\n ...<15 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/agent/abstract.py\", line 270, in run\n await event_stream_handler(_agent_graph.build_run_context(agent_run.ctx), stream)\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 233, in _call_event_stream_handler_activity\n async for event in stream:\n ...<10 lines>...\n )\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 714, in _run_stream\n async for event in self._events_iterator:\n yield event\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 703, in _run_stream\n ctx.state.increment_retries(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n ctx.deps.max_result_retries, error=e, model_settings=ctx.deps.model_settings\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 121, in increment_retries\n raise exceptions.UnexpectedModelBehavior(message) from error\npydantic_ai.exceptions.UnexpectedModelBehavior: Exceeded maximum retries (1) for output validation\n"
},
{
"agent": "development_workflow_agent",
"error": "Exceeded maximum retries (1) for output validation",
"codebase": "frontend",
"traceback": "Traceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py\", line 196, in run\n workflow_result = await temporal_agents[\n ^^^^^^^^^^^^^^^^^^^^^^\n ...<4 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 392, in run\n return await super().run(\n ^^^^^^^^^^^^^^^^^^\n ...<15 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/agent/abstract.py\", line 270, in run\n await event_stream_handler(_agent_graph.build_run_context(agent_run.ctx), stream)\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 233, in _call_event_stream_handler_activity\n async for event in stream:\n ...<10 lines>...\n )\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 714, in _run_stream\n async for event in self._events_iterator:\n yield event\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 637, in _run_stream\n ctx.state.increment_retries(ctx.deps.max_result_retries, model_settings=ctx.deps.model_settings)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 123, in increment_retries\n raise exceptions.UnexpectedModelBehavior(message)\npydantic_ai.exceptions.UnexpectedModelBehavior: Exceeded maximum retries (1) for output validation\n"
},
{
"agent": "business_logic_domain_agent",
"error": "Exceeded maximum retries (1) for output validation",
"codebase": "frontend",
"traceback": "Traceback (most recent call last):\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/src/unoplat_code_confluence_query_engine/services/temporal/temporal_workflows.py\", line 254, in run\n domain_result = await temporal_agents[\n ^^^^^^^^^^^^^^^^^^^^^^\n ...<4 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 392, in run\n return await super().run(\n ^^^^^^^^^^^^^^^^^^\n ...<15 lines>...\n )\n ^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/agent/abstract.py\", line 270, in run\n await event_stream_handler(_agent_graph.build_run_context(agent_run.ctx), stream)\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/durable_exec/temporal/_agent.py\", line 233, in _call_event_stream_handler_activity\n async for event in stream:\n ...<10 lines>...\n )\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 714, in _run_stream\n async for event in self._events_iterator:\n yield event\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 637, in _run_stream\n ctx.state.increment_retries(ctx.deps.max_result_retries, model_settings=ctx.deps.model_settings)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/jayghiya/Documents/unoplat/unoplat-code-confluence/unoplat-code-confluence-query-engine/.venv/lib/python3.13/site-packages/pydantic_ai/_agent_graph.py\", line 123, in increment_retries\n raise exceptions.UnexpectedModelBehavior(message)\npydantic_ai.exceptions.UnexpectedModelBehavior: Exceeded maximum retries (1) for output validation\n"
}
]
],
"non_retryable": true,
"next_retry_delay": null
}Additional Information
Workflow Information
- Repository: fastapi/full-stack-fastapi-template