-
Notifications
You must be signed in to change notification settings - Fork 19
Description
📝 Description
I have six agents that fully comply with the official A2A Inspector specifications and correctly implement all required A2A methods — namely tasks/get and tasks/cancel.
However, when running the TCK against these agents, I consistently encounter errors related to cancellation of already completed tasks.
I’ve already tried introducing artificial delays in the agent’s responses to prevent tasks from completing instantly, but this didn’t resolve the issue.
Could this behavior be related to version compatibility — for instance, between the A2A SDK v0.3.8 and the TCK tests built for v0.3.0?
Does the TCK currently support all v0.3.x agents, or is it limited to v0.3.0?
Any suggestions on why this might be happening and/or how to overcome these failures to ensure that my agents pass all the mandatory tests?
⚙️ Version Information
- A2A SDK:
v0.3.8
🐛 Errors
======================================================= FAILURES =======================================================
___________________________ TestMethodMappingCompliance.test_core_method_mapping_compliance ____________________________
tests/mandatory/protocol/test_a2a_v030_new_methods.py:384: in test_core_method_mapping_compliance
assert cancelled_task["id"] == task_id
^^^^^^^^^^^^^^^^^^^^
E KeyError: 'id'
During handling of the above exception, another exception occurred:
tests/mandatory/protocol/test_a2a_v030_new_methods.py:387: in test_core_method_mapping_compliance
pytest.fail(f"Core method mapping test failed on {transport_type}: {e}")
E Failed: Core method mapping test failed on jsonrpc: 'id'
------------------------------------------------- Captured stdout call -------------------------------------------------
{'error': {'code': -32002, 'message': 'Task cannot be canceled - current state: TaskState.completed'}}
-------------------------------------------------- Captured log call ---------------------------------------------------
ERROR tck.transport.jsonrpc_client.JSONRPCClient:jsonrpc_client.py:143 JSON-RPC error from SUT: {'code': -32002, 'message': 'Task cannot be canceled - current state: TaskState.completed'}
___________________________________________ test_message_send_continue_task ____________________________________________
tests/mandatory/protocol/test_message_send_method.py:183: in test_message_send_continue_task
assert is_json_rpc_success_response(second_resp), f"Task continuation failed: {second_resp}"
E AssertionError: Task continuation failed: {'error': {'code': -32602, 'message': 'Task d51a9cf0-7576-4c65-9007-7766e366c337 is in terminal state: completed'}}
E assert False
E + where False = is_json_rpc_success_response({'error': {'code': -32602, 'message': 'Task d51a9cf0-7576-4c65-9007-7766e366c337 is in terminal state: completed'}})
-------------------------------------------------- Captured log call ---------------------------------------------------
ERROR tck.transport.jsonrpc_client.JSONRPCClient:jsonrpc_client.py:143 JSON-RPC error from SUT: {'code': -32602, 'message': 'Task d51a9cf0-7576-4c65-9007-7766e366c337 is in terminal state: completed'}
_______________________________________________ test_task_history_length _______________________________________________
tests/mandatory/protocol/test_state_transitions.py:80: in test_task_history_length
assert transport_helpers.is_json_rpc_success_response(update_resp)
E AssertionError: assert False
E + where False = <function is_json_rpc_success_response at 0x7fa4d22a65c0>({'error': {'code': -32602, 'message': 'Task 62211576-c2e8-45fe-a170-36900bc0cfd2 is in terminal state: completed'}})
E + where <function is_json_rpc_success_response at 0x7fa4d22a65c0> = <module 'tests.utils.transport_helpers' from '/home/martimsantos/a2a-tck/tests/utils/transport_helpers.py'>.is_json_rpc_success_response
-------------------------------------------------- Captured log call ---------------------------------------------------
ERROR tck.transport.jsonrpc_client.JSONRPCClient:jsonrpc_client.py:143 JSON-RPC error from SUT: {'code': -32602, 'message': 'Task 62211576-c2e8-45fe-a170-36900bc0cfd2 is in terminal state: completed'}
_______________________________________________ test_tasks_cancel_valid ________________________________________________
tests/mandatory/protocol/test_tasks_cancel_method.py:53: in test_tasks_cancel_valid
assert is_json_rpc_success_response(resp), f"Task cancellation failed: {resp}"
E AssertionError: Task cancellation failed: {'error': {'code': -32002, 'message': 'Task cannot be canceled - current state: TaskState.completed'}}
E assert False
E + where False = is_json_rpc_success_response({'error': {'code': -32002, 'message': 'Task cannot be canceled - current state: TaskState.completed'}})
-------------------------------------------------- Captured log call ---------------------------------------------------
ERROR tck.transport.jsonrpc_client.JSONRPCClient:jsonrpc_client.py:143 JSON-RPC error from SUT: {'code': -32002, 'message': 'Task cannot be canceled - current state: TaskState.completed'}
=============================================== short test summary info ================================================
FAILED tests/mandatory/protocol/test_a2a_v030_new_methods.py::TestMethodMappingCompliance::test_core_method_mapping_compliance - Failed: Core method mapping test failed on jsonrpc: 'id'
FAILED tests/mandatory/protocol/test_message_send_method.py::test_message_send_continue_task - AssertionError: Task continuation failed: {'error': {'code': -32602, 'message': 'Task d51a9cf0-7576-4c65-9007-7766e...
FAILED tests/mandatory/protocol/test_state_transitions.py::test_task_history_length - AssertionError: assert False
FAILED tests/mandatory/protocol/test_tasks_cancel_method.py::test_tasks_cancel_valid - AssertionError: Task cancellation failed: {'error': {'code': -32002, 'message': 'Task cannot be canceled - current ...