Feature: Non-Blocking call_tool and request state externalisation

[davemssavage]

This patch enables a non-blocking mode of running tools and introduces a RequestStateManager plugin api - this allows for long running tasks to be rejoined on the client and across process restarts.

Motivation and Context

Rather than introducing new protocol messages as per modelcontextprotocol/modelcontextprotocol#617 this patch uses the existing protocol messages with client side changes to allow the client application to submit a call request and then later join that request to get the response and/or any progress notifications that may have occured. This works on top of the existing session resume processes so a client can submit a task in one process and rejoin from another if a persistent version of RequestStateManager is used (this has been tested using a redis implementation - not included in this patch)

Long running tasks currently consume resources indefinitely on the client blocking until they return, Tool calls can currently timeout allowing control to return to the client however this leads to a messy control flow that exposes the internal details of the protocol implementaton - having to check the type of code is httpx.codes.REQUEST_TIMEOUT.

This patch instead introduces new methods request_call_tool and join_call_tool that handle various complex issues with resume tokens and via state stored in a pluggable RequestStateManager. An InMemoryRequestStateManager is provided as the default, other external state managers can be provided to the ClientSession to allow for persistence outside of process memory.

How Has This Been Tested?

This has been unit tested and tested on a server application that managed multiple sessions without blocking the server side threads.

Breaking Changes

None

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the MCP Documentation
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

Additional context

In order to handle some complex logic around resumption this patch introduces a new ResumeCapability which can be returned during initialisation. It currently is hard coded to assume that if streamable_http protocol is used then server sessions are resumeable. It may be worth while extending this along the lines of modelcontextprotocol/modelcontextprotocol#617 where servers can indicate how long clients might expect a session to be resumable for.

Testing on a non-trivial application has highlighted that at least one progress notification needs to be sent from the server prior to the start_request call returning a request_id. This is due to needing at least one event to trigger the sse.event_id to be passed back to the client for the resume token to be collected. This is a little opaque and relies on the client having passed a progress callback in the start_call_tool method. A new protocol notification sent from the server to indicate it has received the request could be a viable alternative, however one of the benefits of this is patch over modelcontextprotocol/modelcontextprotocol#617 is it is trying to avoid new protocol messages.

[felixweinberger]

This seems closely related to modelcontextprotocol/modelcontextprotocol#1391 where there's an ongoing discussion about the correct abstraction for long-running or async tools.

In order to support this we'll likely need to align on the right path forward for implementation - if we end up adding this at the protocol layer we'll likely want to leverage any messages provided there.

[felixweinberger]

Converting this to draft for now to remove it from the review queue while we wait for the outcome of modelcontextprotocol/modelcontextprotocol#1391