Current implementation of HttpServletStreamableServerTransportProvider or HttpServletSseServerTransportProvider needs strict stickyness

[olamy]

Current implementations are holding mcp sessions into a in memory Map

java-sdk/mcp/src/main/java/io/modelcontextprotocol/server/transport/HttpServletStreamableServerTransportProvider.java

Line 108 in 713ee1a

private final ConcurrentHashMap<String, McpStreamableServerSession> sessions = new ConcurrentHashMap<>();

This make it mandatory to have strict stickyness "a la" browser" in a replicated/high availability environement which doesn't exist for agent (and we cannot rely on llm agents to implement this)
The feature could be

• first very simple option an option for HttpServletStreamableServerTransportProvider.Builder to have an option to provide the map (then users can use different tools to have some shared Map)
• create an interface to handle those sessions (with minimum operations such add, get. delete)

[olamy]

just put a rough idea for this here #500

[olamy]

Hi @tzolov @Kehrlann
I wonder if there is some chance to have a fix for this. I can fix merge conficts from the PR if you think this solution is doable.
Currently, I cannot really have a scalable solution for the MCP server deployed on some pods with replica > 1. If a pod goes down, all MCP sessions are lost, and the following MCP call from the same agent gets some no mcp session id found.

[quaff]

You can use stateless transport instead if it meet your requirement.

[olamy]

You can use stateless transport instead if it meet your requirement.

not sure what you mean here?
If using http streamable, not really sure how to do it differently?
Do you have an example?
thanks

[quaff]

See https://docs.spring.io/spring-ai/reference/1.1-SNAPSHOT/api/mcp/mcp-stateless-server-boot-starter-docs.html#_stateless_mcp_servers

[olamy]

See https://docs.spring.io/spring-ai/reference/1.1-SNAPSHOT/api/mcp/mcp-stateless-server-boot-starter-docs.html#_stateless_mcp_servers

@quaff thanks. But I still struggle to understand how this could help me when using HttpServletStreamableServerTransportProvider (see the description of the issue)

[quaff]

I think the challenge is hard to serialize/deserialize McpSession like HttpSession.

[olamy]

I think the challenge is hard to serialize/deserialize McpSession like HttpSession.

Correct. But an option to try should be given.
At the end of the day, the "client" (which can be an AI agent) should be better at recovering from this and starting a new session.