Skip to content

Data Protection Key Mismatch in Multi-Instance Deployments #829

@bluedog13

Description

@bluedog13

Bug Report: Data Protection Key Mismatch in Multi-Instance Deployments

Summary

When deploying ModelContextProtocol.AspNetCore with multiple replicas (e.g., Azure Container Apps, Docker Compose with load balancing), the application throws CryptographicException due to Data Protection keys not being shared across instances.

Environment

  • Package: ModelContextProtocol.AspNetCore v0.4.0-preview.1
  • Framework: .NET 10.0
  • Deployment: Azure Container Apps with 2+ replicas (also reproducible with Docker Compose + nginx)
  • Configuration: WithHttpTransport(t => t.Stateless = true)

Expected Behavior

When using Stateless = true, the MCP server should work seamlessly across multiple container instances without requiring shared state or synchronized encryption keys.

Actual Behavior

Requests fail with CryptographicException when load-balanced across different container instances:

System.Security.Cryptography.CryptographicException: The key {64967007-59c8-4f17-8cc6-04813b27f26e} was not found in the key ring.
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.DataProtection.DataProtectionCommonExtensions.Unprotect(IDataProtector protector, String protectedData)
   at ModelContextProtocol.AspNetCore.StreamableHttpHandler.GetSessionAsync(HttpContext context, String sessionId)

Root Cause

StreamableHttpHandler.cs uses ASP.NET Core Data Protection to encrypt session IDs, even in stateless mode:

  • Line 33: Creates IDataProtector with purpose "Microsoft.AspNetCore.StreamableHttpHandler.StatelessSessionId"
  • Line 227: Calls Protector.Unprotect(sessionId) - fails when session was created by different instance
  • Line 305: Calls Protector.Protect(sessionJson) - creates encrypted session ID

Source code reference

Reproduction Steps

  1. Deploy MCP server with 2+ replicas using nginx/Azure Container Apps load balancing
  2. Configure sticky sessions in Azure Container Apps: stickySessions: { affinity: 'sticky' }
  3. Make initial MCP request → routed to Instance A
  4. Make subsequent request with same session → may route to Instance B
  5. Instance B tries to decrypt session ID created by Instance A → CryptographicException

Impact

  • Blocks production deployments requiring high availability/scalability
  • Forces single-instance deployment, eliminating redundancy and scaling capabilities
  • Occurs despite Stateless = true configuration, which implies no shared state should be required

Proposed Solutions

Option 1: Make Data Protection Optional (Preferred)

Add configuration to disable session ID encryption when deploying to trusted environments:

.WithHttpTransport(t => {
    t.Stateless = true;
    t.EncryptSessionIds = false; // New option
})

Option 2: Use Signed JWTs Instead of Encrypted Session IDs

Replace Data Protection with JWT signing (using shared key from configuration):

  • Session IDs become self-contained tokens
  • No need for synchronized key rings
  • Can validate across all instances with single shared secret

Option 3: Document Data Protection Configuration Requirements

If encryption is mandatory, update documentation to require:

  • Azure Blob Storage configuration for shared keys in multi-instance deployments
  • Example configuration for common deployment scenarios

Workarounds

Workaround 1: Single Replica Deployment

Set minReplicas: 1 and maxReplicas: 1 in infrastructure
Drawback: No high availability or scaling

Additional Context

  • Azure Container Apps sticky sessions are cookie-based, but MCP protocol may not properly maintain these cookies
  • Even with sticky sessions configured, session routing can still fail
  • This issue affects any multi-instance ASP.NET Core deployment (Kubernetes, AKS, Azure Container Apps, Docker Swarm, etc.)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions