Merge pull request #223 from deploystackio/feat/Streamable-http

Feat/streamable http

Author: Lasim

docs/development/gateway/index.mdx

@@ -14,21 +14,22 @@ The DeployStack Gateway is the local secure proxy that connects developers to th
 
 ## Architecture Overview
 
-The Gateway implements a sophisticated Control Plane / Data Plane architecture with dual transport support:
+The Gateway implements a sophisticated Control Plane / Data Plane architecture with comprehensive transport support:
 
 - **Control Plane**: Authenticates with `cloud.deploystack.io` to download team configurations and access policies
-- **Data Plane**: Manages local MCP server processes with both stdio and SSE transport protocols
+- **Data Plane**: Manages local MCP server processes with stdio, SSE, and Streamable HTTP transport protocols
 - **Security Layer**: Injects credentials securely into process environments without exposing them to developers
 - **Session Management**: Handles secure SSE connections with cryptographic session IDs for VS Code compatibility
+- **Transport Layer**: Supports both legacy SSE transport and modern Streamable HTTP transport for maximum client compatibility
 
 ## Core Features
 
 <Cards>
   <Card 
     icon={<Terminal />} 
-    title="Dual Transport Support"
+    title="Triple Transport Support"
   >
-    Supports both stdio transport for CLI tools and SSE transport for VS Code compatibility
+    Supports stdio transport for CLI tools, SSE transport for VS Code compatibility, and Streamable HTTP for modern MCP clients
   </Card>
 
   <Card 
@@ -118,9 +119,11 @@ Manages the lifecycle of MCP server processes, including:
 - Environment variable injection
 
 ### HTTP Proxy Server
-Exposes dual endpoints for different client types:
-- **GET /sse**: SSE connection establishment for VS Code
+Exposes multiple endpoints for different client types:
+- **GET /sse**: SSE connection establishment for VS Code and legacy clients
 - **POST /message**: Session-based JSON-RPC for SSE clients
+- **POST /mcp**: Streamable HTTP endpoint for modern MCP clients
+- **GET /health**: Health check endpoint for monitoring
 
 ### Session Manager
 Handles secure SSE connections with:
@@ -163,11 +166,16 @@ The Gateway works with MCP server configurations in this format:
 1. **Authentication**: Gateway authenticates with cloud control plane
 2. **Config Download**: Downloads team's MCP server configurations
 3. **Persistent Process Startup**: Starts all configured MCP servers as background processes when gateway launches
-4. **HTTP Server**: Starts local HTTP server with SSE endpoints immediately available (default: `localhost:9095/sse`)
-5. **Request Handling**: Receives MCP requests from development tools and routes to already-running processes
+4. **HTTP Server**: Starts local HTTP server with multiple endpoints immediately available:
+   - SSE endpoint: `localhost:9095/sse` (for VS Code and legacy clients)
+   - Messages endpoint: `localhost:9095/message` (for session-based JSON-RPC)
+   - MCP endpoint: `localhost:9095/mcp` (for modern Streamable HTTP clients)
+   - Health endpoint: `localhost:9095/health` (for monitoring)
+5. **Request Handling**: Receives MCP requests from development tools and intelligently routes to appropriate transport
 6. **Process Management**: Maintains persistent background processes as described in [Gateway Process Management](/development/gateway/process-management).
 7. **Credential Injection**: Securely injects environment variables into running processes at startup
 8. **Tool Routing**: Routes namespaced tool calls to persistent MCP servers via stdio transport
+9. **Transport Selection**: Automatically detects client capabilities and uses appropriate transport (SSE or Streamable HTTP)
 
 For detailed information about the caching system, see [Gateway Caching System](/development/gateway/caching-system).
 
@@ -207,10 +215,3 @@ The Gateway is actively under development. Key areas for contribution:
 - **Performance**: Optimizing stdio communication
 - **Platform Support**: Adding Windows/Linux compatibility
 - **Error Handling**: Robust error recovery
-
-## Roadmap
-
-- **Phase 2**: Enhanced process lifecycle management
-- **Phase 3**: Support for remote MCP servers (HTTP transport)
-- **Phase 4**: Advanced monitoring and analytics
-- **Future**: Plugin system for custom MCP server types

docs/development/gateway/process-management.mdx

@@ -62,6 +62,31 @@ The Gateway's process management system operates on a **persistent background pr
   </Card>
 </Cards>
 
+## Selective Restart Capability
+
+The Gateway supports **selective restart** functionality, allowing individual MCP servers to be managed without requiring a full gateway restart. This feature dramatically improves configuration update performance and eliminates downtime for unchanged servers.
+
+### Key Features
+
+- **Individual Server Control**: Add, remove, or restart specific MCP servers via HTTP API
+- **Change Detection**: Automatically detects added, removed, and modified server configurations
+- **Fallback Safety**: Falls back to full restart if selective operations fail
+- **Zero Downtime**: Unchanged servers continue running during configuration updates
+
+### API Endpoints
+
+The Gateway exposes HTTP endpoints for selective server management:
+
+- `POST /api/mcp/servers` - Add new MCP servers to running gateway
+- `DELETE /api/mcp/servers/:serverName` - Remove specific servers
+- `POST /api/mcp/servers/:serverName/restart` - Restart individual servers
+
+### Implementation Services
+
+- **Selective Restart Service**: Handles HTTP communication with running gateway processes
+- **Configuration Change Service**: Detects configuration differences and orchestrates selective operations
+- **Process Manager Integration**: Provides individual server lifecycle control capabilities
+
 ## Process Lifecycle
 
 ### Gateway Startup Phase

docs/development/gateway/session-management.mdx

@@ -1,6 +1,6 @@
 ---
 title: Session Management
-description: Cryptographically secure session lifecycle management for SSE connections
+description: Cryptographically secure session lifecycle management for SSE and Streamable HTTP connections
 sidebar: Session Management
 icon: Key
 ---
@@ -10,14 +10,16 @@ import { Key, Clock, Shield, Trash2 } from 'lucide-react';
 
 # Session Management
 
-The DeployStack Gateway implements a robust session management system that provides cryptographically secure session handling for persistent SSE connections while ensuring automatic cleanup and resource management.
+The DeployStack Gateway implements a robust session management system that provides cryptographically secure session handling for both persistent SSE connections and optional Streamable HTTP sessions while ensuring automatic cleanup and resource management.
 
 ## Architecture Overview
 
-The session management system consists of two primary components working together to provide secure, persistent connections:
+The session management system consists of multiple components working together to provide secure connections across different transport protocols:
 
 - **SessionManager**: Handles session lifecycle, validation, and SSE stream management
 - **SSEHandler**: Manages Server-Sent Events connections and message routing
+- **StreamableHTTPHandler**: Manages Streamable HTTP connections with optional session support
+- **Transport Layer**: Intelligent routing between SSE and Streamable HTTP based on client capabilities
 
 ## Core Components
 
@@ -78,14 +80,16 @@ private validateSessionId(sessionId: string): boolean {
 ## Session Lifecycle
 
 ### 1. Creation Phase
-**Trigger**: SSE connection establishment via `GET /sse`
+**Triggers**: 
+- SSE connection establishment via `GET /sse`
+- Optional session creation for Streamable HTTP via `POST /mcp` with session headers
 
 **Process:**
 1. Generate cryptographically secure session ID
 2. Create session object with metadata
-3. Associate with SSE stream
+3. Associate with SSE stream (for SSE transport) or track session state (for Streamable HTTP)
 4. Schedule automatic cleanup timer
-5. Send endpoint event to client
+5. Send endpoint event to client (SSE) or return session headers (Streamable HTTP)
 
 **Session Object:**
 ```typescript
@@ -219,6 +223,24 @@ getStatus() {
 - **Cleanup Events**: Cleanup reasons and timing logged
 - **Error Conditions**: Detailed error logging for troubleshooting
 
+## Transport-Specific Session Handling
+
+### SSE Transport Sessions
+SSE transport requires persistent sessions for connection management:
+
+- **Mandatory Sessions**: All SSE connections must have associated sessions
+- **Stream Binding**: Sessions are bound to specific SSE streams
+- **Real-time Communication**: Messages sent via SSE events in real-time
+- **Connection Lifecycle**: Session lifecycle tied to SSE connection state
+
+### Streamable HTTP Transport Sessions
+Streamable HTTP transport supports optional sessions for enhanced functionality:
+
+- **Optional Sessions**: Sessions can be used but are not required
+- **Stateless Operation**: Supports both stateless and session-based operation
+- **Header-Based**: Session IDs passed via `Mcp-Session-Id` header
+- **Flexible Lifecycle**: Sessions can span multiple HTTP requests
+
 ## Integration Points
 
 ### SSE Handler Integration
@@ -235,21 +257,44 @@ this.sseHandler.sendMessage(sessionId, response);
 this.sseHandler.sendError(sessionId, errorResponse);
 ```
 
-### HTTP Proxy Integration
-Session validation in the HTTP proxy:
+### Streamable HTTP Handler Integration
+The session manager provides optional session support for Streamable HTTP:
 
 ```typescript
-// Session validation on each request
-const session = this.sessionManager.getSession(sessionId);
-if (!session) {
-  // Handle invalid session
+// Optional session validation for Streamable HTTP
+const sessionId = request.headers['mcp-session-id'];
+if (sessionId) {
+  const session = this.sessionManager.getSession(sessionId);
+  if (session) {
+    this.sessionManager.updateActivity(sessionId);
+  }
 }
 
-// Activity tracking
-this.sessionManager.updateActivity(sessionId);
+// Stateless operation when no session provided
+if (!sessionId) {
+  // Handle request without session context
+}
+```
+
+### HTTP Proxy Integration
+Session validation across both transports in the HTTP proxy:
 
-// Error counting
-this.sessionManager.incrementErrorCount(sessionId);
+```typescript
+// Transport-aware session handling
+if (isSSETransport) {
+  // SSE requires session validation
+  const session = this.sessionManager.getSession(sessionId);
+  if (!session) {
+    throw new Error('Invalid session for SSE transport');
+  }
+  this.sessionManager.updateActivity(sessionId);
+} else if (isStreamableHTTP && sessionId) {
+  // Streamable HTTP optional session support
+  const session = this.sessionManager.getSession(sessionId);
+  if (session) {
+    this.sessionManager.updateActivity(sessionId);
+  }
+}
 ```
 
 ## Best Practices

docs/index.mdx

@@ -33,7 +33,7 @@ DeployStack is the **Enterprise Control Plane for the Model Context Protocol (MC
 
   <Card 
     icon={<Server />} 
-    href="/self-hosted" 
+    href="/self-hosted/quick-start"
     title="Self-Hosted DeployStack"
   >
     Host DeployStack on your own infrastructure for complete control and security

docs/self-hosted/docker-compose.mdx

@@ -26,26 +26,11 @@ This guide provides step-by-step instructions to install and configure DeploySta
 
 - **RAM**: Ensure your environment has at least 4GB of RAM. Insufficient memory can cause processes to crash.
 - **Docker & Docker Compose**: Make sure both are installed and up-to-date.
-- **Storage**: At least 20GB of available disk space for images and persistent data.
-- **Network**: Stable internet connection for pulling Docker images.
+- **Storage**: At least 2GB of available disk space for images and persistent data.
 
-## Option 1: One-line Setup
+## Beggining the setup for Docker Compose
 
-Deploy DeployStack with a single command:
-
-```bash
-curl -sSL https://raw.githubusercontent.com/deploystackio/deploystack/main/scripts/install.sh | bash
-```
-
-This script will:
-1. Download the docker-compose.yml file
-2. Generate a secure encryption secret
-3. Start all services
-4. Display access information
-
-## Option 2: Manual Setup
-
-Follow these steps for a manual setup with full control.
+Follow these steps for a setup with docker compsoe
 
 ### Step 1: Download Docker Compose File
 
@@ -258,54 +243,13 @@ docker-compose logs backend
 docker-compose logs frontend
 ```
 
-#### Port Conflicts
-
-If ports 3000 or 8080 are already in use:
-
-```bash
-# Update .env file
-FRONTEND_PORT=8081
-BACKEND_PORT=3001
-
-# Restart services
-docker-compose down
-docker-compose up -d
-```
-
-#### Permission Issues
-
-```bash
-# Fix volume permissions
-sudo chown -R $USER:$USER $(docker volume inspect deploystack_backend_persistent --format '{{ .Mountpoint }}')
-```
-
-#### Memory Issues
-
-```bash
-# Check available memory
-free -h
-
-# Monitor container resource usage
-docker stats
-```
-
 ### Getting Help
 
 If you encounter issues not covered here:
 
-1. Check the [Troubleshooting](/troubleshooting) guide
-2. Search existing [GitHub Issues](https://github.com/deploystackio/deploystack/issues)
-3. Join our [Discord community](https://discord.gg/UjFWwByB)
-4. Create a new issue with detailed logs and system information
-
-## Next Steps
-
-Once DeployStack is running:
-
-1. **Create your first workspace** through the web interface
-2. **Configure global settings** for email and authentication
-3. **Set up user roles** and team permissions
-4. **Deploy your first MCP server** from the catalog
+1. Search existing [GitHub Issues](https://github.com/deploystackio/deploystack/issues)
+2. Join our [Discord community](https://discord.gg/42Ce3S7b3b)
+3. Create a new issue with detailed logs and system information
 
 ---