Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
env/
venv/
.venv/
pip-log.txt
pip-delete-this-directory.txt
.mypy_cache/
.coverage
.pytest_cache/
*.egg-info/
docs/
assets/
workflows/
# README.md - needed for Docker build
# *.md - some markdown files needed for build
.git
.gitignore
*.log
tmp/
Dockerfile*
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ config.json
*config*.json
m3_pipeline.json

# Generated Docker files (use workflow to generate)
Dockerfile*
!src/MCPStack/core/docker/dockerfile_generator.py

# Operating System specific files
.DS_Store
Thumbs.db
Expand All @@ -80,4 +84,4 @@ Desktop.ini

# Datasets and other large files
data/
m3_data/
m3_data/
116 changes: 116 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Development Commands

### Core Development
- **Install dependencies**: `uv sync` (or `pip install -e .`)
- **Run tests**: `pytest` or `uv run pytest`
- **Run tests with coverage**: `pytest --cov=src/MCPStack --cov-report=html`
- **Lint code**: `ruff check` (or `uv run ruff check`)
- **Format code**: `ruff format` (or `uv run ruff format`)
- **Pre-commit hooks**: `uv run pre-commit install` then `uv run pre-commit run --all-files`

### CLI Usage
- **MCPStack CLI**: `uv run mcpstack --help` (main CLI interface)
- **List available tools**: `uv run mcpstack list-tools`
- **List presets**: `uv run mcpstack list-presets`
- **Build pipeline**: `uv run mcpstack build --pipeline <path> --config-type <type>`
- **Tool configuration**: `uv run mcpstack tools <tool_name> configure`

### Documentation
- **Build docs**: `./build_docs.sh`
- **Serve docs locally**: `mkdocs serve` (after installing dev dependencies)

## Architecture Overview

MCPStack is a scikit-learn-inspired pipeline orchestrator for Model Context Protocol (MCP) tools that enables stacking and composing MCP tools into pipelines for LLM environments.

### Core Components

#### 1. MCPStackCore (`src/MCPStack/stack.py`)
The main orchestrator class that provides a fluent API for building pipelines:
- **Chaining methods**: `with_tool()`, `with_tools()`, `with_preset()`, `with_config()`
- **Pipeline lifecycle**: `build()` β†’ `run()` β†’ `save()`/`load()`
- **Immutable design**: All `with_*` methods return new instances
- **Validation**: Tools and environment variables validated before initialization

#### 2. Configuration System (`src/MCPStack/core/config.py`)
- **StackConfig**: Central configuration container for logging, env vars, and paths
- **Environment management**: Merges and validates env vars across tools
- **Path resolution**: Auto-detects project root and data directories
- **Tool validation**: Ensures required env vars are present before tool initialization

#### 3. Tool System (`src/MCPStack/core/tool/`)
- **BaseTool**: Abstract base class for all MCP tools with lifecycle methods
- **Registry**: Auto-discovery via `[project.entry-points."mcpstack.tools"]`
- **CLI Integration**: BaseToolCLI for tool-specific command interfaces
- **Lifecycle**: `initialize()` β†’ `actions()` β†’ `post_load()` β†’ `teardown()`

#### 4. Preset System (`src/MCPStack/core/preset/`)
- **Base preset class**: Pre-configured tool combinations for common use cases
- **Registry**: Auto-discovery via entry points
- **Factory pattern**: Create method returns configured MCPStackCore instances

#### 5. MCP Config Generators (`src/MCPStack/core/mcp_config_generator/`)
- **Multiple backends**: FastMCP, Claude Desktop, Universal
- **Host integration**: Generate appropriate configs for different MCP hosts
- **Pluggable**: Registry-based system for adding new config generators

### Key Design Patterns

#### Pipeline Composition
```python
stack = (
MCPStackCore(StackConfig())
.with_tool(ToolA())
.with_tool(ToolB())
.build(type="fastmcp")
)
```

#### Tool Registration
Tools auto-register via pyproject.toml entry points:
```toml
[project.entry-points."mcpstack.tools"]
hello_world = "MCPStack.tools.hello_world:HelloWorldTool"
```

#### Validation First
- Environment variables validated before any tool initialization
- Tools validated against StackConfig requirements
- Fail-fast approach prevents partial initialization

### Development Guidelines

#### Adding New Tools
1. Inherit from `BaseTool` in `src/MCPStack/core/tool/base.py`
2. Implement required methods: `initialize()`, `actions()`
3. Add entry point in pyproject.toml
4. Follow lifecycle: validate β†’ initialize β†’ register β†’ teardown

#### Tool CLI Integration
1. Inherit from `BaseToolCLI` in `src/MCPStack/core/tool/cli/base.py`
2. Register CLI app with tool registry
3. Use `@click.command()` decorators for subcommands

#### Testing
- Test files mirror src structure: `tests/core/`, `tests/tools/`
- Use pytest with async support enabled
- Mock external dependencies and MCP server interactions
- Test both success and error paths

### File Structure Context
- `src/MCPStack/`: Main package with core orchestration
- `src/MCPStack/core/`: Core abstractions (tools, presets, config)
- `src/MCPStack/tools/`: Built-in tool implementations
- `tests/`: Test suite mirroring source structure
- `docs/`: MkDocs documentation source

### Important Notes
- Uses `beartype` for runtime type checking throughout
- Async support via pytest-asyncio for MCP operations
- Rich CLI with typer for enhanced user experience
- FastMCP integration for actual MCP server functionality
- UV preferred for dependency management, but pip also supported
81 changes: 81 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,87 @@ You can also run your pipeline with FastMCP, allowing you to connect to various

<br />

### `Docker Containerization` 🐳

Build and deploy your MCP pipelines as Docker containers for production environments.

#### Prerequisites
- **MCPStack**: `uv add mcpstack` or `pip install mcpstack`
- **Available Presets**: Check with `mcpstack list-presets` (assumes `example_preset` exists)
- **For full testing**: [Claude Desktop](https://claude.ai/desktop) for MCP server usage
- **For building images**: Docker CLI installed and available

#### Quick Start Commands

**Basic Config (Fastest setup)** ⭐ *Recommended for first-time users*
```bash
# Generate Docker config only - creates claude_desktop_config.json
mcpstack build --config-type docker --presets example_preset
```
*Outputs:* `claude_desktop_config.json` for MCP host configuration

**Development Workflow** πŸ› οΈ *Recommended for local development*
```bash
# Generate config, dockerfile, and build image locally
mcpstack build --config-type docker --presets example_preset --profile build-only
```
*Outputs:* `claude_desktop_config.json` + `Dockerfile` + built image for local testing

**Production Pipeline** πŸš€ *Recommended for deployment*
```bash
# Complete workflow: generate, build, and push container image
mcpstack build --config-type docker --presets example_preset --profile build-and-push
```
*Outputs:* Complete CI/CD pipeline with config, dockerfile, built & pushed container image

#### Verification Steps
```bash
# Check available profiles
mcpstack list-profiles --config-type docker

# After running any command, verify outputs:
ls -la claude_desktop_config.json Dockerfile # Check generated files
docker images | grep mcpstack # Check built images
cat claude_desktop_config.json # Config for Claude Desktop
```

Perfect for deploying MCP tools to Kubernetes, Docker Swarm, or any container orchestration system.

<br />

### `Workflow Profiles` βš™οΈ

Define and run complex multi-stage workflows beyond basic config generation using workflow profiles.

#### Discover Available Profiles
```bash
# List all profiles
mcpstack list-profiles

# List profiles for specific config type
mcpstack list-profiles --config-type docker
```

#### Use Built-in Profiles
```bash
# Docker build and push workflow
mcpstack build --config-type docker --presets example_preset --profile build-and-push

# Docker build-only workflow (local development)
mcpstack build --config-type docker --presets example_preset --profile build-only
```

#### Custom Profile Examples
Check the `workflows/` directory for YAML examples:
- `docker-dev.yaml` - Development workflow with custom Dockerfile
- `docker-prod.yaml` - Production CI/CD pipeline with tagging and registry push

Profiles support environment variables like `${GIT_COMMIT}`, `${GIT_BRANCH}`, and `${preset}` for dynamic naming.

Perfect for implementing complex deployment pipelines while maintaining MCPStack's clean architecture.

<br />

<img src="assets/readme/more.gif" width="61.8%" align="left" style="border-radius: 10px;"/>

### `Many Other CLIs Options`
Expand Down
52 changes: 52 additions & 0 deletions docs/API/cli/stack_cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,58 @@
This page documents the **programmatic** CLI class β€” handy if you embed the CLI or generate help programmatically.
For end-user command help, see the **CLI** section of the docs.

## Workflow Profiles

The `build` command supports workflow profiles for executing multi-stage operations beyond basic config generation:

- `--profile NAME`: Run extended workflow profile after basic build

### Profile Discovery

```bash
# List all available profiles
mcpstack list-profiles

# List profiles for specific config type
mcpstack list-profiles --config-type docker
```

### Profile Examples

```bash
# Generate Docker config only (basic usage)
mcpstack build --config-type docker --presets example_preset

# Execute build-and-push workflow profile
mcpstack build --config-type docker --presets example_preset --profile build-and-push

# Execute build-only workflow profile
mcpstack build --config-type docker --presets example_preset --profile build-only
```

### Built-in Docker Profiles

- `build-and-push`: Generate config, dockerfile, build image, push to registry
- `build-only`: Generate config, dockerfile, build image locally

### Custom Profiles

Create YAML profiles in the `workflows/` directory for complex deployment scenarios:

```yaml
name: docker-prod
description: Production deployment with tagging
config_type: docker
stages:
- config.generate
- dockerfile.generate
- image.build:
image: "${preset}:${GIT_COMMIT}"
- image.push:
registry: "docker.io"
tags: ["latest", "${GIT_BRANCH}"]
```

::: MCPStack.cli.StackCLI
options:
show_root_heading: true
Expand Down
82 changes: 82 additions & 0 deletions docs/API/mcp_config_generators/docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# DockerMCP Config Generator

::: MCPStack.core.mcp_config_generator.mcp_config_generators.docker_mcp_config.DockerMCPConfigGenerator
options:
show_root_heading: true
show_source: true

## Docker Workflow

The DockerMCP Config Generator provides unified Docker containerization functionality for MCPStack pipelines. It can:

- **Generate configuration files** for running MCPStack tools inside Docker containers (Claude Desktop integration)
- **Generate Dockerfiles** from MCPStack pipeline configurations
- **Build Docker images** automatically during configuration generation
- **Push images** to Docker registries
- **Support complex configuration** including volumes, ports, networks, and build arguments

## Usage

### Basic Configuration Generation
```python
config = DockerMCPConfigGenerator.generate(
stack=mcp_stack,
image_name="mcpstack:myapp"
)
```

### Complete Docker Workflow
```python
config = DockerMCPConfigGenerator.generate(
stack=mcp_stack,
image_name="myapp:latest",
build_image="myapp:latest", # Build the image
generate_dockerfile=True, # Generate Dockerfile
docker_push=True, # Push to registry
docker_registry_url="docker.io/username/"
)
```

## CLI Integration

The generator integrates with the unified MCPStack CLI through workflow profiles:

```bash
# Generate Docker config only (basic usage)
mcpstack build --config-type docker --presets my_preset

# Generate config, dockerfile, build and push (production pipeline)
mcpstack build --config-type docker --presets my_preset --profile build-and-push

# Generate config, dockerfile, and build locally (development)
mcpstack build --config-type docker --presets my_preset --profile build-only
```

## Parameters

### Core Parameters
- `stack`: MCPStackCore instance
- `image_name`: Docker image name to use in configurations
- `server_name`: Name for the MCP server in Claude config
- `volumes`: Volume mounts for Docker container
- `ports`: Port mappings for Docker container
- `network`: Docker network name

### Docker Building Parameters
- `build_image`: Image name to build (triggers Docker build)
- `generate_dockerfile`: Generate Dockerfile when True
- `dockerfile_path`: Custom path for generated Dockerfile
- `docker_push`: Push built image to registry
- `docker_registry_url`: Docker registry URL
- `build_args`: Dictionary of Docker build arguments

## Integration with MCP Config Generators Registry

The DockerMCP Config Generator is automatically available through the MCP config generators registry:

```python
from MCPStack.core.mcp_config_generator.registry import ALL_MCP_CONFIG_GENERATORS

# The DockerMCP generator is registered as 'docker'
generator = ALL_MCP_CONFIG_GENERATORS['docker']
config = generator.generate(stack, build_image="myimage:latest")
Loading