closes #4 Refactor codebase into a modular structure with backward compatibility

.github/workflows/generate-and-test.yml

@@ -17,13 +17,17 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: 3.9
+          python-version: "3.10"
 
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
           pip install -r requirements.txt
 
+      - name: Install package (editable mode)
+        run: |
+          pip install -e .
+
       - name: Run generator script
         run: |
           mkdir -p tests/out/
@@ -33,6 +37,16 @@ jobs:
             --api-url http://localhost:8000/api \
             --api-token "test-token"
 
+      - name: Run generator via CLI tool
+        run: |
+          mkdir -p tests/out_cli/
+          mcp-generator tests/openapi.yaml --output-dir ./tests/out_cli/ --api-url http://localhost:8000/api --api-token "test-token"
+
+      - name: Run generator via Python module
+        run: |
+          mkdir -p tests/out_module/
+          python -m openapi_mcp_generator.cli tests/openapi.yaml --output-dir ./tests/out_module/ --api-url http://localhost:8000/api --api-token "test-token"
+
       - name: Verify output directory exists
         run: |
           ls ./tests/out/ | grep "openapi-mcp-reference-test-api-"
@@ -44,3 +58,11 @@ jobs:
           grep -q "async def getItems(id: int, verbose: bool, limit: int, ctx: Context) -> str:" ./tests/out/$GENERATED_DIR/mcp_server.py
           grep -q "def get_BadRequestDetails_schema" ./tests/out/$GENERATED_DIR/mcp_server.py
         shell: bash
+
+      - name: Install pytest
+        run: |
+          pip install pytest
+
+      - name: Run tests on generated server output
+        run: |
+          pytest tests/test_generated_server.py

.gitignore

@@ -1,2 +1,5 @@
 project-*.md
 .venv
+build/
+__pycache__/
+*.egg-info/

MANIFEST.in

@@ -0,0 +1,4 @@
+include LICENSE
+include README.md
+include requirements.txt
+recursive-include templates *

MODULAR_REFACTORING.md

@@ -0,0 +1,60 @@
+# Summary of Modular Refactoring
+
+## Overview
+
+We've refactored the original `generator.py` into a proper Python package structure while ensuring backward compatibility. This makes the code more maintainable and easier to use as a library.
+
+## Key Aspects of the Modular Design
+
+1. **Preserved Original Script**: The original `generator.py` script still works exactly as before.
+
+2. **Same Templates Directory**: The refactored code uses the exact same `/templates` directory, with no changes needed.
+
+3. **Backward Compatible**: Users can continue using the script as they always have.
+
+4. **Package Structure**: Added proper Python package structure for better maintainability.
+
+5. **Docker Support**: All Docker-related functionality is preserved without changes.
+
+## File Structure and Relations
+
+```
+generator.py              # Original entry point (unchanged functionality)
+mcp_generator.py          # Alternative entry point using modular code
+openapi_mcp_generator/    # Package containing modular components
+    __init__.py           # Package initialization 
+    cli.py                # Command-line interface
+    generator.py          # Main generator module
+    generators.py         # Code generators
+    http.py               # HTTP client utilities
+    parser.py             # OpenAPI parser
+    project.py            # Project builder (uses templates/)
+templates/                # Original templates directory (unchanged)
+    ...
+```
+
+## How the Modular Code Uses the Original Templates
+
+The `ProjectBuilder` class in `project.py` initializes with the path to the templates directory:
+
+```python
+# In generator.py in the modular package
+TEMPLATE_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "templates")
+```
+
+This ensures that the modular code uses the exact same templates as the original script.
+
+## Advantages of This Approach
+
+1. **Better Organization**: Each component has a single responsibility
+2. **Testability**: Components can be tested independently
+3. **Reusability**: Functions can be imported and used programmatically
+4. **Packageability**: Can be installed as a proper Python package
+
+## Using the Modular Code
+
+You can use the refactored code in multiple ways:
+
+1. Continue using `generator.py` exactly as before
+2. Install as a package and use the `mcp-generator` command
+3. Import functions for use in your own Python code

README.md

@@ -9,10 +9,24 @@ A Python tool that automatically converts OpenAPI specifications into fully func
 - 🔐 Multiple authentication methods
 - ⚡ Async operations & rate limiting
 - 📡 SSE/IO communication protocols
+- 📦 Modular code structure with package support
 
-## Overview
+## Modular Code Structure
 
-This generator creates a fully functional MCP server implementation that exposes API operations defined in an OpenAPI specification as MCP tools and resources. The generated server is packaged with Docker for easy deployment and supports both SSE and IO communication protocols.
+The generator has been refactored into a proper Python package while maintaining backward compatibility:
+
+1. **Original Entry Point**: The `generator.py` script still works exactly as before
+2. **Modular Organization**: Code is now split into focused modules
+3. **Package Installation**: Can be installed as a proper Python package
+4. **Same Templates**: Uses the exact same templates in the `/templates` directory
+5. **Docker Support**: Preserves all Docker functionality
+
+You can use the tool in whatever way you prefer:
+1. Run `generator.py` directly (original approach)
+2. Install as a package and use `mcp-generator` command
+3. Use the module programmatically in your own Python code
+
+For more details on the modular code structure, see [MODULAR_REFACTORING.md](MODULAR_REFACTORING.md).
 
 ## Features
 
@@ -29,25 +43,43 @@ This generator creates a fully functional MCP server implementation that exposes
 
 - Python 3.10+
 - Docker (for running the generated server)
-- uv (Python package manager)
+- pip or uv (Python package manager)
 
 ## Installation
 
+### From Source
+
 ```bash
 # Clone the repository
 git clone https://github.com/abutbul/openapi-mcp-generator.git
 cd openapi-mcp-generator
 
-# Install dependencies using uv
+# Install as a package (development mode)
+pip install -e .
+
+# Or using uv
 uv venv
 source .venv/bin/activate
-uv pip install -r requirements.txt
+uv pip install -e .
+```
+
+### Using pip (once published)
+
+```bash
+pip install openapi-mcp-generator
 ```
 
 ## Usage
 
 ```bash
+# Using the original script (still works the same way)
 python generator.py openapi.yaml --output-dir ./output --api-url https://api.example.com
+
+# Using the new modular CLI tool after installation
+mcp-generator openapi.yaml --output-dir ./output --api-url https://api.example.com
+
+# Using the python module directly
+python -m openapi_mcp_generator.cli openapi.yaml --output-dir ./output
 ```
 
 ### Command Line Options
@@ -81,14 +113,44 @@ The generated `docker.sh` script supports the following commands:
   - `--log-level=LEVEL`: Set logging level (default: info)
 - `stop`: Stop the container
 - `clean`: Remove the container and image
-- `test`: Run the test suite (TBD)
 - `logs`: View container logs
 
-## Documentation
+## Project Structure
+
+The modular generator has the following structure:
 
-For more detailed information, see:
+```
+openapi-mcp-generator/
+├── generator.py              # Original entry point (maintained for backward compatibility)
+├── mcp_generator.py          # New entry point (uses the modular structure)
+├── openapi_mcp_generator/    # Main package (new modular structure)
+│   ├── __init__.py           # Package initialization
+│   ├── cli.py                # Command-line interface
+│   ├── generator.py          # Main generator module
+│   ├── generators.py         # Code generators for tools/resources
+│   ├── http.py               # HTTP client utilities
+│   ├── parser.py             # OpenAPI parser
+│   └── project.py            # Project builder
+├── templates/                # Original templates directory (used by the modular code)
+│   ├── config/
+│   ├── docker/
+│   ├── server/
+│   ├── pyproject.toml
+│   └── requirements.txt
+├── samples/                  # Sample implementations
+├── tests/                    # Test cases
+├── LICENSE                   # License file
+├── README.md                 # This file
+├── pyproject.toml            # Project metadata
+└── setup.py                  # Package setup
+```
 
-TBD
+The modular structure preserves all the existing functionality while making the code more maintainable:
+
+1. The original entry point (`generator.py`) can still be used as before
+2. The existing templates in `/templates` are used by the new modular code
+3. All Docker-related functionality is preserved exactly as it was
+4. The project can now be installed as a proper Python package
 
 ## Sample Implementations
 
@@ -104,7 +166,42 @@ Check out our sample implementations to see the generator in action:
 4. Run the test suite
 5. Submit a pull request
 
-
 ## License
 
-This project is licensed under the MIT License - see the (./LICENSE) file for details.
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## Usage Examples
+
+### 1. Using the library without installing (direct from source)
+
+To generate an MCP server from the Elasticsearch 6.1 spec folder:
+
+```bash
+# Run the original script directly
+python generator.py samples/elasticsearch_6.1/api
+
+# Or use the modular entry point
+python mcp_generator.py samples/elasticsearch_6.1/api
+```
+
+### 2. Using the library after installing with pip
+
+First, install the package (from the project root):
+
+```bash
+pip install .
+```
+
+Then use the CLI tool to convert the Trilium ETAPI spec:
+
+```bash
+mcp-generator samples/TriliumNext/etapi.openapi.yaml
+```
+
+Or use it programmatically in your own Python code:
+
+```python
+from openapi_mcp_generator import generator
+
+generator.generate('samples/TriliumNext/etapi.openapi.yaml')
+```

examples/compatibility_example.py

@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+"""
+Compatibility example showing how to use both the original and modular approaches.
+
+This script demonstrates that both the original and modular code can be used 
+interchangeably with the same templates and functionality.
+"""
+
+import os
+import sys
+from pathlib import Path
+
+# Get the project root directory
+project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+# Example OpenAPI file path - use a sample from the samples directory
+sample_openapi_file = os.path.join(project_root, "tests", "openapi.yaml")
+
+# Create output directories
+original_output_dir = os.path.join(project_root, "output", "original")
+modular_output_dir = os.path.join(project_root, "output", "modular")
+
+os.makedirs(original_output_dir, exist_ok=True)
+os.makedirs(modular_output_dir, exist_ok=True)
+
+# Method 1: Use the original generator.py directly
+print("Method 1: Using the original generator.py directly")
+print("=" * 60)
+original_command = f"python {os.path.join(project_root, 'generator.py')} {sample_openapi_file} --output-dir={original_output_dir} --api-url=http://localhost:8000"
+print(f"Running: {original_command}")
+print("-" * 60)
+os.system(original_command)
+print("\n")
+
+# Method 2: Import and use the modular generator
+print("Method 2: Using the modular package programmatically")
+print("=" * 60)
+sys.path.insert(0, project_root)
+from openapi_mcp_generator.generator import generate_mcp_server
+
+print("Calling generate_mcp_server function from the modular package...")
+project_dir = generate_mcp_server(
+    sample_openapi_file,
+    modular_output_dir,
+    api_url="http://localhost:8000",
+    auth_type="bearer",
+    api_token="your-token"
+)
+print(f"MCP server generated successfully in: {project_dir}")
+print(f"To build and run the Docker container:")
+print(f"  cd {project_dir}")
+print(f"  ./docker.sh build")
+print(f"  ./docker.sh start --transport=sse")
+print("\n")
+
+# Method 3: Use the CLI script if installed
+print("Method 3: Using the CLI tool (if installed with pip install -e .)")
+print("=" * 60)
+print("This would normally be run as:")
+print(f"mcp-generator {sample_openapi_file} --output-dir={modular_output_dir} --api-url=http://localhost:8000")
+print("Or as:")
+print(f"python -m openapi_mcp_generator.cli {sample_openapi_file} --output-dir={modular_output_dir} --api-url=http://localhost:8000")
+print("\n")
+
+print("Summary")
+print("=" * 60)
+print("All methods use the exact same templates in the /templates directory")
+print("All methods generate Docker-ready MCP servers with identical functionality")
+print("All the core features remain the same regardless of which approach you use")

examples/programmatic_usage.py

@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+"""
+Example of using the openapi_mcp_generator package programmatically.
+
+This script shows how to use the modular package in your own Python code.
+"""
+
+from openapi_mcp_generator import generate_mcp_server, parse_openapi_spec
+
+def main():
+    """Example of using the generator programmatically."""
+    # Generate an MCP server for an OpenAPI spec
+    project_dir = generate_mcp_server(
+        openapi_file="path/to/openapi.yaml",
+        output_dir="./output",
+        api_url="https://api.example.com",
+        auth_type="bearer",
+        api_token="your-token"
+    )
+
+    print(f"Server generated at: {project_dir}")
+
+    # Example of parsing an OpenAPI spec
+    spec = parse_openapi_spec("path/to/openapi.yaml")
+    print(f"API title: {spec.get('info', {}).get('title', 'Unknown')}")
+
+if __name__ == "__main__":
+    main()