# Add PyInstaller support for building standalone binaries ## Summary Add support for building standalone executable binaries of mcpgateway using PyInstaller. This will allow users to run mcpgateway without installing Python or managing dependencies. ## Motivation - **Simplified deployment**: Users can download and run a single executable - **No Python required**: Removes the need for Python installation on target systems - **Dependency isolation**: All dependencies are bundled, avoiding version conflicts - **Enterprise-friendly**: Easier to deploy in restricted environments ## Requirements ### 1. Makefile targets Add the following targets to the existing Makefile: - `make binary` - Build binary for current platform - `make binary-all` - Build binaries for all platforms (via Docker/CI) - `make binary-test` - Test the built binary - `make binary-clean` - Clean PyInstaller build artifacts ### 2. GitHub Actions workflow Create `.github/workflows/build-binaries.yml` that: - Triggers on version tags (`v*`) and manual dispatch - Builds binaries for Linux (x64), Windows (x64), and macOS (x64/arm64) - Uploads artifacts to GitHub releases - Tests each binary before upload ### 3. PyInstaller spec file Create `mcpgateway.spec` with proper configuration for: - Including all static assets (templates, static files, alembic migrations) - Hidden imports for FastAPI/Uvicorn/SQLAlchemy - Optional dependencies (Redis, PostgreSQL) - Proper executable naming per platform ## Implementation Details ### File Structure ``` mcp-context-forge/ ├── mcpgateway.spec # PyInstaller specification ├── scripts/ │ └── build_binary.py # Cross-platform build script ├── .github/workflows/ │ └── build-binaries.yml # GitHub Actions workflow └── Makefile # Updated with binary targets ``` ### Critical Data Files to Include - `mcpgateway/templates/` - Jinja2 templates - `mcpgateway/static/` - CSS/JS files - `mcpgateway/alembic.ini` - Alembic configuration - `mcpgateway/alembic/` - Migration scripts ### Hidden Imports Required ```python # Uvicorn 'uvicorn.logging', 'uvicorn.loops.auto', 'uvicorn.protocols.http.auto', 'uvicorn.protocols.websockets.auto', 'uvicorn.lifespan.on', # Database 'sqlalchemy.dialects.sqlite', 'sqlalchemy.dialects.postgresql', 'alembic', # Optional 'redis.asyncio', 'psycopg2', ``` ## Sample Implementation ### Makefile targets ```makefile # ============================================================================= # 📦 BINARY BUILDS (PyInstaller) # ============================================================================= # help: 📦 BINARY BUILDS # help: binary - Build standalone executable for current platform # help: binary-test - Test the built binary # help: binary-clean - Remove PyInstaller build artifacts # ============================================================================= BINARY_NAME = mcpgateway BINARY_DIST = dist/$(BINARY_NAME) ifeq ($(OS),Windows_NT) BINARY_DIST = dist/$(BINARY_NAME).exe endif .PHONY: binary binary-test binary-clean pyinstaller-install pyinstaller-install: ## Install PyInstaller @test -d "$(VENV_DIR)" || $(MAKE) venv @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ python3 -m pip install --quiet --upgrade pyinstaller" binary: pyinstaller-install ## Build standalone executable @echo "📦 Building standalone binary with PyInstaller..." @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ pyinstaller --clean --noconfirm mcpgateway.spec" @echo "✅ Binary built: $(BINARY_DIST)" @echo "📏 Size: $$(du -h $(BINARY_DIST) | cut -f1)" binary-test: ## Test the built binary @echo "🧪 Testing binary..." @test -f "$(BINARY_DIST)" || { echo "❌ Binary not found. Run 'make binary' first."; exit 1; } @echo "1️⃣ Version check:" @$(BINARY_DIST) --version @echo "" @echo "2️⃣ Help output:" @$(BINARY_DIST) --help | head -20 @echo "" @echo "✅ Binary tests passed!" binary-clean: ## Clean PyInstaller artifacts @echo "🧹 Cleaning PyInstaller build artifacts..." @rm -rf build/ dist/ *.spec __pycache__ @find . -name "*.pyc" -delete @echo "✅ PyInstaller artifacts cleaned" ``` ### PyInstaller spec file (`mcpgateway.spec`) ```python # -*- mode: python ; coding: utf-8 -*- import sys import os from pathlib import Path from PyInstaller.utils.hooks import collect_all, collect_data_files, collect_submodules block_cipher = None # Determine platform-specific binary name binary_name = 'mcpgateway' if sys.platform == 'win32': binary_name += '-windows-x64' elif sys.platform == 'darwin': binary_name += '-macos-x64' else: binary_name += '-linux-x64' # Collect mcpgateway data datas = [] binaries = [] hiddenimports = [] # Core package data datas += collect_data_files('mcpgateway', include_py_files=False) hiddenimports += collect_submodules('mcpgateway') # Explicitly add critical data files data_mappings = [ ('mcpgateway/templates', 'mcpgateway/templates'), ('mcpgateway/static', 'mcpgateway/static'), ('mcpgateway/alembic.ini', 'mcpgateway'), ('mcpgateway/alembic', 'mcpgateway/alembic'), ] for src, dst in data_mappings: if Path(src).exists(): datas.append((src, dst)) # FastAPI/Uvicorn hidden imports hiddenimports += [ # Uvicorn core 'uvicorn.logging', 'uvicorn.loops', 'uvicorn.loops.auto', 'uvicorn.protocols', 'uvicorn.protocols.http', 'uvicorn.protocols.http.auto', 'uvicorn.protocols.websockets', 'uvicorn.protocols.websockets.auto', 'uvicorn.lifespan', 'uvicorn.lifespan.on', 'uvicorn.workers', # HTTP/WebSocket 'httpx', 'httpcore', 'h11', 'websockets', 'watchfiles', # FastAPI ecosystem 'starlette', 'fastapi', 'pydantic', 'pydantic_settings', 'anyio', 'sniffio', 'click', 'python_multipart', # Database 'sqlalchemy.dialects.sqlite', 'sqlalchemy.dialects.postgresql', 'alembic', 'alembic.config', 'alembic.script', 'alembic.runtime.migration', # MCP and utilities 'mcp', 'jinja2', 'sse_starlette', 'jsonpath_ng', 'parse', 'filelock', 'zeroconf', 'cryptography', 'jwt', # Optional dependencies 'redis', 'redis.asyncio', 'psycopg2', 'psutil', ] # Exclude unnecessary modules to reduce size excludes = [ 'tkinter', 'matplotlib', 'numpy', 'scipy', 'pandas', 'PIL', 'notebook', 'IPython', ] a = Analysis( ['mcpgateway/cli.py'], pathex=[], binaries=binaries, datas=datas, hiddenimports=hiddenimports, hookspath=[], hooksconfig={}, runtime_hooks=[], excludes=excludes, win_no_prefer_redirects=False, win_private_assemblies=False, cipher=block_cipher, noarchive=False, ) pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher) exe = EXE( pyz, a.scripts, a.binaries, a.zipfiles, a.datas, [], name=binary_name, debug=False, bootloader_ignore_signals=False, strip=True, upx=True, upx_exclude=[], runtime_tmpdir=None, console=True, disable_windowed_traceback=False, argv_emulation=False, target_arch=None, codesign_identity=None, entitlements_file=None, icon=None, # Add icon path if available ) ``` ### GitHub Actions workflow (`.github/workflows/build-binaries.yml`) ```yaml name: Build Binaries on: push: tags: - 'v*' pull_request: paths: - 'mcpgateway/**' - 'pyproject.toml' - '.github/workflows/build-binaries.yml' - 'mcpgateway.spec' workflow_dispatch: inputs: upload_artifacts: description: 'Upload artifacts to release' required: false default: true type: boolean env: PYTHON_VERSION: '3.11' jobs: build: name: Build ${{ matrix.os }} runs-on: ${{ matrix.runs-on }} strategy: fail-fast: false matrix: include: - os: linux runs-on: ubuntu-latest binary_name: mcpgateway-linux-x64 - os: windows runs-on: windows-latest binary_name: mcpgateway-windows-x64.exe - os: macos runs-on: macos-latest binary_name: mcpgateway-macos-x64 steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: ${{ env.PYTHON_VERSION }} - name: Install dependencies run: | python -m pip install --upgrade pip pip install -e ".[dev]" pip install pyinstaller - name: Build with PyInstaller run: | pyinstaller --clean --noconfirm mcpgateway.spec - name: Rename binary shell: bash run: | if [[ "${{ matrix.os }}" == "windows" ]]; then mv dist/mcpgateway*.exe dist/${{ matrix.binary_name }} else mv dist/mcpgateway* dist/${{ matrix.binary_name }} chmod +x dist/${{ matrix.binary_name }} fi - name: Test binary shell: bash run: | echo "Testing binary version..." ./dist/${{ matrix.binary_name }} --version echo "Testing help output..." ./dist/${{ matrix.binary_name }} --help - name: Compress binary shell: bash run: | cd dist if [[ "${{ matrix.os }}" == "windows" ]]; then 7z a -tzip ${{ matrix.binary_name }}.zip ${{ matrix.binary_name }} else tar -czf ${{ matrix.binary_name }}.tar.gz ${{ matrix.binary_name }} fi - name: Upload artifact uses: actions/upload-artifact@v4 with: name: binary-${{ matrix.os }} path: | dist/${{ matrix.binary_name }}.zip dist/${{ matrix.binary_name }}.tar.gz retention-days: 7 release: needs: build runs-on: ubuntu-latest if: startsWith(github.ref, 'refs/tags/v') permissions: contents: write steps: - name: Download all artifacts uses: actions/download-artifact@v4 with: path: ./artifacts - name: List artifacts run: | echo "Downloaded artifacts:" find ./artifacts -type f -ls - name: Create Release uses: softprops/action-gh-release@v2 with: files: ./artifacts/**/* generate_release_notes: true draft: false prerelease: false env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` ## Testing Plan 1. **Local Testing**: - Build binary on each platform - Verify `--version` output matches package version - Test basic server startup - Verify static assets are served correctly - Test database migrations work 2. **CI Testing**: - Automated builds on PR - Smoke tests for each binary - Size checks (warn if >100MB) 3. **Release Testing**: - Manual testing of release artifacts - Installation on clean systems - Verify no Python required ## Success Criteria - [ ] Makefile targets work on Linux/macOS/Windows - [ ] GitHub Actions successfully builds all platforms - [ ] Binaries run without Python installed - [ ] Static assets (templates, CSS, JS) work correctly - [ ] Database migrations function properly - [ ] Binary size is reasonable (<100MB) - [ ] Binaries pass antivirus checks on Windows ## Related Issues - None currently ## Additional Notes - Consider code signing for macOS/Windows in future iterations - May want to add UPX compression toggle for size optimization - Could add ARM builds for Linux/macOS in the future - Consider stripping binaries, and using `upx` compression: `upx --best --lzma dist/mcpgateway` ## Tested on Linux: ``` pyinstaller --onefile \ --name mcpgateway \ --add-data "mcpgateway/templates:mcpgateway/templates" \ --add-data "mcpgateway/static:mcpgateway/static" \ --add-data "mcpgateway/alembic.ini:mcpgateway" \ --add-data "mcpgateway/alembic:mcpgateway/alembic" \ --hidden-import uvicorn.logging \ --hidden-import uvicorn.loops.auto \ --hidden-import uvicorn.protocols.http.auto \ --hidden-import uvicorn.protocols.websockets.auto \ --hidden-import uvicorn.lifespan.on \ --hidden-import sqlalchemy.dialects.sqlite \ --hidden-import alembic \ --collect-submodules mcpgateway \ --strip \ --exclude-module matplotlib \ --exclude-module numpy \ --exclude-module pandas \ --exclude-module scipy \ --exclude-module PIL \ --exclude-module cv2 \ --exclude-module tensorflow \ --exclude-module torch \ --exclude-module sklearn \ mcpgateway/cli.py upx --best --lzma dist/mcpgateway # 37 MB binary ```