ToolHive Buildkite Plugin

A Buildkite plugin that enables running Model Context Protocol (MCP) servers using ToolHive in your CI/CD pipelines.

Features

• Automatic ToolHive Installation: Downloads and installs ToolHive if not already available
• MCP Server Management: Start, manage, and clean up MCP servers during pipeline execution
• Multiple Server Sources: Support for registry servers, Docker images, and protocol schemes (uvx://, npx://, go://)
• Flexible Configuration: Customize transport methods, ports, volumes, secrets, and more

Usage

Add the plugin to your pipeline step:

steps:
  - label: "Run with MCP Server"
    command: "your-command-here"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"  # Server from ToolHive registry

Registry Server Example

steps:
  - label: "Use Fetch MCP Server"
    command: "curl http://localhost:8080/some-endpoint"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          transport: "stdio"
          proxy-port: 8080

Docker Image Example

steps:
  - label: "Use Custom MCP Server"
    command: "your-command"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "my-registry/my-mcp-server:latest"
          transport: "sse"
          volumes:
            - "/host/path:/container/path:ro"

Protocol Scheme Example

steps:
  - label: "Use Python MCP Server"
    command: "your-command"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "uvx://[email protected]"
          transport: "streamable-http"
          args:
            - "--verbose"
            - "--config=/path/to/config"

With Secrets

steps:
  - label: "Use GitHub MCP Server"
    command: "your-command"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "github"
          secrets:
            - name: "github-token"
              target: "GITHUB_PERSONAL_ACCESS_TOKEN"
            - name: "api-key"
              target: "API_KEY"

Configuration

Required

Option	Type	Description
server	String	The MCP server to run (registry name, Docker image, or protocol scheme)

Optional

Option	Type	Default	Description
name	String	Auto-generated	Custom name for the MCP server instance
transport	String	"" (auto)	Transport method: stdio, sse, or streamable-http
proxy-port	Integer	Random	Specific port for the ToolHive proxy
secrets	Array	[]	Secrets to pass to the MCP server
volumes	Array	[]	Volume mounts in format "host-path:container-path[:ro]"
args	Array	[]	Additional arguments to pass to the MCP server
permission-profile	String	Default	Permission profile for the MCP server
toolhive-version	String	Latest	Specific version of ToolHive to download
cleanup	Boolean	true	Whether to clean up the MCP server on exit
mcp-config-file	String	./mcp_servers.json	Path where to generate the MCP config file
mcp-config-cleanup	Boolean	true	Whether to remove the MCP config file on exit

Secrets Configuration

Secrets are configured as an array of objects with name and target properties:

secrets:
  - name: "secret-name-in-toolhive"
    target: "ENVIRONMENT_VARIABLE_NAME"

The name refers to a secret stored in ToolHive's secret management system, and target is the environment variable name that will be set in the MCP server container.

Note that secrets must be created in ToolHive before they can be used in the plugin.

Volume Configuration

Volumes are specified as strings in Docker volume format:

volumes:
  - "/host/path:/container/path"      # Read-write mount
  - "/host/path:/container/path:ro"   # Read-only mount

Server Types

Registry Servers

Use servers from the ToolHive registry:

# Fetch MCP server
server: "fetch"

# GitHub MCP server
server: "github"

# Filesystem MCP server
server: "filesystem"

Docker Images

Use any Docker image that implements the MCP protocol:

# Custom Docker image
server: "my-registry/my-mcp-server:v1.0.0"

# GitHub Container Registry image
server: "ghcr.io/org/mcp-server:latest"

Protocol Schemes

Use package managers to run MCP servers:

# Python via uv
server: "uvx://[email protected]"

# Node.js via npm
server: "npx://[email protected]"

# Go module
server: "go://github.com/org/go-mcp-server"

MCP Configuration File Generation

The plugin automatically generates an MCP configuration file that contains connection details for all spawned MCP servers. This makes it easy for MCP clients to discover and connect to available servers.

Configuration File Format

The generated file follows the standard MCP configuration format:

{
  "mcpServers": {
    "fetch-server": {
      "url": "http://localhost:8080/mcp",
      "type": "streamable-http"
    },
    "github-server": {
      "url": "http://localhost:8081/sse#github-server",
      "type": "sse"
    }
  }
}

URL Format Details

• SSE servers: http://localhost:{port}/sse#{server-name}
• Streamable HTTP servers: http://localhost:{port}/mcp
• Type: Either "sse" or "streamable-http"

Environment Variable

The plugin exports BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE pointing to the generated configuration file:

echo "MCP config file: $BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE"
cat $BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE

Usage with MCP Clients

steps:
  - command: |
      # Use the generated MCP configuration with your tools
      my-mcp-client --config $BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE
      
      # Or read the configuration programmatically
      python -c "
      import json
      import os
      config_file = os.environ['BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE']
      with open(config_file) as f:
          config = json.load(f)
          print('Available MCP servers:', list(config['mcpServers'].keys()))
      "
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          mcp-config-file: "./my_mcp_config.json"

How It Works

1. Environment Hook: Checks if ToolHive is available, downloads it if needed
2. Pre-Command Hook: Starts the specified MCP server with the given configuration
3. Command Execution: Your pipeline command runs with the MCP server available
4. Pre-Exit Hook: Stops and removes the MCP server (if cleanup is enabled)

Server Naming

The plugin automatically generates unique server names to avoid conflicts:

• Uses custom name if provided via the name option
• Otherwise generates: build-{BUILD_NUMBER}-step-{STEP_KEY}-{SERVER_NAME}
• Names are normalized (lowercase, special characters replaced with hyphens)

Requirements

• Docker or Podman container runtime
• Internet access to download ToolHive (if not already installed)
• Sufficient permissions to run containers

Troubleshooting

ToolHive Installation Issues

If ToolHive fails to install:

1. Check internet connectivity
2. Verify the GitHub releases are accessible
3. Ensure sufficient disk space
4. Check file permissions in the installation directory

MCP Server Startup Issues

If the MCP server fails to start:

1. Check the server logs: thv logs <server-name>
2. Verify the server configuration
3. Ensure required secrets are available
4. Check container runtime (Docker/Podman) status

Port Conflicts

If you encounter port conflicts:

1. Use the proxy-port option to specify a different port
2. Check for other services using the same port
3. Use dynamic port allocation (default behavior)

Examples

Complete Example with All Options

steps:
  - label: "Complex MCP Server Setup"
    command: |
      echo "MCP server is running"
      curl http://localhost:9000/health
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "my-registry/custom-mcp:v2.0.0"
          name: "my-custom-server"
          transport: "sse"
          proxy-port: 9000
          secrets:
            - name: "api-token"
              target: "API_TOKEN"
            - name: "db-password"
              target: "DATABASE_PASSWORD"
          volumes:
            - "./config:/app/config:ro"
            - "./data:/app/data"
          args:
            - "--log-level=debug"
            - "--config=/app/config/server.yml"
          permission-profile: "network"
          toolhive-version: "v0.0.33"
          cleanup: true
          mcp-config-file: "./custom_mcp_config.json"
          mcp-config-cleanup: false

Multiple Steps with Different Servers

steps:
  - label: "Step 1: Use Fetch Server"
    command: "test-fetch-functionality"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          
  - label: "Step 2: Use GitHub Server"
    command: "test-github-integration"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "github"
          secrets:
            - name: "github-token"
              target: "GITHUB_PERSONAL_ACCESS_TOKEN"

Multiple MCP Servers in One Step

You can run multiple MCP servers in a single step by calling the plugin multiple times:

steps:
  - label: "Use Multiple MCP Servers"
    command: |
      echo "Both servers are now running"
      curl http://localhost:8080/fetch-endpoint
      curl http://localhost:8081/github-endpoint
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          name: "fetch-server"
          proxy-port: 8080
      - StacklokLabs/toolhive#v0.0.2:
          server: "github"
          name: "github-server"
          proxy-port: 8081
          secrets:
            - name: "github-token"
              target: "GITHUB_PERSONAL_ACCESS_TOKEN"

Important Notes for Multiple Servers:

• Each server must have a unique name to avoid conflicts
• Each server should use a different proxy-port if specified
• All servers will be automatically cleaned up at the end of the step
• Servers are started in the order they appear in the plugin list

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test the plugin
5. Submit a pull request

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Links

• ToolHive Documentation
• Model Context Protocol
• Buildkite Plugin Documentation
• GitHub Repository