|
| 1 | +--- |
| 2 | +title: HashiCorp Vault integration |
| 3 | +description: |
| 4 | + Learn how to securely manage MCP server secrets using HashiCorp Vault with |
| 5 | + ToolHive Kubernetes Operator. |
| 6 | +--- |
| 7 | + |
| 8 | +This tutorial shows how to integrate HashiCorp Vault with the ToolHive |
| 9 | +Kubernetes Operator to securely manage secrets for your MCP servers. Using |
| 10 | +Vault's Agent Injector, you can automatically provision secrets into MCP server |
| 11 | +pods without exposing sensitive data in your Kubernetes manifests. |
| 12 | + |
| 13 | +As an example, we'll be deploying a GitHub MCP server. |
| 14 | + |
| 15 | +:::info[Prerequisites] |
| 16 | + |
| 17 | +Before starting this tutorial, ensure you have: |
| 18 | + |
| 19 | +- A Kubernetes cluster with the ToolHive Operator installed |
| 20 | +- kubectl configured to access your cluster |
| 21 | +- Helm 3.x installed |
| 22 | +- Basic familiarity with HashiCorp Vault concepts |
| 23 | +- A GitHub Personal Access Token (PAT) |
| 24 | + |
| 25 | +If you need help installing the ToolHive Operator, see the |
| 26 | +[Kubernetes quickstart guide](./quickstart-k8s.mdx). |
| 27 | + |
| 28 | +::: |
| 29 | + |
| 30 | +## Overview |
| 31 | + |
| 32 | +The integration works by using HashiCorp Vault's Agent Injector to automatically |
| 33 | +inject secrets into MCP server pods. When you add specific annotations to your |
| 34 | +MCPServer resource, the Vault Agent Injector: |
| 35 | + |
| 36 | +1. Detects the annotations and injects a Vault Agent sidecar |
| 37 | +2. Authenticates with Vault using Kubernetes service account tokens of the |
| 38 | + `proxyrunner` pod |
| 39 | +3. Retrieves secrets from Vault and writes them to a shared volume |
| 40 | +4. Makes the secrets available as environment variables to your MCP server pod |
| 41 | + |
| 42 | +## Step 1: Install and configure Vault |
| 43 | + |
| 44 | +First, install Vault with the Agent Injector enabled in your Kubernetes cluster. |
| 45 | + |
| 46 | +### Install Vault using Helm |
| 47 | + |
| 48 | +Add the HashiCorp Helm repository and install Vault: |
| 49 | + |
| 50 | +```bash |
| 51 | +# Add HashiCorp Helm repository |
| 52 | +helm repo add hashicorp https://helm.releases.hashicorp.com |
| 53 | +helm repo update |
| 54 | + |
| 55 | +# Create vault namespace |
| 56 | +kubectl create namespace vault |
| 57 | + |
| 58 | +# Install Vault with Agent Injector |
| 59 | +helm install vault hashicorp/vault \ |
| 60 | + --namespace vault \ |
| 61 | + --set "server.dev.enabled=true" \ |
| 62 | + --set "server.dev.devRootToken=dev-only-token" \ |
| 63 | + --set "injector.enabled=true" |
| 64 | +``` |
| 65 | + |
| 66 | +:::warning[Development setup only] |
| 67 | + |
| 68 | +This tutorial uses Vault in development mode (`server.dev.enabled=true`) with a |
| 69 | +static root token for simplicity. **Do not use this configuration in |
| 70 | +production**. For production deployments, follow the [Vault production hardening |
| 71 | +guide][vault-hardening]. |
| 72 | + |
| 73 | +::: |
| 74 | + |
| 75 | +Wait for the Vault pod to be ready: |
| 76 | + |
| 77 | +```bash |
| 78 | +kubectl wait --for=condition=ready pod vault-0 \ |
| 79 | + --namespace vault \ |
| 80 | + --timeout=300s |
| 81 | +``` |
| 82 | + |
| 83 | +### Configure Vault authentication |
| 84 | + |
| 85 | +Configure Vault to authenticate Kubernetes service accounts: |
| 86 | + |
| 87 | +```bash |
| 88 | +# Get the Vault pod name |
| 89 | +VAULT_POD=$(kubectl get pods --namespace vault \ |
| 90 | + -l app.kubernetes.io/name=vault \ |
| 91 | + -o jsonpath="{.items[0].metadata.name}") |
| 92 | + |
| 93 | +# Enable Kubernetes auth method |
| 94 | +kubectl exec --namespace vault "$VAULT_POD" -- \ |
| 95 | + vault auth enable kubernetes |
| 96 | + |
| 97 | +# Configure Kubernetes auth |
| 98 | +kubectl exec --namespace vault "$VAULT_POD" -- \ |
| 99 | + vault write auth/kubernetes/config \ |
| 100 | + kubernetes_host="https://kubernetes.default.svc:443" \ |
| 101 | + kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \ |
| 102 | + token_reviewer_jwt=@/var/run/secrets/kubernetes.io/serviceaccount/token |
| 103 | +``` |
| 104 | + |
| 105 | +### Set up secrets engine and policies |
| 106 | + |
| 107 | +Enable a key-value secrets engine and create the necessary policies: |
| 108 | + |
| 109 | +```bash |
| 110 | +# Enable KV secrets engine |
| 111 | +kubectl exec --namespace vault "$VAULT_POD" -- \ |
| 112 | + vault secrets enable -path=workload-secrets kv-v2 |
| 113 | + |
| 114 | +# Create Vault policy for MCP workloads |
| 115 | +kubectl exec --namespace vault "$VAULT_POD" -- \ |
| 116 | + sh -c 'vault policy write toolhive-workload-secrets - << EOF |
| 117 | +path "auth/token/lookup-self" { capabilities = ["read"] } |
| 118 | +path "auth/token/renew-self" { capabilities = ["update"] } |
| 119 | +path "workload-secrets/data/github-mcp/*" { capabilities = ["read"] } |
| 120 | +EOF' |
| 121 | + |
| 122 | +# Create Kubernetes auth role |
| 123 | +kubectl exec --namespace vault "$VAULT_POD" -- \ |
| 124 | + vault write auth/kubernetes/role/toolhive-mcp-workloads \ |
| 125 | + bound_service_account_names="*-proxy-runner,mcp-*" \ |
| 126 | + bound_service_account_namespaces="toolhive-system" \ |
| 127 | + policies="toolhive-workload-secrets" \ |
| 128 | + audience="https://kubernetes.default.svc.cluster.local" \ |
| 129 | + ttl="1h" \ |
| 130 | + max_ttl="4h" |
| 131 | +``` |
| 132 | + |
| 133 | +## Step 2: Store secrets in Vault |
| 134 | + |
| 135 | +Create secrets for your MCP servers in Vault. This example shows how to store a |
| 136 | +GitHub personal access token: |
| 137 | + |
| 138 | +```bash |
| 139 | +# Store GitHub MCP server configuration |
| 140 | +kubectl exec --namespace vault "$VAULT_POD" -- \ |
| 141 | + vault kv put workload-secrets/github-mcp/config \ |
| 142 | + token="ghp_your_github_token_here" \ |
| 143 | + organization="your-org" |
| 144 | +``` |
| 145 | + |
| 146 | +You can verify the secret was stored correctly: |
| 147 | + |
| 148 | +```bash |
| 149 | +kubectl exec --namespace vault "$VAULT_POD" -- \ |
| 150 | + vault kv get workload-secrets/github-mcp/config |
| 151 | +``` |
| 152 | + |
| 153 | +## Step 3: Configure your MCPServer resource |
| 154 | + |
| 155 | +Create an MCPServer resource with Vault annotations to enable automatic secret |
| 156 | +injection. The key is using the `podTemplateMetadataOverrides` field to add |
| 157 | +annotations to the proxy runner pods: |
| 158 | + |
| 159 | +```yaml title="github-mcp-with-vault.yaml" |
| 160 | +apiVersion: toolhive.stacklok.dev/v1alpha1 |
| 161 | +kind: MCPServer |
| 162 | +metadata: |
| 163 | + name: github-vault |
| 164 | + namespace: toolhive-system |
| 165 | +spec: |
| 166 | + image: ghcr.io/github/github-mcp-server:latest |
| 167 | + transport: stdio |
| 168 | + port: 9095 |
| 169 | + permissionProfile: |
| 170 | + type: builtin |
| 171 | + name: network |
| 172 | + resources: |
| 173 | + limits: |
| 174 | + cpu: '100m' |
| 175 | + memory: '128Mi' |
| 176 | + requests: |
| 177 | + cpu: '50m' |
| 178 | + memory: '64Mi' |
| 179 | + resourceOverrides: |
| 180 | + proxyDeployment: |
| 181 | + podTemplateMetadataOverrides: |
| 182 | + annotations: |
| 183 | + # Enable Vault Agent injection |
| 184 | + vault.hashicorp.com/agent-inject: 'true' |
| 185 | + vault.hashicorp.com/role: 'toolhive-mcp-workloads' |
| 186 | + |
| 187 | + # Inject GitHub configuration secret |
| 188 | + vault.hashicorp.com/agent-inject-secret-github-config: 'workload-secrets/data/github-mcp/config' |
| 189 | + vault.hashicorp.com/agent-inject-template-github-config: | |
| 190 | + {{- with secret "workload-secrets/data/github-mcp/config" -}} |
| 191 | + GITHUB_PERSONAL_ACCESS_TOKEN={{ .Data.data.token }} |
| 192 | + {{- end -}} |
| 193 | +``` |
| 194 | +
|
| 195 | +### Understanding the annotations |
| 196 | +
|
| 197 | +The key annotations that enable Vault integration are: |
| 198 | +
|
| 199 | +- `vault.hashicorp.com/agent-inject: "true"` - Enables Vault Agent injection for |
| 200 | + this pod |
| 201 | +- `vault.hashicorp.com/role: "toolhive-mcp-workloads"` - Specifies the Vault |
| 202 | + role to use for authentication |
| 203 | +- `vault.hashicorp.com/agent-inject-secret-github-config` - Tells Vault to |
| 204 | + retrieve a secret and make it available as a file |
| 205 | +- `vault.hashicorp.com/agent-inject-template-github-config` - Uses a Vault |
| 206 | + template to format the secret as environment variables |
| 207 | + |
| 208 | +When ToolHive detects the `vault.hashicorp.com/agent-inject` annotation, it |
| 209 | +automatically configures the proxy runner to read environment variables from the |
| 210 | +`/vault/secrets/` directory where the Vault Agent writes the rendered templates. |
| 211 | + |
| 212 | +## Step 4: Deploy your MCPServer |
| 213 | + |
| 214 | +Apply your MCPServer configuration: |
| 215 | + |
| 216 | +```bash |
| 217 | +kubectl apply -f github-mcp-with-vault.yaml |
| 218 | +``` |
| 219 | + |
| 220 | +Monitor the deployment to ensure both the Vault Agent and ToolHive proxy runner |
| 221 | +start successfully: |
| 222 | + |
| 223 | +```bash |
| 224 | +# Watch the pod start up |
| 225 | +kubectl get pods -n toolhive-system -w |
| 226 | +
|
| 227 | +# Get the pod name |
| 228 | +POD_NAME=$(kubectl get pods -n toolhive-system \ |
| 229 | + -l app.kubernetes.io/instance=github-vault \ |
| 230 | + -o jsonpath="{.items[0].metadata.name}") |
| 231 | +
|
| 232 | +# Check pod logs |
| 233 | +kubectl logs -n toolhive-system $POD_NAME -c vault-agent |
| 234 | +kubectl logs -n toolhive-system $POD_NAME -c toolhive |
| 235 | +``` |
| 236 | + |
| 237 | +You should see the Vault Agent successfully authenticate and retrieve secrets, |
| 238 | +and the ToolHive proxy runner start with the injected environment variables. |
| 239 | + |
| 240 | +## Step 5: Verify the integration |
| 241 | + |
| 242 | +Test that your MCP server has access to the secrets by checking the running pod: |
| 243 | + |
| 244 | +```bash |
| 245 | +# Get the proxy pod name - note the instance name is the same |
| 246 | +# as the name of our MCPServer |
| 247 | +PROXY_POD_NAME=$(kubectl get pods -n toolhive-system \ |
| 248 | + -l app.kubernetes.io/instance=github-vault \ |
| 249 | + -o jsonpath="{.items[0].metadata.name}") |
| 250 | +
|
| 251 | +# Get the mcp server pod name - note the instance name is the same |
| 252 | +# as the name of our MCPServer |
| 253 | +MCP_POD_NAME=$(kubectl get pods -ntoolhive-system \ |
| 254 | + -lapp=github-vault,toolhive-tool-type=mcp \ |
| 255 | + -ojsonpath='{.items[0].metadata.name}') |
| 256 | +
|
| 257 | +# Verify the Vault Agent wrote the secret file |
| 258 | +kubectl exec -n toolhive-system "$PROXY_POD_NAME" -c toolhive -- \ |
| 259 | + cat /vault/secrets/github-config |
| 260 | +
|
| 261 | +# Check that the environment variable is available to the MCP server |
| 262 | +kubectl get pod $MCP_POD_NAME -n toolhive-system -o jsonpath='{range .spec.containers[?(@.name=="mcp")].env[*]}{.name}{"="}{.value}{"\n"}{end}' |
| 263 | +``` |
| 264 | + |
| 265 | +## Security best practices |
| 266 | + |
| 267 | +:::tip[Production recommendations] |
| 268 | + |
| 269 | +- Use Vault in production mode with proper TLS certificates |
| 270 | +- Implement least-privilege policies for secret access |
| 271 | +- Enable audit logging in Vault |
| 272 | +- Regularly rotate Vault tokens and secrets |
| 273 | +- Monitor Vault Agent logs for authentication issues |
| 274 | +- Use namespace isolation for different environments |
| 275 | + |
| 276 | +::: |
| 277 | + |
| 278 | +## Related information |
| 279 | + |
| 280 | +- [Kubernetes quickstart guide](./quickstart-k8s.mdx) |
| 281 | +- [Secrets management guide](../guides-cli/secrets-management.mdx) |
| 282 | +- [HashiCorp Vault documentation](https://developer.hashicorp.com/vault/docs) |
| 283 | +- [Vault Agent Injector documentation][vault-injector] |
| 284 | + |
| 285 | +[vault-hardening]: |
| 286 | + https://developer.hashicorp.com/vault/tutorials/operations/production-hardening |
| 287 | +[vault-injector]: |
| 288 | + https://developer.hashicorp.com/vault/docs/platform/k8s/injector |
0 commit comments