Skip to content

Commit 19e5cab

Browse files
timoschdtimoschd
committed
added example github actions to deploy to Google CloudRun; added publishing to docker hub
1 parent f24f0a6 commit 19e5cab

File tree

7 files changed

+743
-18
lines changed

7 files changed

+743
-18
lines changed

‎.env.example‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,20 @@
44
# Port to expose the service on (default: 8085)
55
PORT=8085
66

7-
# BigQuery project ID (default: data-warehouse-414312)
8-
BIGQUERY_PROJECT=data-warehouse-414312
7+
# BigQuery project ID
8+
BIGQUERY_PROJECT=your-project-id
99

10-
# BigQuery location/region (default: europe-west4)
11-
BIGQUERY_LOCATION=europe-west4
10+
# BigQuery location/region (e.g., us-central1, europe-west4)
11+
BIGQUERY_LOCATION=us-central1
12+
13+
# Transport mode: stdio (default), http, or sse
14+
# Use 'stdio' for local MCP clients, 'http' or 'sse' for cloud deployments
15+
MCP_TRANSPORT=http
1216

1317
# Optional: Path to BigQuery service account key file
14-
# BIGQUERY_KEY_FILE=secrets/data-warehouse-414312-a4116c865c13.json
18+
# If not provided, Application Default Credentials (ADC) will be used
19+
# BIGQUERY_KEY_FILE=secrets/service-account-key.json
20+
21+
# Optional: Filter to specific datasets (comma-separated)
22+
# If not provided, all datasets in the project will be accessible
23+
# BIGQUERY_DATASETS=dataset1,dataset2,dataset3

‎.github/workflows/README.md‎

Lines changed: 262 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,262 @@
1+
# GitHub Actions Workflows
2+
3+
This directory contains GitHub Actions workflows for building and deploying the MCP BigQuery server.
4+
5+
## Docker Hub Publishing
6+
7+
The `docker-publish.yml` workflow automatically builds and publishes Docker images to Docker Hub.
8+
9+
### Setup Instructions
10+
11+
1. **Create a Docker Hub account** (if you don't have one):
12+
- Go to https://hub.docker.com/signup
13+
14+
2. **Create an access token**:
15+
- Log in to Docker Hub
16+
- Go to Account Settings → Security → New Access Token
17+
- Name: `github-actions`
18+
- Permissions: Read, Write, Delete
19+
- Copy the token (you won't see it again!)
20+
21+
3. **Configure GitHub Secrets**:
22+
23+
Go to your GitHub repository → Settings → Secrets and variables → Actions, and add:
24+
25+
| Secret Name | Description | Example Value |
26+
| -------------------- | ------------------------------ | ------------------ |
27+
| `DOCKERHUB_USERNAME` | Your Docker Hub username | `yourusername` |
28+
| `DOCKERHUB_TOKEN` | Docker Hub access token | `dckr_pat_xxx...` |
29+
30+
4. **Push to trigger build**:
31+
```bash
32+
git push origin main
33+
```
34+
35+
### Workflow Features
36+
37+
- ✅ **Multi-architecture builds** (linux/amd64, linux/arm64)
38+
- ✅ **Automatic tagging** based on git tags and branches
39+
- ✅ **Pull request builds** (without pushing)
40+
- ✅ **Build caching** for faster builds
41+
- ✅ **README sync** to Docker Hub
42+
- ✅ **Manual trigger** via workflow_dispatch
43+
44+
### Image Tags
45+
46+
The workflow creates the following tags:
47+
48+
- `latest` - Latest build from main branch
49+
- `main` - Latest build from main branch
50+
- `v1.2.3` - Semantic version tags (when you push a git tag)
51+
- `v1.2` - Major.minor version
52+
- `v1` - Major version
53+
- `main-abc1234` - Branch name with commit SHA
54+
- `pr-123` - Pull request number
55+
56+
### Using the Published Image
57+
58+
```bash
59+
# Pull the latest image
60+
docker pull yourusername/mcp-server-bigquery:latest
61+
62+
# Pull a specific version
63+
docker pull yourusername/mcp-server-bigquery:v0.3.0
64+
65+
# Run the container
66+
docker run -p 8080:8080 \
67+
-e BIGQUERY_PROJECT=your-project-id \
68+
-e BIGQUERY_LOCATION=us-central1 \
69+
-e MCP_TRANSPORT=http \
70+
yourusername/mcp-server-bigquery:latest
71+
```
72+
73+
### Creating a Release
74+
75+
To publish a versioned release:
76+
77+
```bash
78+
# Tag the release
79+
git tag -a v0.3.0 -m "Release v0.3.0"
80+
git push origin v0.3.0
81+
82+
# The workflow will automatically build and push:
83+
# - yourusername/mcp-server-bigquery:v0.3.0
84+
# - yourusername/mcp-server-bigquery:v0.3
85+
# - yourusername/mcp-server-bigquery:v0
86+
# - yourusername/mcp-server-bigquery:latest
87+
```
88+
89+
## Cloud Run Deployment
90+
91+
The `deploy-cloud-run.yml.example` file provides an example workflow for deploying to Google Cloud Run.
92+
93+
### Setup Instructions
94+
95+
1. **Copy the example file:**
96+
```bash
97+
cp .github/workflows/deploy-cloud-run.yml.example .github/workflows/deploy-cloud-run.yml
98+
```
99+
100+
2. **Create an Artifact Registry repository:**
101+
```bash
102+
gcloud artifacts repositories create mcp-server-bigquery \
103+
--repository-format=docker \
104+
--location=us-central1 \
105+
--description="MCP BigQuery Server Docker images"
106+
```
107+
108+
3. **Set up Workload Identity Federation (Recommended):**
109+
110+
This is the secure, keyless authentication method for GitHub Actions.
111+
112+
```bash
113+
# Set variables
114+
export PROJECT_ID="your-gcp-project-id"
115+
export POOL_NAME="github-actions-pool"
116+
export PROVIDER_NAME="github-provider"
117+
export SERVICE_ACCOUNT_NAME="github-actions-sa"
118+
export REPO="your-github-username/mcp-server-bigquery"
119+
120+
# Create Workload Identity Pool
121+
gcloud iam workload-identity-pools create $POOL_NAME \
122+
--location="global" \
123+
--display-name="GitHub Actions Pool"
124+
125+
# Create Workload Identity Provider
126+
gcloud iam workload-identity-pools providers create-oidc $PROVIDER_NAME \
127+
--location="global" \
128+
--workload-identity-pool=$POOL_NAME \
129+
--display-name="GitHub Provider" \
130+
--attribute-mapping="google.subject=assertion.sub,attribute.actor=assertion.actor,attribute.repository=assertion.repository" \
131+
--issuer-uri="https://token.actions.githubusercontent.com"
132+
133+
# Create Service Account
134+
gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME \
135+
--display-name="GitHub Actions Service Account"
136+
137+
# Grant permissions to the service account
138+
gcloud projects add-iam-policy-binding $PROJECT_ID \
139+
--member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
140+
--role="roles/run.admin"
141+
142+
gcloud projects add-iam-policy-binding $PROJECT_ID \
143+
--member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
144+
--role="roles/artifactregistry.writer"
145+
146+
gcloud projects add-iam-policy-binding $PROJECT_ID \
147+
--member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
148+
--role="roles/iam.serviceAccountUser"
149+
150+
# Allow GitHub Actions to impersonate the service account
151+
gcloud iam service-accounts add-iam-policy-binding \
152+
$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com \
153+
--role="roles/iam.workloadIdentityUser" \
154+
--member="principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_NAME/attribute.repository/$REPO"
155+
```
156+
157+
Note: Replace `PROJECT_NUMBER` with your actual GCP project number (find it with `gcloud projects describe $PROJECT_ID --format="value(projectNumber)"`).
158+
159+
4. **Configure GitHub Secrets:**
160+
161+
Go to your GitHub repository → Settings → Secrets and variables → Actions, and add:
162+
163+
| Secret Name | Description | Example Value |
164+
| ------------------------ | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
165+
| `GCP_PROJECT_ID` | Your GCP project ID | `my-project-123` |
166+
| `WIF_PROVIDER` | Workload Identity Federation provider resource name | `projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/github-actions-pool/providers/github-provider` |
167+
| `WIF_SERVICE_ACCOUNT` | Service account email for Workload Identity Federation | `[email protected]` |
168+
| `BIGQUERY_PROJECT` | BigQuery project ID (can be same as GCP_PROJECT_ID or different) | `my-data-project` |
169+
| `BIGQUERY_LOCATION` | BigQuery location/region | `us-central1` or `europe-west4` |
170+
171+
5. **Optional: Service Account Key (Alternative to Workload Identity Federation):**
172+
173+
If you prefer using a service account key instead of Workload Identity Federation:
174+
175+
```bash
176+
# Create service account
177+
gcloud iam service-accounts create github-actions \
178+
--display-name="GitHub Actions"
179+
180+
# Grant permissions
181+
gcloud projects add-iam-policy-binding $PROJECT_ID \
182+
--member="serviceAccount:github-actions@$PROJECT_ID.iam.gserviceaccount.com" \
183+
--role="roles/run.admin"
184+
185+
gcloud projects add-iam-policy-binding $PROJECT_ID \
186+
--member="serviceAccount:github-actions@$PROJECT_ID.iam.gserviceaccount.com" \
187+
--role="roles/artifactregistry.writer"
188+
189+
# Create and download key
190+
gcloud iam service-accounts keys create key.json \
191+
--iam-account=github-actions@$PROJECT_ID.iam.gserviceaccount.com
192+
```
193+
194+
Then update the workflow to use `google-github-actions/auth@v2` with `credentials_json` instead of Workload Identity Federation.
195+
196+
6. **Customize the workflow:**
197+
198+
Edit `.github/workflows/deploy-cloud-run.yml` and adjust:
199+
- `REGION`: Your preferred Cloud Run region
200+
- `SERVICE_NAME`: Your service name
201+
- Resource limits (memory, CPU, instances)
202+
- Environment variables
203+
204+
7. **Push to trigger deployment:**
205+
```bash
206+
git add .github/workflows/deploy-cloud-run.yml
207+
git commit -m "Add Cloud Run deployment workflow"
208+
git push origin main
209+
```
210+
211+
### Workflow Features
212+
213+
- ✅ **Keyless authentication** using Workload Identity Federation
214+
- ✅ **Multi-architecture support** (builds for linux/amd64)
215+
- ✅ **Image tagging** with both commit SHA and `latest`
216+
- ✅ **Automatic deployment** on push to main branch
217+
- ✅ **Manual trigger** via workflow_dispatch
218+
- ✅ **Secure secrets management** via GitHub Secrets
219+
- ✅ **Auto-scaling** with configurable min/max instances
220+
221+
### Testing the Deployment
222+
223+
After deployment, test your Cloud Run service:
224+
225+
```bash
226+
# Get the service URL
227+
SERVICE_URL=$(gcloud run services describe mcp-server-bigquery \
228+
--region us-central1 \
229+
--format 'value(status.url)')
230+
231+
# Test health endpoint
232+
curl $SERVICE_URL/health
233+
234+
# Expected response:
235+
# {"status":"healthy","service":"bigquery-mcp-server"}
236+
```
237+
238+
### Troubleshooting
239+
240+
**Build fails:**
241+
- Check that Artifact Registry repository exists
242+
- Verify service account has `artifactregistry.writer` role
243+
244+
**Deployment fails:**
245+
- Verify service account has `run.admin` role
246+
- Check that all required secrets are set in GitHub
247+
- Review Cloud Run logs: `gcloud run services logs read mcp-server-bigquery --region us-central1`
248+
249+
**Service fails to start:**
250+
- Check environment variables are set correctly
251+
- Verify BigQuery permissions for the Cloud Run service account
252+
- Review logs for authentication errors
253+
254+
### Cost Optimization
255+
256+
The example workflow configures:
257+
- `--min-instances 0`: Scales to zero when not in use (no cost when idle)
258+
- `--max-instances 10`: Limits maximum concurrent instances
259+
- `--memory 512Mi`: Minimal memory allocation
260+
- `--cpu 1`: Single CPU core
261+
262+
Adjust these based on your usage patterns and budget.

‎.github/workflows/SECRETS.md‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# GitHub Secrets Configuration
2+
3+
This document lists all the secrets needed for the GitHub Actions workflows.
4+
5+
## Required Secrets
6+
7+
Configure these in your GitHub repository: **Settings → Secrets and variables → Actions**
8+
9+
### For Docker Hub Publishing
10+
11+
Required for the `docker-publish.yml` workflow:
12+
13+
| Secret Name | Description | How to Get |
14+
| -------------------- | ------------------------------ | ----------------------------------------------------------------------------------------------- |
15+
| `DOCKERHUB_USERNAME` | Your Docker Hub username | Your Docker Hub account username |
16+
| `DOCKERHUB_TOKEN` | Docker Hub access token | Docker Hub → Account Settings → Security → New Access Token (with Read, Write, Delete permissions) |
17+
18+
### For Cloud Run Deployment (Workload Identity Federation - Recommended)
19+
20+
| Secret Name | Description | How to Get |
21+
| --------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
22+
| `GCP_PROJECT_ID` | Your GCP project ID | `gcloud config get-value project` |
23+
| `WIF_PROVIDER` | Workload Identity Federation provider resource name | Format: `projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_NAME/providers/PROVIDER_NAME` |
24+
| `WIF_SERVICE_ACCOUNT` | Service account email for Workload Identity Federation | Format: `SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com` |
25+
| `BIGQUERY_PROJECT` | BigQuery project ID (can be same or different from GCP) | Your BigQuery project ID |
26+
| `BIGQUERY_LOCATION` | BigQuery location/region | e.g., `us-central1`, `europe-west4`, `asia-northeast1` |
27+
28+
### For Service Account Key (Alternative)
29+
30+
If not using Workload Identity Federation, use these instead:
31+
32+
| Secret Name | Description | How to Get |
33+
| ------------------- | ------------------------------------ | ----------------------------------------------------------------------------------------------- |
34+
| `GCP_PROJECT_ID` | Your GCP project ID | `gcloud config get-value project` |
35+
| `GCP_SA_KEY` | Service account JSON key (base64) | Create key, then: `cat key.json \| base64` |
36+
| `BIGQUERY_PROJECT` | BigQuery project ID | Your BigQuery project ID |
37+
| `BIGQUERY_LOCATION` | BigQuery location/region | e.g., `us-central1`, `europe-west4` |
38+
39+
## Optional Secrets
40+
41+
| Secret Name | Description | Example |
42+
| --------------------- | ---------------------------------------------- | ---------------------------------------- |
43+
| `BIGQUERY_DATASETS` | Comma-separated list of datasets to filter | `analytics,marketing,sales` |
44+
| `SLACK_WEBHOOK_URL` | Slack webhook for deployment notifications | `https://hooks.slack.com/services/...` |
45+
46+
## Getting Your Project Number
47+
48+
You'll need your GCP project number (not project ID) for Workload Identity Federation:
49+
50+
```bash
51+
gcloud projects describe YOUR_PROJECT_ID --format="value(projectNumber)"
52+
```
53+
54+
## Example Values
55+
56+
Here's what your secrets might look like (with fake values):
57+
58+
```
59+
GCP_PROJECT_ID=my-bigquery-project
60+
WIF_PROVIDER=projects/123456789/locations/global/workloadIdentityPools/github-pool/providers/github
61+
WIF_SERVICE_ACCOUNT=github-actions@my-bigquery-project.iam.gserviceaccount.com
62+
BIGQUERY_PROJECT=my-bigquery-project
63+
BIGQUERY_LOCATION=us-central1
64+
```
65+
66+
## Security Best Practices
67+
68+
1. ✅ **Use Workload Identity Federation** instead of service account keys when possible
69+
2. ✅ **Limit service account permissions** to only what's needed (run.admin, artifactregistry.writer)
70+
3. ✅ **Rotate service account keys** regularly if using key-based auth
71+
4. ✅ **Use separate service accounts** for different environments (dev, staging, prod)
72+
5. ✅ **Enable audit logging** for service account usage
73+
6. ✅ **Use GitHub environment protection rules** for production deployments
74+
75+
## Verifying Secrets
76+
77+
After adding secrets, you can verify they're set correctly:
78+
79+
1. Go to your repository on GitHub
80+
2. Navigate to **Settings → Secrets and variables → Actions**
81+
3. You should see all required secrets listed (values are hidden)
82+
4. Trigger the workflow manually to test: **Actions → Deploy to Google Cloud Run → Run workflow**
83+
84+
## Troubleshooting
85+
86+
**"Secret not found" error:**
87+
- Ensure secret names match exactly (case-sensitive)
88+
- Verify secrets are added to the correct repository
89+
- Check if using organization-level secrets vs repository-level secrets
90+
91+
**Authentication fails:**
92+
- Verify WIF_PROVIDER format is correct
93+
- Ensure service account has necessary IAM roles
94+
- Check that Workload Identity binding is configured correctly
95+
96+
**Deployment succeeds but service fails:**
97+
- Check Cloud Run logs for runtime errors
98+
- Verify BIGQUERY_PROJECT and BIGQUERY_LOCATION are correct
99+
- Ensure Cloud Run service account has BigQuery permissions

0 commit comments

Comments
 (0)