Skip to content

Commit 83a2fb2

Browse files
authored
Added job to backfill ghost_uuid in sites table (#1432)
Added job to backfill `ghost_uuid` in `sites` table ref https://linear.app/ghost/issue/BER-2964 Added job that backfills the `ghost_uuid` field recently added to the `sites` table
1 parent 3060f89 commit 83a2fb2

File tree

10 files changed

+706
-3
lines changed

10 files changed

+706
-3
lines changed

biome.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
"groups": [
2424
["vitest", "@cucumber/cucumber"],
2525
":BLANK_LINE:",
26-
":NODE:",
26+
[":NODE:", ":BUN:"],
2727
":BLANK_LINE:",
2828
":PACKAGE:",
2929
":BLANK_LINE:",

jobs/__template__/index.test.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
import mysql from 'mysql2/promise';
2-
31
import {
42
afterAll,
53
beforeAll,
@@ -8,6 +6,9 @@ import {
86
expect,
97
test,
108
} from 'bun:test';
9+
10+
import mysql from 'mysql2/promise';
11+
1112
import { doSomething } from './index';
1213

1314
describe('__JOB_NAME__', () => {
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
FROM oven/bun:1.3.0-alpine@sha256:37e6b1cbe053939bccf6ae4507977ed957eaa6e7f275670b72ad6348e0d2c11f
2+
3+
WORKDIR /opt/job
4+
5+
COPY index.ts .
6+
7+
ENTRYPOINT ["bun", "run", "index.ts"]
8+
CMD []

jobs/backfill-ghost-uuid/README.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# backfill-ghost-uuid
2+
3+
Backfills the `ghost_uuid` field in the `sites` table by fetching the UUID from
4+
each site's `/ghost/api/admin/site/` endpoint
5+
6+
For each `site` in the database without a `ghost_uuid`, this job:
7+
8+
1. Makes a request to `https://<site_host>/ghost/api/admin/site/`
9+
2. Extracts the `site_uuid` from the response
10+
3. Updates the site record in the database
11+
12+
Sites are processed sequentially with a 500ms delay between requests to avoid
13+
tripping rate limits. If a request fails, the job logs a warning and continues
14+
to the next site
15+
16+
## Prerequisites
17+
18+
- Globally available `bun`
19+
- Globally available `docker`
20+
21+
## Running locally
22+
23+
```bash
24+
DB_HOST=... \
25+
DB_PORT=... \
26+
DB_USER=... \
27+
DB_PASSWORD=... \
28+
DB_NAME=... \
29+
bun run index.ts
30+
```
31+
32+
## Running tests
33+
34+
```bash
35+
./run-tests.sh
36+
```
37+
38+
## Production deployment
39+
40+
Build and push the Docker image to GCP:
41+
42+
```bash
43+
./gcloud-push.sh
44+
```
45+
46+
## Production execution
47+
48+
Use the [GCP console](https://console.cloud.google.com/run/jobs/create) to setup and execute the job
49+
50+
Ensure the relevant environment variables are set on the job:
51+
52+
```text
53+
DB_HOST
54+
DB_PORT
55+
DB_USER
56+
DB_PASSWORD
57+
DB_NAME
58+
```
59+
60+
If using a socket connection, set `DB_SOCKET_PATH` instead of `DB_HOST` and `DB_PORT`
61+
62+
## Notes
63+
64+
- The job is idempotent and can be run multiple times safely
65+
- Sites are processed sequentially to avoid tripping rate limits
66+
- Failed requests are logged but don't stop the job from processing remaining sites
67+
- The job is designed to be run periodically to catch any sites that failed on previous runs
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
services:
2+
mysql-test:
3+
image: mysql:8.4.6@sha256:5367102acfefeaa47eb0eb57c8d4f8b96c8c14004859131eac9bbfaa62f81e34
4+
environment:
5+
MYSQL_ROOT_PASSWORD: root
6+
MYSQL_DATABASE: backfill-ghost-uuid
7+
ports:
8+
- "3308:3306"
9+
tmpfs:
10+
- /var/lib/mysql
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
IMAGE_NAME="backfill-ghost-uuid-job"
6+
7+
PROJECT_ID=$(gcloud config get-value project 2>/dev/null)
8+
if [ -z "$PROJECT_ID" ]; then
9+
echo "Error: Could not determine GCP project ID"
10+
exit 1
11+
fi
12+
13+
echo ""
14+
echo "Building and pushing to GCP..."
15+
echo ""
16+
echo "Project: $PROJECT_ID"
17+
echo "Image: $IMAGE_NAME"
18+
echo ""
19+
20+
echo "Building Docker image..."
21+
echo ""
22+
docker build --platform linux/amd64 -t "$IMAGE_NAME" .
23+
echo ""
24+
25+
IMAGE_URL="gcr.io/$PROJECT_ID/$IMAGE_NAME:latest"
26+
echo "Tagging image: $IMAGE_URL"
27+
docker tag "$IMAGE_NAME" "$IMAGE_URL"
28+
echo ""
29+
30+
echo "Pushing image to Google Container Registry..."
31+
echo ""
32+
docker push "$IMAGE_URL"
33+
echo ""
34+
35+
echo ""
36+
echo "Push complete!"
37+
echo ""
38+
echo "Built and pushed image to '$IMAGE_URL'"
39+
echo ""

0 commit comments

Comments
 (0)