Skip to content

Commit 4742308

Browse files
claudegeclos
claude
authored and
geclos
committed
feat: add Helm chart and Dockerfile for deploying Latitude to Kubernetes
Adds a production-ready Helm chart under charts/latitude/ that deploys all four services: web (TanStack Start SSR), api (Hono HTTP), ingest (telemetry ingestion), and workers (Redpanda background jobs). Includes a multi-stage Dockerfile with per-service build targets (api, web, ingest, workers, migrations) and a health endpoint for the web app. Key design decisions: - All four app services deployed with health probes, optional HPA/PDB/Ingress - Secret management supports both inline values (dev/test) and an external pre-existing Kubernetes Secret (production) via existingSecret - ConfigMap and Secret cover all LAT_* env vars including Kafka/Redpanda, object storage (S3), Weaviate, ClickHouse migration vars, and admin DB URL - Pre-install/pre-upgrade migration Job runs Postgres (drizzle-kit), ClickHouse (goose), and Weaviate migrations before app pods roll out - Workers deployment uses extended terminationGracePeriodSeconds for in-flight job completion - Pod annotations include config/secret checksums to trigger rolling restarts on configuration changes - VITE_LAT_* vars documented as build-time (client bundle) with SSR runtime fallback Made-with: Cursor
1 parent d047342 commit 4742308

31 files changed

+1707
-25
lines changed

‎.dockerignore‎

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,17 @@
1-
**/.git/
2-
**/.next/
3-
**/dist/
1+
.git/
2+
.github/
3+
.husky/
4+
.cursor/
5+
.turbo/
6+
.pnpm-store/
47
**/node_modules/
5-
**/docker/
8+
**/dist/
9+
**/coverage/
10+
**/.turbo/
11+
**/.next/
12+
docker/
13+
charts/
614
*.md
7-
.git
8-
.pnpm-store
9-
Dockerfile
10-
docker
11-
node_modules
1215
npm-debug.log
13-
out
16+
.env*
17+
!.env.example

‎Dockerfile‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# ---------------------------------------------------------------------------
4+
# Base image — shared by all stages
5+
# ---------------------------------------------------------------------------
6+
FROM node:25-slim AS base
7+
8+
RUN corepack enable && corepack prepare pnpm@10.30.3 --activate
9+
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*
10+
11+
WORKDIR /app
12+
13+
# ---------------------------------------------------------------------------
14+
# Install dependencies
15+
# ---------------------------------------------------------------------------
16+
FROM base AS deps
17+
18+
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
19+
COPY apps/api/package.json apps/api/package.json
20+
COPY apps/web/package.json apps/web/package.json
21+
COPY apps/ingest/package.json apps/ingest/package.json
22+
COPY apps/workers/package.json apps/workers/package.json
23+
COPY apps/workflows/package.json apps/workflows/package.json
24+
25+
# Copy all package.json files from packages/
26+
COPY packages/ /tmp/packages-src/
27+
RUN find /tmp/packages-src -name 'package.json' -not -path '*/node_modules/*' | while read src; do \
28+
dest="packages/${src#/tmp/packages-src/}"; \
29+
mkdir -p "$(dirname "$dest")"; \
30+
cp "$src" "$dest"; \
31+
done && rm -rf /tmp/packages-src
32+
33+
RUN --mount=type=cache,id=pnpm,target=/root/.local/share/pnpm/store \
34+
pnpm install --frozen-lockfile
35+
36+
# ---------------------------------------------------------------------------
37+
# Source — full repo with deps installed
38+
# ---------------------------------------------------------------------------
39+
FROM base AS source
40+
41+
COPY --from=deps /app/node_modules ./node_modules
42+
COPY --from=deps /app/apps/*/node_modules ./
43+
COPY --from=deps /app/packages/ ./packages/
44+
COPY . .
45+
RUN --mount=type=cache,id=pnpm,target=/root/.local/share/pnpm/store \
46+
pnpm install --frozen-lockfile
47+
48+
# ---------------------------------------------------------------------------
49+
# Build — compile all packages
50+
# ---------------------------------------------------------------------------
51+
FROM source AS build
52+
53+
ARG VITE_LAT_API_URL
54+
ARG VITE_LAT_WEB_URL
55+
56+
RUN pnpm build
57+
58+
# ---------------------------------------------------------------------------
59+
# Target: api
60+
# ---------------------------------------------------------------------------
61+
FROM base AS api
62+
63+
COPY --from=build /app ./
64+
65+
ENV NODE_ENV=production
66+
EXPOSE 3001
67+
68+
CMD ["node", "apps/api/dist/server.js"]
69+
70+
# ---------------------------------------------------------------------------
71+
# Target: ingest
72+
# ---------------------------------------------------------------------------
73+
FROM base AS ingest
74+
75+
COPY --from=build /app ./
76+
77+
ENV NODE_ENV=production
78+
EXPOSE 3002
79+
80+
CMD ["node", "apps/ingest/dist/server.js"]
81+
82+
# ---------------------------------------------------------------------------
83+
# Target: workers
84+
# ---------------------------------------------------------------------------
85+
FROM base AS workers
86+
87+
COPY --from=build /app ./
88+
89+
ENV NODE_ENV=production
90+
EXPOSE 9090
91+
92+
CMD ["node", "apps/workers/dist/server.js"]
93+
94+
# ---------------------------------------------------------------------------
95+
# Target: web (TanStack Start / Vite SSR)
96+
# ---------------------------------------------------------------------------
97+
FROM base AS web
98+
99+
COPY --from=build /app ./
100+
101+
ENV NODE_ENV=production
102+
EXPOSE 3000
103+
104+
CMD ["node", "apps/web/.output/server/index.mjs"]
105+
106+
# ---------------------------------------------------------------------------
107+
# Target: migrations
108+
# Runs Postgres (drizzle-kit), ClickHouse (goose), and Weaviate migrations.
109+
# ---------------------------------------------------------------------------
110+
FROM base AS migrations
111+
112+
RUN apt-get update && \
113+
apt-get install -y --no-install-recommends curl && \
114+
GOOSE_VERSION=3.24.3 && \
115+
ARCH=$(dpkg --print-architecture) && \
116+
curl -fsSL "https://github.com/pressly/goose/releases/download/v${GOOSE_VERSION}/goose_linux_${ARCH}" \
117+
-o /usr/local/bin/goose && \
118+
chmod +x /usr/local/bin/goose && \
119+
apt-get purge -y curl && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*
120+
121+
COPY --from=build /app ./
122+
123+
ENV NODE_ENV=production
124+
125+
CMD ["sh", "-c", "pnpm --filter @platform/db-postgres pg:migrate && pnpm --filter @platform/db-clickhouse ch:up && pnpm --filter @platform/db-weaviate wv:migrate"]

‎apps/api/src/middleware/touch-buffer.ts‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ interface TouchBufferConfig {
2626
*
2727
* Performance impact:
2828
* - Reduces writes by 90%+ (30s window / avg 100ms request = 300:1 reduction)
29-
* - lastUsedAt accuracy reduced to flush interval (±30 seconds by default)
29+
* - lastUsedAt accuracy reduced to flush interval (+/-30 seconds by default)
3030
*
3131
* Usage:
3232
* ```typescript
@@ -111,11 +111,9 @@ class TouchBuffer {
111111
}).pipe(Effect.provide(apiKeyRepoLayer), Effect.provide(sqlClientLayer)),
112112
)
113113

114-
const duration = Date.now() - startTime
115-
logger.info(`Flushed ${keyIds.length} touch updates in ${duration}ms`)
114+
logger.info(`Flushed ${keyIds.length} touch updates in ${Date.now() - startTime}ms`)
116115
} catch (error) {
117-
const errorMessage = error instanceof Error ? error.message : "Unknown error"
118-
logger.error(`Failed to flush touch updates: ${errorMessage}`)
116+
logger.error(`Failed to flush touch updates: ${error instanceof Error ? error.message : "Unknown error"}`)
119117

120118
// Re-add failed keys to buffer for retry (with limit to prevent unbounded growth)
121119
for (const [keyId, timestamp] of batch) {

‎apps/web/src/routeTree.gen.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ import { Route as AuthenticatedRouteImport } from './routes/_authenticated'
1616
import { Route as AuthenticatedIndexRouteImport } from './routes/_authenticated/index'
1717
import { Route as AuthConfirmRouteImport } from './routes/auth/confirm'
1818
import { Route as AuthCliRouteImport } from './routes/auth/cli'
19+
import { Route as ApiHealthRouteImport } from './routes/api/health'
1920
import { Route as AuthenticatedSettingsRouteImport } from './routes/_authenticated/settings'
2021
import { Route as ApiAuthSplatRouteImport } from './routes/api/auth/$'
2122
import { Route as AuthenticatedProjectsProjectIdRouteImport } from './routes/_authenticated/projects/$projectId'
@@ -60,6 +61,11 @@ const AuthCliRoute = AuthCliRouteImport.update({
6061
path: '/auth/cli',
6162
getParentRoute: () => rootRouteImport,
6263
} as any)
64+
const ApiHealthRoute = ApiHealthRouteImport.update({
65+
id: '/api/health',
66+
path: '/api/health',
67+
getParentRoute: () => rootRouteImport,
68+
} as any)
6369
const AuthenticatedSettingsRoute = AuthenticatedSettingsRouteImport.update({
6470
id: '/settings',
6571
path: '/settings',
@@ -121,6 +127,7 @@ export interface FileRoutesByFullPath {
121127
'/login': typeof LoginRoute
122128
'/signup': typeof SignupRoute
123129
'/settings': typeof AuthenticatedSettingsRoute
130+
'/api/health': typeof ApiHealthRoute
124131
'/auth/cli': typeof AuthCliRoute
125132
'/auth/confirm': typeof AuthConfirmRoute
126133
'/projects/$projectId': typeof AuthenticatedProjectsProjectIdRouteWithChildren
@@ -137,6 +144,7 @@ export interface FileRoutesByTo {
137144
'/login': typeof LoginRoute
138145
'/signup': typeof SignupRoute
139146
'/settings': typeof AuthenticatedSettingsRoute
147+
'/api/health': typeof ApiHealthRoute
140148
'/auth/cli': typeof AuthCliRoute
141149
'/auth/confirm': typeof AuthConfirmRoute
142150
'/': typeof AuthenticatedIndexRoute
@@ -155,6 +163,7 @@ export interface FileRoutesById {
155163
'/login': typeof LoginRoute
156164
'/signup': typeof SignupRoute
157165
'/_authenticated/settings': typeof AuthenticatedSettingsRoute
166+
'/api/health': typeof ApiHealthRoute
158167
'/auth/cli': typeof AuthCliRoute
159168
'/auth/confirm': typeof AuthConfirmRoute
160169
'/_authenticated/': typeof AuthenticatedIndexRoute
@@ -175,6 +184,7 @@ export interface FileRouteTypes {
175184
| '/login'
176185
| '/signup'
177186
| '/settings'
187+
| '/api/health'
178188
| '/auth/cli'
179189
| '/auth/confirm'
180190
| '/projects/$projectId'
@@ -191,6 +201,7 @@ export interface FileRouteTypes {
191201
| '/login'
192202
| '/signup'
193203
| '/settings'
204+
| '/api/health'
194205
| '/auth/cli'
195206
| '/auth/confirm'
196207
| '/'
@@ -208,6 +219,7 @@ export interface FileRouteTypes {
208219
| '/login'
209220
| '/signup'
210221
| '/_authenticated/settings'
222+
| '/api/health'
211223
| '/auth/cli'
212224
| '/auth/confirm'
213225
| '/_authenticated/'
@@ -226,6 +238,7 @@ export interface RootRouteChildren {
226238
DesignSystemRoute: typeof DesignSystemRoute
227239
LoginRoute: typeof LoginRoute
228240
SignupRoute: typeof SignupRoute
241+
ApiHealthRoute: typeof ApiHealthRoute
229242
AuthCliRoute: typeof AuthCliRoute
230243
AuthConfirmRoute: typeof AuthConfirmRoute
231244
ApiAuthSplatRoute: typeof ApiAuthSplatRoute
@@ -282,6 +295,13 @@ declare module '@tanstack/react-router' {
282295
preLoaderRoute: typeof AuthCliRouteImport
283296
parentRoute: typeof rootRouteImport
284297
}
298+
'/api/health': {
299+
id: '/api/health'
300+
path: '/api/health'
301+
fullPath: '/api/health'
302+
preLoaderRoute: typeof ApiHealthRouteImport
303+
parentRoute: typeof rootRouteImport
304+
}
285305
'/_authenticated/settings': {
286306
id: '/_authenticated/settings'
287307
path: '/settings'
@@ -400,6 +420,7 @@ const rootRouteChildren: RootRouteChildren = {
400420
DesignSystemRoute: DesignSystemRoute,
401421
LoginRoute: LoginRoute,
402422
SignupRoute: SignupRoute,
423+
ApiHealthRoute: ApiHealthRoute,
403424
AuthCliRoute: AuthCliRoute,
404425
AuthConfirmRoute: AuthConfirmRoute,
405426
ApiAuthSplatRoute: ApiAuthSplatRoute,

‎apps/web/src/routes/api/health.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
import { createFileRoute } from "@tanstack/react-router"
2+
3+
export const Route = createFileRoute("/api/health")({
4+
server: {
5+
handlers: {
6+
GET: async () => {
7+
return new Response(JSON.stringify({ status: "ok" }), {
8+
status: 200,
9+
headers: { "Content-Type": "application/json" },
10+
})
11+
},
12+
},
13+
},
14+
})

‎apps/workers/src/server.ts‎

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import { existsSync } from "node:fs"
2+
import { createServer } from "node:http"
23
import { fileURLToPath } from "node:url"
34
import { createPollingOutboxConsumer } from "@platform/events-outbox"
45
import {
@@ -22,8 +23,24 @@ if (existsSync(envFilePath)) {
2223

2324
const pgPool = getPostgresPool(10)
2425
const logger = createLogger("workers")
26+
let ready = false
27+
28+
const healthPort = Number(process.env.LAT_WORKERS_HEALTH_PORT) || 9090
29+
const healthServer = createServer((req, res) => {
30+
if (req.url === "/health" && req.method === "GET") {
31+
const status = ready ? 200 : 503
32+
res.writeHead(status, { "Content-Type": "application/json" })
33+
res.end(JSON.stringify({ status: ready ? "ok" : "starting" }))
34+
} else {
35+
res.writeHead(404)
36+
res.end()
37+
}
38+
})
39+
40+
healthServer.listen(healthPort, () => {
41+
logger.info(`workers health check listening on :${healthPort}/health`)
42+
})
2543

26-
// Simple event handler that logs events (placeholder for actual side effect processing)
2744
const eventHandler = {
2845
handle: (event: {
2946
id: string
@@ -36,18 +53,14 @@ const eventHandler = {
3653
}),
3754
}
3855

39-
// Initialize workers asynchronously
4056
const initializeWorkers = async () => {
41-
// Load Redpanda configuration
4257
const kafkaConfig = Effect.runSync(loadKafkaConfig())
4358
const kafkaClient = createKafkaClient(kafkaConfig)
4459

45-
// Create Redpanda publisher (async initialization)
4660
const eventsPublisher = await createRedpandaEventsPublisher({
4761
kafka: kafkaClient,
4862
})
4963

50-
// Create outbox consumer (polls Postgres outbox and publishes to Redpanda)
5164
const outboxConsumer = createPollingOutboxConsumer(
5265
{
5366
pool: pgPool,
@@ -57,19 +70,18 @@ const initializeWorkers = async () => {
5770
eventsPublisher,
5871
)
5972

60-
// Create Redpanda event consumer (consumes from Redpanda and processes events)
6173
const redpandaConsumer = createRedpandaEventsConsumer({
6274
kafka: kafkaClient,
6375
groupId: kafkaConfig.groupId,
6476
})
6577

66-
// Create span ingestion worker (consumes from span-ingestion topic and writes to ClickHouse)
6778
const spanIngestionWorker = createSpanIngestionWorker(kafkaClient, `${kafkaConfig.groupId}-span-ingestion`)
6879

69-
// Start consumers
7080
outboxConsumer.start()
7181
await redpandaConsumer.start(eventHandler)
7282
await spanIngestionWorker.start()
83+
84+
ready = true
7385
logger.info("workers ready - outbox consumer and Redpanda consumer started")
7486

7587
return { outboxConsumer, redpandaConsumer, spanIngestionWorker }
@@ -80,9 +92,11 @@ const workersPromise = initializeWorkers().catch((error) => {
8092
process.exit(1)
8193
})
8294

83-
// Graceful shutdown
8495
process.on("SIGINT", async () => {
8596
logger.info("shutting down workers...")
97+
ready = false
98+
healthServer.close()
99+
86100
try {
87101
const { outboxConsumer, redpandaConsumer, spanIngestionWorker } = await workersPromise
88102
await outboxConsumer.stop()
@@ -91,7 +105,7 @@ process.on("SIGINT", async () => {
91105
} catch (error) {
92106
logger.error("Error during shutdown (workers may not have started)", error)
93107
}
94-
await pgPool.end()
95108

109+
await pgPool.end()
96110
process.exit(0)
97111
})

0 commit comments

Comments
 (0)