Skip to content

Commit c02bcd9

Browse files
committed
feat(dev-playground): durable-task example + typed SSE client helper
The reference implementation for plugin authors. A demo plugin covering both TaskFlow recovery patterns (manual via `ctx.previousEvents` and structural via `step()`), a typed frontend SSE consumer (`subscribeToTaskflowTask<TEvents>`), and the `connectSSE` parser extension that captures `event:` field names. Bottom-up teaching: server plugin → bridge → client helper → React UI. Server demo plugin (`apps/dev-playground/server/durable-task-example-plugin.ts`): - `count-to-n` task — manual recovery via `ctx.previousEvents`. Ticks once per `sleepMs`, emitting typed `tick` events. On recovery, scans the event log to find the last persisted tick and resumes from there. The pattern for "checkpoint is the last time I emitted X" with no expensive computation to memoize. - `pipeline-with-steps` task — automatic recovery via `step()`. Wraps each stage (extract → transform → load) with `step()`, which memoizes its result in the WAL the first time it runs. On recovery, completed stages return the cached value without re-executing. The pattern for stages that are expensive (LLM calls, large queries) and unsafe to replay. - Routes (mounted under `/api/durable-example`): - `POST /run`, `POST /run-pipeline` — start + bridge SSE via `executeTask`. - `POST /crash/:id` — `simulateCrash` (gated behind `NODE_ENV !== "production"`). - `POST /stop/:id` — cooperative `taskflow.stop({ reason })`. - `POST /nudge-recovery` — re-submits the original input so the same IK triggers stale-Running recovery (`engine.resume()` only applies to Suspended tasks, so the demo "nudges" the engine). - `GET /reattach/:id` — bridges an SSE stream onto an existing task by IK via `subscribe()` + `setupSseHeaders` + `writeSseFrame` directly (the `executeTask` path would derive a new IK). Performs an OBO ownership check via `asUser(req).reconnect(id, userId)` before subscribing (F11 fix). - Registers with `enableTestMode: true` so `simulateCrash` is available; the route handler additionally gates on `NODE_ENV` so a misconfigured production deployment can't crash live tasks. `apps/dev-playground/server/index.ts`: - Registers the demo plugin and enables test mode on the TaskFlow config (`taskflow: { engine: { enableTestMode: true } }`) so `simulateCrash` is callable from the demo route. Client React route (`apps/dev-playground/client/src/routes/durable-task.route.tsx`): - Exercises both tasks end-to-end: `POST /run` then opens an SSE stream via `subscribeToTaskflowTask<CountEvents>`. Renders `tick` / `recovered` events for `count-to-n`; renders `stage_started` / `stage_done` / `recovered` for `pipeline-with-steps` (which surfaces "from cache" on recovered stages). - Buttons for Stop, Crash, Nudge, and Reattach exercise the full cancellation / crash / recovery / re-attach loop. - Adds nav entries in `__root.tsx`, `index.tsx`, and the TanStack-generated `routeTree.gen.ts`. Typed client helper (`packages/appkit-ui/src/js/sse/subscribe-taskflow-task.ts`): - `subscribeToTaskflowTask<TEvents>(url, { onEvent, onComplete, onError, signal? })` — typed async API consuming the AppKit SSE bridge. Each `event: <name>` frame is dispatched to `onEvent[name]` with `payload` typed as `TEvents[name]`. - Terminal events (`completed`, `failed`, `cancelled`) resolve / reject the returned promise so plugins can `await` the durable run without an event handler. - `Last-Event-ID` reconnection: the helper tracks the highest seen `id:` frame and reattaches with that header on transient network failure. Tests assert the reconnect math is correct. - Includes tests for happy-path streaming, terminal events, abort via `AbortSignal`, and Last-Event-ID reconnect. `connect-sse` extension (`packages/appkit-ui/src/js/sse/connect-sse.ts`, `types.ts`, `index.ts`): - The generic SSE parser captures `event:` field names alongside `data:` payloads. `SSEMessage` gains `event?: string` so any AppKit SSE consumer can inspect the event name without re-parsing. Tests cover multi-line `data:` joining, CRLF normalisation, and comment-frame handling. - Export the new typed helper from `index.ts`. Gitignore: - `apps/dev-playground/.gitignore` adds `tasks.*` / `*.wal` patterns as a defensive belt-and-braces around the existing `.appkit/` exclusion. The demo plugin may configure storage at the playground root for diagnostics; the additional patterns keep `tasks.db` and the rotating WAL out of git regardless of `databasePath`. Verify: - `pnpm -r typecheck`, `pnpm build`, `pnpm test` (125 files, 2304 tests) all green. - `pnpm exec biome check` clean on touched files. - `pnpm exec knip` clean. Risk. Demo plugin is unauthenticated by design (it ships with the dev playground, not the SDK). `/crash/:id` returns 404 in production via `NODE_ENV` gate; `enableTestMode` flips on `simulateCrash`. The demo route handlers do not enforce auth — they assume the playground sits behind the Databricks Apps proxy. Document in deployment notes. Not in this PR. No production-plugin changes. No doc rewrite — that's PR 7. The `subscribeToTaskflowTask` helper currently requires plugin authors to redeclare `TEvents` client-side; a future follow-up (F26) would derive it from the registered `TaskHandle`. Stacked on: stack/taskflow/analytics-migration. Signed-off-by: ditadi <victordperd@gmail.com>
1 parent 9a2926d commit c02bcd9

13 files changed

Lines changed: 2322 additions & 1 deletion

File tree

apps/dev-playground/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,5 +5,8 @@ playwright-report/
55
# Auto-generated types (endpoint-specific, varies per developer)
66
shared/appkit-types/serving.d.ts
77

8+
tasks.*
9+
*.wal
10+
811
# TaskFlow durable storage (SQLite + WAL); per-machine, never checked in.
912
.appkit/

apps/dev-playground/client/src/routeTree.gen.ts

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ import { Route as LakebaseRouteRouteImport } from './routes/lakebase.route'
2020
import { Route as JobsRouteRouteImport } from './routes/jobs.route'
2121
import { Route as GenieRouteRouteImport } from './routes/genie.route'
2222
import { Route as FilesRouteRouteImport } from './routes/files.route'
23+
import { Route as DurableTaskRouteRouteImport } from './routes/durable-task.route'
2324
import { Route as DataVisualizationRouteRouteImport } from './routes/data-visualization.route'
2425
import { Route as ChartInferenceRouteRouteImport } from './routes/chart-inference.route'
2526
import { Route as ArrowAnalyticsRouteRouteImport } from './routes/arrow-analytics.route'
@@ -81,6 +82,11 @@ const FilesRouteRoute = FilesRouteRouteImport.update({
8182
path: '/files',
8283
getParentRoute: () => rootRouteImport,
8384
} as any)
85+
const DurableTaskRouteRoute = DurableTaskRouteRouteImport.update({
86+
id: '/durable-task',
87+
path: '/durable-task',
88+
getParentRoute: () => rootRouteImport,
89+
} as any)
8490
const DataVisualizationRouteRoute = DataVisualizationRouteRouteImport.update({
8591
id: '/data-visualization',
8692
path: '/data-visualization',
@@ -113,6 +119,7 @@ export interface FileRoutesByFullPath {
113119
'/arrow-analytics': typeof ArrowAnalyticsRouteRoute
114120
'/chart-inference': typeof ChartInferenceRouteRoute
115121
'/data-visualization': typeof DataVisualizationRouteRoute
122+
'/durable-task': typeof DurableTaskRouteRoute
116123
'/files': typeof FilesRouteRoute
117124
'/genie': typeof GenieRouteRoute
118125
'/jobs': typeof JobsRouteRoute
@@ -131,6 +138,7 @@ export interface FileRoutesByTo {
131138
'/arrow-analytics': typeof ArrowAnalyticsRouteRoute
132139
'/chart-inference': typeof ChartInferenceRouteRoute
133140
'/data-visualization': typeof DataVisualizationRouteRoute
141+
'/durable-task': typeof DurableTaskRouteRoute
134142
'/files': typeof FilesRouteRoute
135143
'/genie': typeof GenieRouteRoute
136144
'/jobs': typeof JobsRouteRoute
@@ -150,6 +158,7 @@ export interface FileRoutesById {
150158
'/arrow-analytics': typeof ArrowAnalyticsRouteRoute
151159
'/chart-inference': typeof ChartInferenceRouteRoute
152160
'/data-visualization': typeof DataVisualizationRouteRoute
161+
'/durable-task': typeof DurableTaskRouteRoute
153162
'/files': typeof FilesRouteRoute
154163
'/genie': typeof GenieRouteRoute
155164
'/jobs': typeof JobsRouteRoute
@@ -170,6 +179,7 @@ export interface FileRouteTypes {
170179
| '/arrow-analytics'
171180
| '/chart-inference'
172181
| '/data-visualization'
182+
| '/durable-task'
173183
| '/files'
174184
| '/genie'
175185
| '/jobs'
@@ -188,6 +198,7 @@ export interface FileRouteTypes {
188198
| '/arrow-analytics'
189199
| '/chart-inference'
190200
| '/data-visualization'
201+
| '/durable-task'
191202
| '/files'
192203
| '/genie'
193204
| '/jobs'
@@ -206,6 +217,7 @@ export interface FileRouteTypes {
206217
| '/arrow-analytics'
207218
| '/chart-inference'
208219
| '/data-visualization'
220+
| '/durable-task'
209221
| '/files'
210222
| '/genie'
211223
| '/jobs'
@@ -225,6 +237,7 @@ export interface RootRouteChildren {
225237
ArrowAnalyticsRouteRoute: typeof ArrowAnalyticsRouteRoute
226238
ChartInferenceRouteRoute: typeof ChartInferenceRouteRoute
227239
DataVisualizationRouteRoute: typeof DataVisualizationRouteRoute
240+
DurableTaskRouteRoute: typeof DurableTaskRouteRoute
228241
FilesRouteRoute: typeof FilesRouteRoute
229242
GenieRouteRoute: typeof GenieRouteRoute
230243
JobsRouteRoute: typeof JobsRouteRoute
@@ -317,6 +330,13 @@ declare module '@tanstack/react-router' {
317330
preLoaderRoute: typeof FilesRouteRouteImport
318331
parentRoute: typeof rootRouteImport
319332
}
333+
'/durable-task': {
334+
id: '/durable-task'
335+
path: '/durable-task'
336+
fullPath: '/durable-task'
337+
preLoaderRoute: typeof DurableTaskRouteRouteImport
338+
parentRoute: typeof rootRouteImport
339+
}
320340
'/data-visualization': {
321341
id: '/data-visualization'
322342
path: '/data-visualization'
@@ -361,6 +381,7 @@ const rootRouteChildren: RootRouteChildren = {
361381
ArrowAnalyticsRouteRoute: ArrowAnalyticsRouteRoute,
362382
ChartInferenceRouteRoute: ChartInferenceRouteRoute,
363383
DataVisualizationRouteRoute: DataVisualizationRouteRoute,
384+
DurableTaskRouteRoute: DurableTaskRouteRoute,
364385
FilesRouteRoute: FilesRouteRoute,
365386
GenieRouteRoute: GenieRouteRoute,
366387
JobsRouteRoute: JobsRouteRoute,

apps/dev-playground/client/src/routes/__root.tsx

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,14 @@ function RootComponent() {
6464
Reconnect
6565
</Button>
6666
</Link>
67+
<Link to="/durable-task" className="no-underline">
68+
<Button
69+
variant="ghost"
70+
className="text-foreground hover:text-secondary-foreground"
71+
>
72+
Durable Task
73+
</Button>
74+
</Link>
6775
<Link to="/telemetry" className="no-underline">
6876
<Button
6977
variant="ghost"

0 commit comments

Comments
 (0)