You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Server methods returning AsyncIterable are automatically detected and
streamed chunk-by-chunk to the client, where they're consumed via
for await...of. Supports consumer cancellation (break sends
stream-cancel), error propagation, concurrent streams, per-chunk
output validation, and interceptor integration.
Adds streaming-demo example, docs page, 9 integration tests.
Removes PLANNING.md — all roadmap items now complete.
Copy file name to clipboardExpand all lines: .journal/2026-02-07.md
+47Lines changed: 47 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -81,3 +81,50 @@ Also fixed a pre-existing inconsistency: `ioredis`, `kafkajs`, `socket.io`, and
81
81
-**`pnpm-lock.yaml`** — will need `pnpm install` to reconcile the lockfile after this change
82
82
-**CI** — adapter tests that import these packages will need them installed explicitly in the test environment (likely already the case via devDependencies or workspace config)
83
83
-**Docs** — README adapter sections should note the required peer install (e.g., `npm install kkrpc kafkajs`)
84
+
85
+
## 01:05 — AsyncIterable Streaming Support (PLANNING.md item 2)
86
+
87
+
### Core Decision/Topic
88
+
89
+
Implemented **streaming / subscription support** — the final major feature from the roadmap. Server methods can now return `AsyncIterable` (async generators), and kkrpc automatically streams chunks to the client where they're consumed via `for await...of`.
90
+
91
+
### Options Considered
92
+
93
+
-**Dedicated subscription API** (e.g. `channel.subscribe("topic")`) vs **transparent AsyncIterable detection** — chose transparent detection. If a handler returns something with `Symbol.asyncIterator`, kkrpc streams it automatically. No new API surface for users to learn; async generators "just work".
94
+
-**Backpressure protocol** (consumer ACKs each chunk before producer sends next) vs **fire-and-forget chunks with buffering** — chose buffering. Simpler protocol, and most RPC streaming use cases (progress updates, log tailing, countdowns) don't need backpressure. Can be added later if needed.
95
+
-**Per-chunk interceptor calls** vs **interceptors wrap initial handler only** — chose handler-only. Interceptors see the method call that starts the stream but don't fire per-chunk. This matches how middleware works in gRPC and tRPC.
96
+
97
+
### Final Decision & Rationale
98
+
99
+
**Protocol:** 4 new message types added to the wire format:
100
+
-`stream-chunk` — producer → consumer, carries one value
-`stream-cancel` — consumer → producer, signals `break` in `for await`
104
+
105
+
The initial response carries `{ __stream: true }` to signal the consumer to create an AsyncIterable instead of resolving the Promise directly.
106
+
107
+
**Consumer-side queue pattern:** Chunks arrive asynchronously; consumer reads synchronously via `next()`. Bridged with a buffer + waiters queue — chunks that arrive before `next()` are buffered; `next()` calls that arrive before a chunk park as pending resolvers.
108
+
109
+
**Cancellation:**`break` in `for await` triggers the iterator's `return()` method, which sends `stream-cancel`. Producer uses `AbortController` to stop its iteration loop.
110
+
111
+
**Validation integration:** Per-chunk output validation supported — if a schema is configured for the method's output, each chunk is validated before sending.
112
+
113
+
### Key Changes Made
114
+
115
+
| File | Change |
116
+
|---|---|
117
+
|`packages/kkrpc/src/serialization.ts`| Extended `Message.type` union with 4 streaming message types |
0 commit comments