You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+14-2Lines changed: 14 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,7 @@ Durable Streams provides a simple, production-proven protocol for creating and c
11
11
Modern applications frequently need ordered, durable sequences of data that can be replayed from arbitrary points and tailed in real time. Common patterns include:
12
12
13
13
-**AI conversation streaming** - Stream LLM token responses with resume capability across reconnections
14
+
-**Agentic apps** - Stream tool outputs and progress events with replay and clean reconnect semantics
14
15
-**Database synchronization** - Stream database changes to web, mobile, and native clients
15
16
-**Collaborative editing** - Sync CRDTs and operational transforms across devices
16
17
-**Real-time updates** - Push application state to clients with guaranteed delivery
@@ -19,9 +20,11 @@ Modern applications frequently need ordered, durable sequences of data that can
19
20
20
21
While durable streams exist throughout backend infrastructure (database WALs, Kafka topics, event stores), they aren't available as a first-class primitive for client applications. There's no simple, HTTP-based durable stream that sits alongside databases and object storage as a standard cloud primitive.
21
22
22
-
Applications typically implement ad-hoc solutions for resumable streaming—combinations of databases, queues, polling mechanisms, and custom offset tracking. Most implementations handle reconnection poorly: streaming responses break when clients switch tabs, experience brief network interruptions, or refresh pages.
23
+
WebSocket and SSE connections are easy to start, but they're fragile in practice: tabs get suspended, networks flap, devices switch, pages refresh. When that happens, you either lose in-flight data or build a bespoke backend storage and client resume protocol on top.
23
24
24
-
**Durable Streams addresses this gap.** It's a minimal HTTP-based protocol for durable, offset-based streaming designed for client applications across all platforms: web browsers, mobile apps, native clients, IoT devices, and edge workers. Based on 1.5 years of production use at [Electric](https://electric-sql.com/) for real-time Postgres sync.
25
+
AI products make this painfully visible. Token streaming is the UI for chat and copilots, and agentic apps stream progress events, tool outputs, and partial results over long-running sessions. When the stream fails, the product fails—even if the model did the right thing.
26
+
27
+
**Durable Streams addresses this gap.** It's a minimal HTTP-based protocol for durable, offset-based streaming designed for client applications across all platforms: web browsers, mobile apps, native clients, IoT devices, and edge workers. Based on 1.5 years of production use at [Electric](https://electric-sql.com/) for real-time Postgres sync, reliably delivering millions of state changes every day.
-**ETag** - Efficient revalidation for unchanged data
353
356
-**Request collapsing** - Multiple clients requesting same offset collapsed to single upstream request
354
357
358
+
## Performance
359
+
360
+
Durable Streams is built for production scale:
361
+
362
+
-**Low latency** - Sub-15ms end-to-end delivery in production deployments
363
+
-**High concurrency** - Tested with millions of concurrent clients subscribed to a single stream without degradation
364
+
-**Minimal overhead** - The protocol itself adds minimal overhead; throughput scales with your infrastructure
365
+
-**Horizontal scaling** - Offset-based design enables aggressive caching at CDN edges, so read-heavy workloads (common in sync and AI scenarios) scale horizontally without overwhelming origin servers
366
+
355
367
## Relationship to Backend Streaming Systems
356
368
357
369
Backend streaming systems like Kafka, RabbitMQ, and Kinesis excel at server-to-server messaging and backend event processing. Durable Streams complements these systems by solving a different problem: **reliably streaming data to client applications**.
0 commit comments