Skip to content

Commit 9ed751e

Browse files
matthew-pettymatthew-petty
authored
feat(stream-object): add /stream-object endpoint with SDK support (#73)
* feat(gateway): add /stream-object endpoint for streaming JSON objects Add a new /stream-object endpoint that streams partial JSON objects as they're generated using Server-Sent Events (SSE). This follows Anthropic's input_json_delta pattern for real-time structured output. Features: - Partial JSON parsing using partial-json library - Emits partial-object events with parsed objects as tokens arrive - Emits object event with final validated object - Full SSE event schema: session, partial-object, object, result, done - Uses --json-schema CLI flag for constrained decoding Closes #67 * fix(gateway): improve error handling and validation in stream-object - Add logging for unexpected errors in partial JSON parsing - Distinguish interrupts from real errors in buffer processing - Send error event when structured_output missing and JSON parse fails - Send error event for internal stream processing errors instead of re-throwing - Add JSON Schema validation requiring valid schema keywords - Apply JSON Schema validation to generateObjectRequest as well Tests: - Add test for buffer processing on CLI close - Add test for non-JSON line handling - Add test for unparseable JSON fallback case - Add test for stdin.end() verification - Add tests for JSON Schema validation * feat(typescript-sdk): add streamObject() for streaming JSON objects Add streamObject() function to the TypeScript SDK that consumes the gateway's /stream-object endpoint: - Stream partial objects via partialObjectStream (async iterable) - Get final validated object via object promise - Session ID resolves early, usage resolves at end - Zod schema validation for partial objects (best-effort) and final object (strict) - Comprehensive test coverage (17 tests) - Example usage in examples/stream-object.ts Part of #67 * fix(gateway): use prompt injection for stream-object to enable real partial streaming The --json-schema flag doesn't stream JSON tokens incrementally - it streams natural language text and only provides the JSON at the end in structured_output. This change uses prompt injection instead: - Inject schema into prompt with JSON output instructions - Add JSON generator system prompt - Parse accumulated text with partial-json library - Add parseJsonResponse() fallback for markdown/text wrapping Workaround for: anthropics/claude-code#15511 * fix(example): show user-visible updates and debug info in stream-object - Number updates based on meaningful changes (day count) shown to user - Track total stream updates from gateway separately - Display debug info at the bottom with both counts and token usage * fix(stream-object): improve error handling and add comprehensive tests Addresses review feedback from PR review agents: Gateway improvements: - Fix silent failure in close handler catch block (now logs and emits error) - Add diagnostic info to parseJsonResponse errors (tracks all parse attempts) - Add logging for content_block_delta events missing text field - Improve cleanup function with try/catch for kill(), SIGKILL escalation logging - Simplify buildStreamObjectArgs to return string[] (remove unused values) - Update endpoint docstring with precise streaming limitation explanation SDK improvements: - Use StreamObjectOptions<T> interface in function signature (was inline type) - Optimize isCriticalEvent check with const Set instead of array New tests: - Empty response handling (CLI returns no JSON) - Markdown code block stripping fallback - JSON embedded in surrounding text extraction - Array schema parsing * test(sdk): add abort signal integration test for streamObject Adds test infrastructure and integration test for aborting streams mid-flight: - createDelayedSSEStream: mock SSE stream with configurable delays - createDelayedMockSSEResponse: wrapper for delayed SSE responses - Test verifies: partial objects received, abort triggered, AbortError thrown * feat(python-sdk): add stream_object() for streaming JSON objects Add stream_object() function to the Python SDK for streaming structured JSON responses via SSE. Follows patterns from TypeScript SDK and stream_text() implementation. Features: - StreamObjectResult[T] dataclass with partial_object_stream async iterator - Pydantic validation: best-effort for partials, strict for final object - Futures for session_id(), usage(), and object() resolution - HTTPObjectStreamContext async context manager for resource cleanup Includes: - 10 comprehensive tests covering success, errors, and edge cases - Example script demonstrating travel itinerary streaming - NO_OBJECT error code for incomplete streams * chore(ts-sdk): update stream-object example to 3-day itinerary Align TypeScript SDK example with Python SDK example for parity. Both now generate a 3-day Tokyo itinerary with 2-3 activities per day. * fix(stream-object): address PR review feedback Improvements from comprehensive PR review: Python SDK: - Add T = TypeVar("T", bound=BaseModel) for better type safety - Make SSE event models frozen with ConfigDict - Refine SSE event type annotations (dict[str, Any] | None) - Add explicit httpx exception handling (ReadTimeout, RemoteProtocolError) - Improve docstrings documenting validation rejection - Add cancellation mid-stream test (parity with TypeScript abort test) TypeScript SDK: - Add @throws JSDoc annotations for validation errors - Clarify partial object comments - Document object promise rejection on validation failure Gateway: - Fix silent failure: parseJsonResponse now returns extraction strategy - Emit warning event when fallback extraction is used (markdown-block, etc.) - Fix silent failure: emit warning event on buffer data loss (clean CLI exit) - Add streamObject integration tests All tests passing (204 total) * test(stream-object): add timeout and connection error tests - Add timeout behavior test for TypeScript SDK streamObject - Add mid-stream connection error test for TypeScript SDK - Add connection/timeout/protocol error tests for Python SDK - Fix Python SDK to wrap httpx exceptions during request phase * docs: regenerate OpenAPI spec for stream-object endpoint
1 parent 1e14a82 commit 9ed751e

File tree

30 files changed

+4399
-29
lines changed

30 files changed

+4399
-29
lines changed

‎bun.lock‎

Lines changed: 5 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/openapi.yaml‎

Lines changed: 206 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,36 @@ components:
6363
format: email
6464
schema:
6565
type: object
66-
additionalProperties: {}
66+
properties:
67+
type:
68+
type: string
69+
enum: &a1
70+
- object
71+
- array
72+
- string
73+
- number
74+
- integer
75+
- boolean
76+
- "null"
77+
properties:
78+
type: object
79+
additionalProperties: {}
80+
items: {}
81+
$ref:
82+
type: string
83+
allOf:
84+
type: array
85+
items: {}
86+
anyOf:
87+
type: array
88+
items: {}
89+
oneOf:
90+
type: array
91+
items: {}
92+
enum:
93+
type: array
94+
items: {}
95+
const: {}
6796
maxTokens:
6897
type: integer
6998
exclusiveMinimum: 0
@@ -86,6 +115,48 @@ components:
86115
format: email
87116
required:
88117
- prompt
118+
StreamObjectRequest:
119+
type: object
120+
properties:
121+
system:
122+
type: string
123+
prompt:
124+
type: string
125+
sessionId:
126+
type: string
127+
model:
128+
type: string
129+
userEmail:
130+
type: string
131+
format: email
132+
schema:
133+
type: object
134+
properties:
135+
type:
136+
type: string
137+
enum: *a1
138+
properties:
139+
type: object
140+
additionalProperties: {}
141+
items: {}
142+
$ref:
143+
type: string
144+
allOf:
145+
type: array
146+
items: {}
147+
anyOf:
148+
type: array
149+
items: {}
150+
oneOf:
151+
type: array
152+
items: {}
153+
enum:
154+
type: array
155+
items: {}
156+
const: {}
157+
required:
158+
- prompt
159+
- schema
89160
UsageInfo:
90161
type: object
91162
properties:
@@ -164,7 +235,7 @@ components:
164235
type: string
165236
code:
166237
type: string
167-
enum: &a3
238+
enum: &a4
168239
- VALIDATION_ERROR
169240
- INTERNAL_ERROR
170241
- UNKNOWN_ERROR
@@ -183,12 +254,12 @@ components:
183254
properties:
184255
status:
185256
type: string
186-
enum: &a1
257+
enum: &a2
187258
- healthy
188259
- unhealthy
189260
claudeCli:
190261
type: string
191-
enum: &a2
262+
enum: &a3
192263
- available
193264
- unavailable
194265
timestamp:
@@ -247,10 +318,10 @@ paths:
247318
properties:
248319
status:
249320
type: string
250-
enum: *a1
321+
enum: *a2
251322
claudeCli:
252323
type: string
253-
enum: *a2
324+
enum: *a3
254325
timestamp:
255326
type: string
256327
error:
@@ -298,10 +369,10 @@ paths:
298369
properties:
299370
status:
300371
type: string
301-
enum: *a1
372+
enum: *a2
302373
claudeCli:
303374
type: string
304-
enum: *a2
375+
enum: *a3
305376
timestamp:
306377
type: string
307378
error:
@@ -439,7 +510,7 @@ paths:
439510
type: string
440511
code:
441512
type: string
442-
enum: *a3
513+
enum: *a4
443514
rawText:
444515
type: string
445516
required:
@@ -478,7 +549,7 @@ paths:
478549
type: string
479550
code:
480551
type: string
481-
enum: *a3
552+
enum: *a4
482553
rawText:
483554
type: string
484555
required:
@@ -511,7 +582,29 @@ paths:
511582
format: email
512583
schema:
513584
type: object
514-
additionalProperties: {}
585+
properties:
586+
type:
587+
type: string
588+
enum: *a1
589+
properties:
590+
type: object
591+
additionalProperties: {}
592+
items: {}
593+
$ref:
594+
type: string
595+
allOf:
596+
type: array
597+
items: {}
598+
anyOf:
599+
type: array
600+
items: {}
601+
oneOf:
602+
type: array
603+
items: {}
604+
enum:
605+
type: array
606+
items: {}
607+
const: {}
515608
maxTokens:
516609
type: integer
517610
exclusiveMinimum: 0
@@ -562,7 +655,7 @@ paths:
562655
type: string
563656
code:
564657
type: string
565-
enum: *a3
658+
enum: *a4
566659
rawText:
567660
type: string
568661
required:
@@ -601,7 +694,7 @@ paths:
601694
type: string
602695
code:
603696
type: string
604-
enum: *a3
697+
enum: *a4
605698
rawText:
606699
type: string
607700
required:
@@ -652,7 +745,106 @@ paths:
652745
type: string
653746
code:
654747
type: string
655-
enum: *a3
748+
enum: *a4
749+
rawText:
750+
type: string
751+
required:
752+
- error
753+
- code
754+
"401":
755+
description: Missing authorization header
756+
content:
757+
application/json:
758+
schema:
759+
type: object
760+
properties:
761+
error:
762+
type: string
763+
required:
764+
- error
765+
"403":
766+
description: Invalid API key
767+
content:
768+
application/json:
769+
schema:
770+
type: object
771+
properties:
772+
error:
773+
type: string
774+
required:
775+
- error
776+
/stream-object:
777+
post:
778+
summary: Stream structured JSON objects via SSE
779+
description: "Streams partial JSON objects as they're generated using Server-Sent Events (SSE). Provides real-time streaming of structured data with partial object parsing. Event types: session (session ID), partial-object (partial JSON with parsed object), object (final validated object), result (final usage stats), error (errors), done (stream complete)."
780+
tags:
781+
- Streaming
782+
security:
783+
- BearerAuth: []
784+
requestBody:
785+
content:
786+
application/json:
787+
schema:
788+
type: object
789+
properties:
790+
system:
791+
type: string
792+
prompt:
793+
type: string
794+
sessionId:
795+
type: string
796+
model:
797+
type: string
798+
userEmail:
799+
type: string
800+
format: email
801+
schema:
802+
type: object
803+
properties:
804+
type:
805+
type: string
806+
enum: *a1
807+
properties:
808+
type: object
809+
additionalProperties: {}
810+
items: {}
811+
$ref:
812+
type: string
813+
allOf:
814+
type: array
815+
items: {}
816+
anyOf:
817+
type: array
818+
items: {}
819+
oneOf:
820+
type: array
821+
items: {}
822+
enum:
823+
type: array
824+
items: {}
825+
const: {}
826+
required:
827+
- prompt
828+
- schema
829+
responses:
830+
"200":
831+
description: SSE stream with real-time partial JSON objects
832+
content:
833+
text/event-stream:
834+
schema:
835+
type: string
836+
"400":
837+
description: Validation error
838+
content:
839+
application/json:
840+
schema:
841+
type: object
842+
properties:
843+
error:
844+
type: string
845+
code:
846+
type: string
847+
enum: *a4
656848
rawText:
657849
type: string
658850
required:

‎packages/gateway/__tests__/helpers.ts‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,7 @@ export function createStreamResultMessage(
121121
overrides: Partial<{
122122
session_id: string;
123123
usage: { input_tokens: number; output_tokens: number };
124+
structured_output?: unknown;
124125
}> = {},
125126
): string {
126127
return JSON.stringify({
@@ -134,6 +135,23 @@ export function createStreamResultMessage(
134135
});
135136
}
136137

138+
/**
139+
* Creates a stream_event message with content_block_delta (for streaming partial text).
140+
* This simulates the output from --include-partial-messages flag.
141+
*/
142+
export function createStreamEventDelta(text: string): string {
143+
return JSON.stringify({
144+
type: "stream_event",
145+
event: {
146+
type: "content_block_delta",
147+
delta: {
148+
type: "text_delta",
149+
text,
150+
},
151+
},
152+
});
153+
}
154+
137155
// =============================================================================
138156
// SSE Helpers
139157
// =============================================================================

0 commit comments

Comments
 (0)