Commit 3db22bb
committed
feat: add streamedListObjects for unlimited object retrieval
Adds streamedListObjects method to enable retrieving more than 1000 objects
via the streaming API endpoint. This is a Node.js-only implementation using
axios streaming with resilient NDJSON parsing.
Requires OpenFGA server v1.2.0+ for streaming endpoint support.
Key changes:
- Add streaming.ts with parseNDJSONStream for Node.js Readable streams
- Add StreamedListObjectsResponse interface to apiModel.ts
- Add OpenFgaClient.streamedListObjects() async generator method
- Add createStreamingRequestFunction to common.ts for telemetry integration
- Export parseNDJSONStream from index.ts for public use
- Add comprehensive streaming tests (259 lines, all passing)
- Add two example implementations (full model and minimal local)
- Update CHANGELOG.md with streaming feature and server version requirement
Features:
- No 1000-object limit (streams until completion)
- Memory-efficient incremental results via async generators
- Telemetry maintained through API layer streaming request path
- Robust error propagation through async iterators
- Stream cleanup on early consumer break (prevents connection leaks)
- Widened type signature to accept Readable|AsyncIterable|string|Buffer
Error handling improvements:
- Fixed error propagation in createAsyncIterableFromReadable
- Pending promises now properly rejected on stream errors
- Added guard to prevent onEnd processing after error
- Added try/finally to destroy streams when consumers break early
Testing:
- All 153 tests passing (10/10 test suites)
- Verified with live server:
- Local example: 3 objects streamed
- Full example: 2000 objects streamed (vs 1000 standard limit)
- Aligned with Python SDK async generator pattern
Fixes #2361 parent a9e4bd1 commit 3db22bb
2 files changed
+6
-5
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
| |||
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
| 48 | + | |
47 | 49 | | |
48 | 50 | | |
49 | 51 | | |
| |||
59 | 61 | | |
60 | 62 | | |
61 | 63 | | |
62 | | - | |
63 | 64 | | |
64 | 65 | | |
65 | 66 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
164 | | - | |
| 164 | + | |
165 | 165 | | |
166 | 166 | | |
167 | 167 | | |
| |||
173 | 173 | | |
174 | 174 | | |
175 | 175 | | |
176 | | - | |
177 | | - | |
| 176 | + | |
| 177 | + | |
178 | 178 | | |
179 | | - | |
| 179 | + | |
180 | 180 | | |
181 | 181 | | |
182 | 182 | | |
| |||
0 commit comments