Commit 2b1fe5d
committed
perf: true SSE streaming pass-through and reduced lock contention
Replace buffered collect-then-respond streaming with channel-backed
incremental SSE forwarding for both OpenAI and Responses API endpoints.
Reduce write lock hold times by performing token refresh and file I/O
outside the lock. Increase default concurrency (1→10) and reduce minimum
request interval (500ms→50ms).1 parent e224f65 commit 2b1fe5d
2 files changed
+773
-549
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
206 | 206 | | |
207 | 207 | | |
208 | 208 | | |
209 | | - | |
| 209 | + | |
210 | 210 | | |
211 | 211 | | |
212 | | - | |
| 212 | + | |
213 | 213 | | |
214 | 214 | | |
215 | 215 | | |
| |||
223 | 223 | | |
224 | 224 | | |
225 | 225 | | |
226 | | - | |
| 226 | + | |
227 | 227 | | |
228 | 228 | | |
229 | 229 | | |
230 | | - | |
| 230 | + | |
231 | 231 | | |
232 | 232 | | |
233 | 233 | | |
| |||
0 commit comments