Fix FastAPI SSE streaming compatibility #60
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove anyio.create_task_group() from receive_messages() to fix the RuntimeError "Attempted to exit cancel scope in a different task than it was entered in" when using the SDK with FastAPI's SSE streaming. The issue occurred because FastAPI can move async generators between different asyncio tasks during the streaming lifecycle, which conflicts with anyio's cancel scope tracking. Changes: - Remove task group usage from receive_messages() - Read stderr sequentially after stdout completes - Add test to ensure no task groups are used - Fix existing test expectation for buffer overflow This is a minimal fix that maintains compatibility while solving the core issue. The trade-off is that stderr is read after stdout instead of concurrently, but this is unlikely to cause issues in practice. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Collaborator
Author
|
Fixed linting issues - removed trailing whitespace and added newline at end of file. |
The test was checking source code for specific strings rather than testing behavior. The existing subprocess tests already verify the functionality works correctly.
Add critical safety mechanisms to the sequential stderr reading approach: - 10MB memory limit on stderr collection with truncation message - 30 second timeout protection to prevent hanging - Proper exit code based error detection instead of string matching - Debug logging for stderr on successful runs - Better exception handling for edge cases This addresses potential production issues while maintaining FastAPI compatibility. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
chrislloyd
approved these changes
Jul 2, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes RuntimeError when using the SDK with FastAPI's Server-Sent Events streaming (#4).
Changes
anyio.create_task_group()fromreceive_messages()Technical Details
FastAPI can move async generators between different asyncio tasks during streaming, which conflicts with anyio's requirement that cancel scopes must be entered and exited in the same task.
The solution removes concurrent task creation and reads streams sequentially, eliminating the task boundary crossing issue.
Testing
test_fastapi_streaming_compatibility.pyto verify no task groups are used