-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Fix infinite iter_chunks()
on empty response
#11397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -30,6 +30,7 @@ | |
"EofStream", | ||
"StreamReader", | ||
"DataQueue", | ||
"empty_stream_reader", | ||
) | ||
|
||
_T = TypeVar("_T") | ||
|
@@ -604,6 +605,18 @@ def read_nowait(self, n: int = -1) -> bytes: | |
EMPTY_PAYLOAD: Final[StreamReader] = EmptyStreamReader() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We'll probably want a followup PR to remove EMPTY_PAYLOAD too, without backporting. |
||
|
||
|
||
def empty_stream_reader() -> "EmptyStreamReader": | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a point to having a function here, and not just importing EmptyStreamReader? |
||
"""Create a fresh EmptyStreamReader instance. | ||
|
||
This function should be used instead of the global EMPTY_PAYLOAD | ||
to avoid state sharing issues between requests. | ||
|
||
Returns: | ||
A new EmptyStreamReader instance. | ||
""" | ||
return EmptyStreamReader() | ||
|
||
|
||
class DataQueue(Generic[_T]): | ||
"""DataQueue is a general-purpose blocking queue with one reader.""" | ||
|
||
|
Original file line number | Diff line number | Diff line change | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -1131,6 +1131,49 @@ async def test_empty_stream_reader_iter_chunks() -> None: | |||||||||||||||||||
await iter_chunks.__anext__() | ||||||||||||||||||||
|
||||||||||||||||||||
|
||||||||||||||||||||
async def test_empty_stream_reader_function() -> None: | ||||||||||||||||||||
"""Test the empty_stream_reader() function creates fresh instances.""" | ||||||||||||||||||||
reader1 = streams.empty_stream_reader() | ||||||||||||||||||||
reader2 = streams.empty_stream_reader() | ||||||||||||||||||||
|
||||||||||||||||||||
# Should be different instances | ||||||||||||||||||||
assert reader1 is not reader2 | ||||||||||||||||||||
assert reader1 is not streams.EMPTY_PAYLOAD | ||||||||||||||||||||
assert reader2 is not streams.EMPTY_PAYLOAD | ||||||||||||||||||||
|
||||||||||||||||||||
# Both should start with fresh state | ||||||||||||||||||||
assert reader1._read_eof_chunk is False | ||||||||||||||||||||
assert reader2._read_eof_chunk is False | ||||||||||||||||||||
|
||||||||||||||||||||
|
||||||||||||||||||||
async def test_empty_stream_reader_no_state_sharing() -> None: | ||||||||||||||||||||
"""Test that fresh EmptyStreamReader instances don't share state.""" | ||||||||||||||||||||
reader1 = streams.empty_stream_reader() | ||||||||||||||||||||
reader2 = streams.empty_stream_reader() | ||||||||||||||||||||
|
||||||||||||||||||||
# Use reader1 - this will modify its internal state | ||||||||||||||||||||
chunks = [] | ||||||||||||||||||||
async for chunk in reader1.iter_chunks(): | ||||||||||||||||||||
chunks.append(chunk) | ||||||||||||||||||||
if len(chunks) > 5: # Safety break | ||||||||||||||||||||
break | ||||||||||||||||||||
Comment on lines
+1156
to
+1159
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Coverage doesn't seem happy with these. Not sure if it's just confused, as it seems to suggest that the break line is reached without any of the above lines being reached... |
||||||||||||||||||||
|
||||||||||||||||||||
assert len(chunks) == 0 # Should terminate normally with no chunks | ||||||||||||||||||||
Comment on lines
+1155
to
+1161
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If the expected behaviour is that we don't iterate here, then we should do something like:
Suggested change
|
||||||||||||||||||||
assert reader1._read_eof_chunk is True # State should be modified | ||||||||||||||||||||
|
||||||||||||||||||||
# reader2 should still have fresh state and work correctly | ||||||||||||||||||||
assert reader2._read_eof_chunk is False | ||||||||||||||||||||
|
||||||||||||||||||||
chunks = [] | ||||||||||||||||||||
async for chunk in reader2.iter_chunks(): | ||||||||||||||||||||
chunks.append(chunk) | ||||||||||||||||||||
if len(chunks) > 5: # Safety break | ||||||||||||||||||||
break | ||||||||||||||||||||
|
||||||||||||||||||||
assert len(chunks) == 0 # Should also terminate normally with no chunks | ||||||||||||||||||||
assert reader2._read_eof_chunk is True | ||||||||||||||||||||
|
||||||||||||||||||||
|
||||||||||||||||||||
@pytest.fixture | ||||||||||||||||||||
async def buffer(loop: asyncio.AbstractEventLoop) -> streams.DataQueue[bytes]: | ||||||||||||||||||||
return streams.DataQueue(loop) | ||||||||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This one is presumably also unused now.