Skip to content

Commit 4e33e3f

Browse files
committed
Add close() method to asyncio.StreamReader
When creating a sub-process using `asyncio.create_subprocess_exec()`, it returns a `Process` instance that has a `stdout` property. This property is intended to be an asyncio version of the `stdout` property of the `Popen` instance from the `subprocess` module. An important aspect of `Popen.stdout` property is that you can close it. This is a signal to the sub-process that is generating output that it should cleanly terminate. This is a common pattern in processes used in shell pipelines. Indeed, the object located at `Popen.stdout` has a `close()` method. This pattern is demonstrated below: ```python import subprocess proc = subprocess.Popen(["yes"], stdout=subprocess.PIPE) # start subprocess data = proc.stdout.read(4096) # get data proc.stdout.close() # signal to process to cleanly shutdown proc.wait() # wait for shutdown ``` Unfortunately this pattern cannot be reproduced easily with the `stdout` property of the `Process` instance returned from `asyncio.create_subprocess_exec()` because `stdout` is an instance of `StreamReader` which does not have the `close()` method. This change adds a `close()` method to the `StreamReader` class so that asyncio version of the `subprocess` module may support this pattern of managing sub-processes. This change is consistent with the asyncio ecosystem as the companion `StreamWriter` class already has a `close()` method, along with other methods that expose its inner "transport" object. It's also trivial to implement, since it's essentially a wrapper method around the inner transport object's `close()` method.
1 parent c4d37ee commit 4e33e3f

File tree

5 files changed

+41
-0
lines changed

5 files changed

+41
-0
lines changed

Doc/library/asyncio-stream.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,10 @@ StreamReader
290290
Return ``True`` if the buffer is empty and :meth:`feed_eof`
291291
was called.
292292

293+
.. method:: close()
294+
295+
Invoke ``close()`` on the underlying transport (if one exists).
296+
293297

294298
StreamWriter
295299
============

Lib/asyncio/streams.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -465,6 +465,10 @@ def set_exception(self, exc):
465465
if not waiter.cancelled():
466466
waiter.set_exception(exc)
467467

468+
def close(self):
469+
if self._transport is not None:
470+
self._transport.close()
471+
468472
def _wakeup_waiter(self):
469473
"""Wakeup read*() functions waiting for data or EOF."""
470474
waiter = self._waiter

Lib/test/test_asyncio/test_subprocess.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,17 @@
3535
'data = sys.stdin.buffer.read()',
3636
'sys.stdout.buffer.write(data)'))]
3737

38+
# Program generating infinite data
39+
PROGRAM_YES = [
40+
sys.executable, '-c', """\
41+
import sys
42+
while True:
43+
try:
44+
sys.stdout.buffer.write(b"y\\n")
45+
except BrokenPipeError:
46+
break
47+
"""]
48+
3849

3950
def tearDownModule():
4051
asyncio._set_event_loop_policy(None)
@@ -879,6 +890,24 @@ async def main():
879890

880891
self.loop.run_until_complete(main())
881892

893+
def test_subprocess_break_pipe(self):
894+
# See https://github.com/python/cpython/issues/130925
895+
async def main():
896+
proc = await asyncio.create_subprocess_exec(*PROGRAM_YES,
897+
stdout=asyncio.subprocess.PIPE,
898+
stderr=asyncio.subprocess.DEVNULL)
899+
try:
900+
# just make sure the program has executed correctly
901+
data = await proc.stdout.readline()
902+
self.assertEqual(data, b"y\n")
903+
finally:
904+
# we are testing that the following method exists and
905+
# has the intended effect of signaling the sub-process to terminate
906+
proc.stdout.close()
907+
await proc.wait()
908+
909+
self.loop.run_until_complete(main())
910+
882911

883912
if sys.platform != 'win32':
884913
# Unix

Misc/ACKS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -836,6 +836,7 @@ Roberto Hueso Gomez
836836
Jim Hugunin
837837
Greg Humphreys
838838
Chris Hunt
839+
Rian Hunter
839840
Eric Huss
840841
Nehal Hussain
841842
Taihyun Hwang
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
:func:`asyncio.StreamReader.close` now exists so that it's possible to
2+
signal to sub-processes executed via :func:`asyncio.create_subprocess_exec`
3+
that they may cease generating output and exit cleanly.

0 commit comments

Comments
 (0)