You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
asyncio: cancelling a task awaiting subprocess stdin.wait_closed() cancels the shared close future, then pipe_connection_lost() raises InvalidStateError #162217
For a subprocess created with asyncio.create_subprocess_exec(..., stdin=PIPE), proc.stdin.wait_closed() awaits the protocol's shared _stdin_closed future directly. SubprocessStreamProtocol._get_close_waiter() returns it, and StreamWriter.wait_closed() awaits it without shield(). If the task awaiting wait_closed() is cancelled after proc.stdin.close() but before the pipe's connection_lost callback runs, the cancellation cancels that shared future.
When SubprocessStreamProtocol.pipe_connection_lost(0, None) runs later, it calls self._stdin_closed.set_result(None) on the already-cancelled future. The event loop then logs:
Exception in callback SubprocessStreamProtocol.pipe_connection_lost()
...
asyncio.exceptions.InvalidStateError: invalid state
StreamReaderProtocol guards the equivalent future for sockets (if not self._closed.done(): ...); SubprocessStreamProtocol.pipe_connection_lost() has no such check.
Minimal, deterministic reproducer:
importasyncioimportsysasyncdefmain(mode: str) ->int:
loop=asyncio.get_running_loop()
errors= []
loop.set_exception_handler(lambda_loop, ctx: errors.append(ctx))
proc=awaitasyncio.create_subprocess_exec(
"cat", stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
proc.stdin.close()
asyncdefcloser():
ifmode=="shield":
awaitasyncio.shield(proc.stdin.wait_closed())
else:
awaitproc.stdin.wait_closed()
task=asyncio.create_task(closer())
awaitasyncio.sleep(0) # let closer() reach the await on the close futuretask.cancel() # cancel between close() and pipe_connection_lost()try:
awaittaskexceptasyncio.CancelledError:
passprint("stdin close future:", proc._protocol._stdin_closed)
awaitproc.communicate()
awaitproc.wait()
for_inrange(5):
awaitasyncio.sleep(0)
forctxinerrors:
print("loop exception:", repr(ctx.get("exception")), "|", ctx.get("message"))
return1iferrorselse0if__name__=="__main__":
sys.exit(asyncio.run(main(sys.argv[1] iflen(sys.argv) >1else"plain")))
Output on 3.14.4 (Linux):
$ python3 repro.py
stdin close future: <Future cancelled>
loop exception: InvalidStateError('invalid state') | Exception in callback SubprocessStreamProtocol.pipe_connection_lost()
$ python3 repro.py shield
stdin close future: <Future finished result=None>
The visible impact is an error logged from a loop callback (for example during application shutdown, when a task blocked in wait_closed() gets cancelled). Process I/O is otherwise unaffected. In our case it appeared intermittently in a test suite (about 5 in 2,000 runs under load) until we stopped awaiting wait_closed() on that path.
Possible fixes, either of which avoids the error:
In SubprocessStreamProtocol.pipe_connection_lost(), only call set_result() / set_exception() on _stdin_closed when it is not already done, mirroring StreamReaderProtocol.
Or have wait_closed() (or _get_close_waiter()) shield the shared future, so cancelling one waiter does not cancel it for everyone.
Activity