AbstractEventLoop.stop() doesn't stop if nothing has been executed

[electroglyph]

Bug report

Bug description:

import asyncio
from threading import Thread
import time
loop = asyncio.new_event_loop()
def example():
    print('starting loop...')
    loop.run_forever()
async def test():
    Thread(None, example).start()
asyncio.run(test())
time.sleep(1)
loop.stop()
print('hang...')

This hangs forever. I would expect it to stop. If I add any task to the loop first, then it stops. I have an async threadpool for no-GIL, and to stop it I need to add dummy task to each loop on stop. It seems like a bug, is it a bug?

CPython versions tested on:

3.12, 3.13

Operating systems tested on:

Windows

[qtrcipher]

The problem appears to be related to how the event loop is being stopped from outside the thread where it's running. To understand this better, let's look at the relevant parts of the asyncio documentation:

startLine: 128
endLine: 139

This section explains that when stop() is called while run_forever() is running, the loop will run the current batch of callbacks and then exit. However, in your case, the loop doesn't seem to be exiting as expected.

The reason for this behavior might be that when loop.stop() is called from outside the thread where the loop is running, it doesn't immediately interrupt the loop's execution. Instead, it sets a flag that the loop checks periodically.

To address this issue, we can modify the code to ensure that the loop has a chance to process the stop request. Here's a potential solution:

import asyncio
from threading import Thread
import time


def run_event_loop(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()

async def stop_loop(loop):
    loop.stop()
    while loop.is_running():
        await asyncio.sleep(0.1)

async def main():
    loop = asyncio.new_event_loop()
    thread = Thread(target=run_event_loop, args=(loop,))
    thread.start()

    # Give the loop a chance to start
    await asyncio.sleep(0.5)

    # Stop the loop
    await stop_loop(loop)
    
    thread.join()
    print('Loop stopped successfully')

if __name__ == '__main__':
    asyncio.run(main())

In this modified version:

1. We create a separate function run_event_loop to run the event loop in the thread.
2. We use asyncio.set_event_loop(loop) to ensure the correct loop is used in the thread.
3. We create an async stop_loop function that stops the loop and waits for it to finish.
4. In the main function, we start the thread, wait a bit for it to start, then stop the loop.
5. We use thread.join() to wait for the thread to finish.

This approach should reliably stop the event loop without hanging. The key is to stop the loop from within an async context and wait for it to finish stopping.

As for whether this is a bug in asyncio, it's more of a tricky behavior when dealing with event loops across threads. The asyncio documentation could perhaps be clearer about the potential issues when stopping a loop from a different thread.

[electroglyph]

The problem appears to be related to how the event loop is being stopped from outside the thread where it's running. To understand this better, let's look at the relevant parts of the asyncio documentation:

startLine: 128
endLine: 139

This section explains that when stop() is called while run_forever() is running, the loop will run the current batch of callbacks and then exit. However, in your case, the loop doesn't seem to be exiting as expected.

The reason for this behavior might be that when loop.stop() is called from outside the thread where the loop is running, it doesn't immediately interrupt the loop's execution. Instead, it sets a flag that the loop checks periodically.

To address this issue, we can modify the code to ensure that the loop has a chance to process the stop request. Here's a potential solution:

import asyncio
from threading import Thread
import time


def run_event_loop(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()

async def stop_loop(loop):
    loop.stop()
    while loop.is_running():
        await asyncio.sleep(0.1)

async def main():
    loop = asyncio.new_event_loop()
    thread = Thread(target=run_event_loop, args=(loop,))
    thread.start()

    # Give the loop a chance to start
    await asyncio.sleep(0.5)

    # Stop the loop
    await stop_loop(loop)
    
    thread.join()
    print('Loop stopped successfully')

if __name__ == '__main__':
    asyncio.run(main())

In this modified version:

1. We create a separate function run_event_loop to run the event loop in the thread.

2. We use asyncio.set_event_loop(loop) to ensure the correct loop is used in the thread.

3. We create an async stop_loop function that stops the loop and waits for it to finish.

4. In the main function, we start the thread, wait a bit for it to start, then stop the loop.

5. We use thread.join() to wait for the thread to finish.

This approach should reliably stop the event loop without hanging. The key is to stop the loop from within an async context and wait for it to finish stopping.

As for whether this is a bug in asyncio, it's more of a tricky behavior when dealing with event loops across threads. The asyncio documentation could perhaps be clearer about the potential issues when stopping a loop from a different thread.

i'm not sure that blindly posting AI replies is actually helpful. that code hangs forever for me.

[graingert]

You need to use call_soon_threadsafe to wake up the loop:

Also it's better to always use asyncio.run to start a loop and avoid new_event_loop, and loop.run_forever and loop.stop. They are low level primativtes that are for asyncio.run to call

import asyncio
import concurrent.futures

stop = asyncio.Event()
loop_fut = concurrent.futures.Future()

async def main():
    loop_fut.set_result(asyncio.get_running_loop())
    await stop.wait()

with concurrent.futures.ThreadPoolExecutor(1) as tpe:
    for f in concurrent.futures.as_completed([loop_fut, tpe.submit(asyncio.run, main())]):
        loop = f.result()
        if f is not loop_fut:
            break
        loop.call_soon_threadsafe(stop.set)

There's a similar implementation available in anyio.from_thread.create_blocking_portal that does all this for you:

https://anyio.readthedocs.io/en/stable/threads.html#calling-asynchronous-code-from-an-external-thread

[electroglyph]

i've implemented my own threadpool, hence why i'm using loops. it works great, minus the unused loops refusing to stop. i can kill the threads or use dummy coros to make the loops stoppable, but....shouldn't the loop stop if it's just sitting there not doing anything and i've politely asked it to stop?

[graingert]

i've implemented my own threadpool, hence why i'm using loops.

You don't need to be calling new_event_loop - you can use asyncio.run

it works great, minus the unused loops refusing to stop. i can kill the threads or use dummy coros to make the loops stoppable, but....shouldn't the loop stop if it's just sitting there not doing anything and i've politely asked it to stop?

The loop isn't doing nothing, it's blocked waiting for events on the self._ssock part of the waker socketpair, so the only way to wake it up is with call_soon_threadsafe

It's best not to call loop.stop as there's several phases of event loop teardown that are missed if you do this, this is why you should use asyncio.run and wait on an asyncio.Event

[electroglyph]

okay, thanks for the advice. i'll refactor and see how things compare.