Document tradeoffs

[ulope]

Please add some explanation on what tradeoffs one takes on when using this library.

For example from very briefly looking at the code it seems that using this forces pure Python asyncio code to used.
Is there a performance (or other) impact?

How can things break with nested loops?

Thanks.

[blazespinnaker]

Yeah, I'd second that. This library is very widely referenced across stackoverflow, @erdewit

After using lua coroutines, I have to say the asyncio folks really need to rethink things.

[blazespinnaker]

Interesting rebuttal to dynamic coroutines: https://mail.python.org/pipermail/python-dev/2015-April/139695.html

The issue boils down to the fact that when you allow the event_loop to be accessed wherever, the author feels you start to lose grasp of how execution control is being passed. But I'm not entirely sure the control flow is always obvious with lexical coroutines either.

Coroutines, at least for me, enable the benefits of concurrency (mostly with IO, but there are others) while reducing race conditions and increasing performance. Like threads, I never expected to know exactly what code is executing at any particular time. For those that can follow their coroutine code flow perfectly, that's great, but I severely doubt that is the common case among those responsible for doing actual real world work, especially as third party libraries are integrated into larger projects.

So, really I think the issue is less about being able step through all the code, and more about expectations of safety, fairness and starvation.

This library doesn't use threads, so I don't see how safety could be impacted. However, with coroutine the issue of fairness can be tricky, especially when its usage becomes dynamic like lua. And on the issue of fairness, I don't think this patch quite works. But let me know if my code is making the wrong assumptions about what re-entrant means.

Try these two out:

import nest_asyncio
nest_asyncio.apply()
import asyncio

async def echo(x):
    for i in range(0,5):
        print(x,i)
        sleep(1)

def testNested():
    for i in range(0,5):
        asyncio.get_event_loop().create_task(echo(i))
    inner()

def inner():
    for i in range(5,10):
        asyncio.get_event_loop().create_task(echo(i))
    sleep(20)

def sleep(z):
    asyncio.run(asyncio.sleep(z))

testNested()

import asyncio

async def echoUnnested(x):
    for i in range(0,5):
        print(x,i)
        await asyncio.sleep(1)

async def testUnnested():
   for i in range(0,10):
        asyncio.get_event_loop().create_task(echoUnnested(i))
   await asyncio.sleep(20)

asyncio.run(testUnnested())  

The latter unpatched approach runs as expected.

Unfortunately, something very unexpected is going on with the former. Looking at the patch code, run is supposed to use the already created event_loop rather making a new one. But the scheduled tasks are clearly not being executed concurrently.

[blazespinnaker]

That all said, I would really like dynamic / re-entrant coroutines. Without it, supporting asyncio will require significant refactoring of a lot of libraries, and for those trying to just get things done, that seems more important than philosophical discussions on what levels of theoretical code complexity are acceptable.

..which looks like gevent fits the bill.

[erdewit]

For example from very briefly looking at the code it seems that using this forces pure Python asyncio code to used.
Is there a performance (or other) impact?

The performance impact depends entirely on the situation. It's negligible for what I use it for. With PyPy the nested variant can actually be quite a bit faster than the unnested variant for some reason. Best thing is to measure it for your own code.

How can things break with nested loops?

I don't know really. This library gets used in ways that I'd never expected and if that leads to any issues then they would supposedly be filed here in the issue tracker.

[blazespinnaker]

@erdewit I think most people are using as a hack to get things to work in jupyter and aren't measuring the fairness / performance. Is there something wrong with the code above? As far as I can tell the hack doesn't work. Which isn't surprising, as I dug into the code asyncio is really not meant to be nested.

I also looked at another repo that you were using asyncio in nested manner, and there are bugs, quite possibly because of this contradicting use.

I'd suggest recommending that users look at gevent. It is meant to be nested and is similar to the lua approach to coroutines.

Also, I wouldn't just go by lack of reports. It's like the sherlock holmes mystery when the dogs didn't bark.

[erdewit]

@blazespinnaker
Your nested example uses coroutines that never await. You need await as a kind of portal for the flow of control to jump into other coroutines (this is just how Python works). Without it the coroutines can be scheduled concurrently but will execute serially. Exception here is the first iteration of the coroutines, which is considered ready by asyncio and executed in the same iteration of the event loop.

Or in other words, to get cooperative concurrency to work, the coroutines must be cooperative.

[blazespinnaker]

@erdewit Thanks. How would you write the code then? With a custom run routine? I was going to try that, but as I looked into the asyncio code itself I saw too many stumbling blocks. Many which would likely increase in number as asyncio is uprgaded in the stdlib.

My sleep above uses run, which I assumed (wrongly I guess) that it would turn control over to the event_loop similar to what happens in lua / gevent.

I get what you're trying to do, but I think the forces of asyncio are going to work against this. I'd use gevent which is specifically designed to make this use case work.

[erdewit]

My sleep above uses run, which I assumed (wrongly I guess) that it would turn control over to the event_loop similar to what happens in lua / gevent.

The sleep does actually let the event loop run, which can then run other cooperative tasks. In your example there are no other cooperative tasks (there's nobody awaiting) so it appears nothing happens.

The coroutines can be made cooperative by using await asyncio.sleep(1) instread of sleep, but then you don't need any nesting to begin with.

To be clear, this library does not solve Python's colored function problem in the way gevent (or lua) has it solved.

[blazespinnaker]

Well, there are no other tasks only if we're not using the same event_loop where the previous ones were created. I guess there is a new event loop in there somewhere I missed. If so, that can lead to serious starvation.

[erdewit]

No, there's only one event loop. There's also only one queue for the scheduled callbacks. It has time/insert-order priority, so every task gets its turn - no starvation there.

[blazespinnaker]

Hmm, we seem to be going back and forth a bit here. If you look at the code above, create_task will schedule a task to run soon. Before sleep is executed in inner(), there should be about 10 tasks scheduled for execution on the event loop. Is the problem get_event_loop() call in asyncio.get_event_loop().create_task()?

If, as you say above:

"The sleep does actually let the event loop run, which can then run other cooperative tasks. In your example there are no other cooperative tasks (there's nobody awaiting) so it appears nothing happens."

Then the already scheduled tasks should run. I think the confusion here is that you are saying "there are no other cooperative tasks" and I'm not sure what that means, because asyncio.get_event_loop().create_task() should have scheduled 10 on the thread local event loop.

And indeed, when you run the code above, it runs the first 10 tasks very immediately on the first sleep but then starvation occurs when the internal nested sleeps in each task is hit, because they are not giving time / returning control to the thread local event loop during their sleeping period.

Note that I pasted in the patch code when I posted the bug, you can run it after the import asyncio.

[erdewit]

I think the confusion here is that you are saying "there are no other cooperative tasks" and I'm not sure what that means

It's mentioned a few times already, but it means that your coroutines contain no await statement.

await is the magic keyword for yielding and resuming coroutines and that is what makes cooperative concurrency possible. In gevent these yield/resume points are also present but hidden from the user.

[blazespinnaker]

Ah, ok I see the confusion:

"My sleep above uses run, which I assumed (wrongly I guess) that it would turn control over to the event_loop similar to what happens in lua / gevent."

The sleep does actually let the event loop run, which can then run other cooperative tasks. In your example there are no other cooperative tasks (there's nobody awaiting) so it appears nothing happens.

Rather, I think what you meant to say was sleep would have worked if I used await instead of run. The bit about there not being other cooperative tasks threw me as there are tasks scheduled on the event_loop.

As I mentioned, I wrongly assumed that the patch library would allow for nested calls to the async library and use the same event loop. And of course I can't use await in a nested call. Using run, however, leads to the starvation I observed above.

Hopefully this makes sense to us both now.

[blazespinnaker]

fwiw, this causes starvation as well:

async def sleepAsync(x):
    await asyncio.sleep(x)

def sleep(x):
    asyncio.run(sleepAsync(x))

As I mentioned above, I believe that the nested run is not using the same global event loop or at at the very least it's not slicing out time to tasks already scheduled on it.

When I say starvation, you just need to imagine any long running blocking IO being called on a nested run. Any tasks scheduled outside the run won't be fairly executed because the nested run doesn't give them any time.