Asyncio and task.cancel #14027

PatLub85 · 2024-03-04T17:28:34Z

PatLub85
Mar 4, 2024

Following the discussion "Asyncio and while loop #14008", I finalized my program which works but has a bug.
If I click "Right"-"Stop"-"Forward"-"Backward" it's ok
On the other hand, "Right"-"Forward"-"Stop" the hcsr() function loops and it seems that the task.cancel() command of the main_2 function is no longer taken into account.
It's really difficult to follow the different steps of a program asyncio.

main.py

import sys
sys.path.append("/lib/utemplate")
from microdot import Microdot,Response,send_file
from utemplate import Template
import network
import esp
import time
import gc
import uasyncio as asyncio

gc.collect()
esp.osdebug(None)
ssid = 'MicroPython-AP'
password = '123456789'
ip_ap='192.168.4.10'

def wifi_ap():
    ap = network.WLAN(network.AP_IF)
    ap.active(True)
    ap.ifconfig((ip_ap,'255.255.255.0','192.168.4.1','192.168.4.1'))
    ap.config(essid=ssid, password=password)
    while ap.active() == False:
         pass
    print('Active server')
    print(ap.ifconfig())
    return()
        
async def stop():
    print('Stop')
    await asyncio.sleep_ms(10)    

async def left():
    print('Turn left')
    await asyncio.sleep_ms(10)

async def right():
    print('Turn right')
    await asyncio.sleep_ms(10)    

async def forward():
    print('Forward')
    await asyncio.sleep_ms(10)

async def back():
    print('Back')
    await asyncio.sleep_ms(10)
    
async def hcsr():
    while True:
        print('Hcsr04')
        await asyncio.sleep(1)
            
async def main_2(cmd):
    if not task.done():
        print("cancel task")
        task.cancel()
    asyncio.create_task(cmd)
    await asyncio.sleep(0.1)
    
async def main(cmd):
    global task
    asyncio.create_task(cmd)
    task=asyncio.create_task(hcsr())
    await asyncio.sleep(0.1)

wifi_ap()
app = Microdot()
Response.default_content_type = 'text/html'

@app.route('/')
async def index(request):
    return Template('index2.html').render()

@app.route('/reponse',methods = ['POST', 'GET'])
async def commande (request):
    command=request.body.decode('ascii')
    if command=='back' or command=='stop':
        command=globals()[command]
        asyncio.run(main_2(command()))
    else:
        command=globals()[command]
        asyncio.run(main(command()))
    return ("OK")

@app.route('/static/<path:path>')
async def static(request, path):
    if '..' in path:
        return 'Not found', 404
    return send_file('static/' + path)

@app.get('/shutdown')
async def shutdown(request):
    request.app.shutdown()
    return 'The server is shutting down...'

app.run(debug=True)

index2.html (I removed the CSS file)

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Test bouton</title>
  </head>
  <body>
    <h2>CAROBOT</h2>
    <div class="box2" >
      <nav id="toggle">
        <input
          id="switch"
          type="checkbox"
          class="checkbox"
          value="TOGGLE"
          onclick="toggleOnOff()"
        />
        <label for="switch" class="toggle">
          <p>OFF &#160 ON</p>
        </label>
      </nav>
    </div>
    
    <div class="box">
      <button
        id="bouton1"
        class="styling button"
        type="button"
        onclick="toggleButtonSwitch(this.value)"
        value="forward"
      >
        FORWARD
      </button>
      <button
        id="bouton2"
        class="styling button"
        type="button"
        onclick="toggleButtonSwitch(this.value)"
        value="left"
      >
        LEFT
      </button>
      <button
        id="bouton3"
        class="styling button"
        type="button"
        onclick="toggleButtonSwitch(this.value)"
        value="stop"
      >
        STOP
      </button>

      <button
        id="bouton4"
        class="styling button"
        type="button"
        onclick="toggleButtonSwitch(this.value)"
        value="right"
      >
        RIGHT
      </button>
      <button
        id="bouton5"
        class="styling button"
        type="button"
        onclick="toggleButtonSwitch(this.value)"
        value="back"
      >
        BACK
      </button>
    </div>
    <script src="static/js/index2.js"></script>
  </body>
</html>

and index2.js

function toggleOnOff(event) {
  let switchBouton = document.getElementById("switch");
  let toggleValeur = "";
  if (switchBouton.checked) {
    console.log("OFF");
    toggleValeur = "OFF";
    fetch("/shutdown")
  } else {
    console.log("ON");
    toggleValeur = "ON";
    
  }
}
function toggleButtonSwitch(event) {
  console.log(event);
  fetch("/reponse",{
  "method":"POST",
  "body":(event)});
  return;
}

Is there a program like pythontutor (https://pythontutor.com/visualize.html#mode=edit) to help debugging?
Thank you in advance for your help

glenn20 · 2024-03-05T08:12:15Z

glenn20
Mar 5, 2024

I'm not sure if this is the cause of the issues you are seeing, but from a quick read of your code:

@app.route('/reponse',methods = ['POST', 'GET'])
async def commande (request):
    command=request.body.decode('ascii')
    if command=='back' or command=='stop':
        command=globals()[command]
        asyncio.run(main_2(command()))
    else:
        command=globals()[command]
        asyncio.run(main(command()))
    return ("OK")

should be:

@app.route('/reponse',methods = ['POST', 'GET'])
async def commande (request):
    command=request.body.decode('ascii')
    if command=='back' or command=='stop':
        command=globals()[command]
        asyncio.run(main_2(command))
    else:
        command=globals()[command]
        asyncio.run(main(command))
    return ("OK")

Its a little hard to follow without type hints, but it appears that main() and main_2() are expecting awaitables as arguments. By invoking the functions before passing them to main, you are passing them the generator that is returned from "calling" an async function without await-ing it. I confess I am unsure if that can stand-in for the awaitable itself in create_task() in micropython.

1 reply

glenn20 Mar 5, 2024

Whoops - brain fade. Ignore my comment above ...

dpgeorge · 2024-03-05T11:47:08Z

dpgeorge
Mar 5, 2024
Maintainer

It looks like your program is calling asyncio.run() in more than one place. That won't work because there can only be one instance of the event loop (which is started by asyncio.run()).

When you call app.run(debug=True) it will start the asyncio loop by calling asyncio.run(...).

Then in your commande function you should not call asyncio.run(main_2(command())). Instead you either await main_2(...), or asyncio.create_task(main_2(...)). Same for asyncio.run(main(command())).

4 replies

PatLub85 Mar 5, 2024
Author

Hi Damien,
I understood the error of running multiple asyncio.run(). I modified the async def command (request) as you suggested. I think we need to add await
I also added a few prints to check the sequence of tasks and if I do once on the right and once on the left, then stop, I have 'cancel task' but the hcsr() function does not stop

async def main(cmd):
    global task
    asyncio.create_task(cmd)
    print('3')
    task=asyncio.create_task(hcsr())
    print('4')
    await asyncio.sleep(0.1)

@app.route('/reponse',methods = ['POST', 'GET'])
async def commande (request):
    command=request.body.decode('ascii')
    if command=='back' or command=='stop':
        command=globals()[command]
        asyncio.create_task (main_2(command()))
        print('1')
        await asyncio.sleep(0.1)
    else:
        command=globals()[command]
        asyncio.create_task (main(command()))
        print('2')
        await asyncio.sleep(0.1)
    return ("OK")

I get the following sequence

2
3
4
Forward
Hcsr04
POST /reponse 200
Hcsr04
Hcsr04
1
cancel task
Stop
POST /reponse 200
2
3
4
Turn right
Hcsr04
POST /reponse 200
Hcsr04
Hcsr04
Hcsr04
2
3
4
Turn left
Hcsr04
POST /reponse 200
Hcsr04
Hcsr04
Hcsr04
1
cancel task
Stop
POST /reponse 200
Hcsr04
Hcsr04
....

dpgeorge Mar 6, 2024
Maintainer

The problem is that you are creating multiple hcsr tasks. The "Turn right" and then "Turn left" commands both create hcsr tasks (see the sequence of 3 then 4 printed before them), and then the "cancel task" is only cancelling the second `hcsr" task.

You could debug this better by having the hcsr task print itself as part of its message, eg print("Hcsr04", asyncio.current_task()).

PatLub85 Mar 6, 2024
Author

Your explanation is clear and logical.
The next question is how to find these different tasks. I tried to number them like this:

task_number=0
async def main(cmd):
    global task, task_number
    asyncio.create_task(cmd)
    task='task'+str(task_number)
    print ('task',task)
    task=asyncio.create_task(hcsr())
    task_number+=1
    await asyncio.sleep(0.1)

If I do right, then left I have task0 and task1
task is a Task object and we cannot do:

async def main_2(cmd):
    if not task.done():
        for i in range(task_number,-1,-1):
            task+str(i).cancel()

How to find them in main_2?

PatLub85 Mar 6, 2024
Author

I found a solution but tell me if it is pythonic

flag=False
@app.route('/reponse',methods = ['POST', 'GET'])
async def commande (request):
    command=request.body.decode('ascii')
    command=globals()[command]
    asyncio.create_task (main(command()))
    await asyncio.sleep(0.1)
    return ("OK")
async def main(cmd):
    global task
    asyncio.create_task(cmd)
    await asyncio.sleep(0.1)
async def stop():
    global flag,task
    print('Stop')
    if flag:
        task.cancel()
        flag=False
    await asyncio.sleep_ms(10)
async def forward():
    global flag,task
    print('Forward')
    if flag:
        task.cancel()
        flag=False    
    task=asyncio.create_task(hcsr())
    flag=True
    await asyncio.sleep_ms(10)

bixb922 · 2024-03-06T17:57:28Z

bixb922
Mar 6, 2024

I couldn't comment if that's pythonic. However I would structure the tasks another way. While creating and cancelling tasks is very efficient in asyncio, I found it not very clear, because I tend to loose track of which task is running, so some additional code would be needed just for housekeeping of active tasks. Also, cancelling a task is not instantaneous and may lead to having two instances of a Coro running in different tasks, especially if there is some task rundown required. This is one of the very few ways to have race conditions in asyncio, and would need careful attention. Also, there is no need for a "forward" task to run simultaneously with a "backward" task. In fact that would be an error.

So I would create two tasks: one for the movement, and another one for the obstacle detection, both running always (i.e. with a while True:). I'd use queue (see https://github.com/peterhinch/micropython-async/blob/master/v3/primitives/queue.py) to queue commands to the movement task. That way, both the webserver and the obstacle detection task can queue commands to the movement task calling for example queue.put("forward")

The movement task's code would be something like this:

async def movement(queue):
     # do some initialization
    while True:
            command = await queue.get()
            if command == "forward":
               print("forward")
            elif command == "backward":
                 ... etc ...

async def obstacle_detection(queue):
    while True:
        ... detect obstacle ...
        ... queue movement command ....

async def main():
    queue = Queue(maxsize=10)
    asyncio.gather( obstacle_detection(queue), movement(queue) )

asyncio.run(main())

2 replies

PatLub85 Mar 6, 2024
Author

Hi,
thank you for your comments. Indeed I noticed that the program sometimes froze for no reason.
Tomorrow I will try to focus on the queues, but as it will be the first time, I will certainly have other questions

PatLub85 Mar 7, 2024
Author

What I understood:
If the command sent by the server arrives in the main function, I put it in the queue

async def main(cmd):
    queue = Queue(maxsize=10)
    asyncio.gather( hcsr(queue), movement(queue,cmd) )
    await queue.put(cmd)

The function def movement(queue) retrieves the command using the command = await queue.get()

On the other hand, I didn't understand what you mean: queue movement command in obstacle_detection(queue)

bixb922 · 2024-03-07T12:46:41Z

bixb922
Mar 7, 2024

If the command sent by the server arrives in the main function, I put it in the queue

No, that's not what I meant. From your original code I understood that the commands (i.e. move forward, backward) are received by a microdot action. What I propose is that the action queue the command. You will need to make the queuevariable global.

queue = Queue(10)
@app.route('/reponse',methods = ['POST', 'GET'])
async def commande (request):
    command=request.body.decode('ascii')
    if command in ['back', 'stop', 'forward', 'left', 'right']:
        await queue.put(command)
    else:
        print("unknown command", command )
    return ("OK")

Since queue is now a global variable, the main program does not need to initialize it. In the main program, there is no command yet, so you can't pass that to movement()

async def main():
    asyncio.gather( hcsr(queue), movement(queue) )
asyncio.run(main())

The asyncio.gather() creates both tasks and then awaits until both terminate. Since there is a while loop in both tasks, asyncio.gather() will never return. You also could write:

async def main():
    asyncio.create_task( hcsr(queue) )
    asyncio.create_task( movement(queue) )
    await asyncio.sleep(10)
asyncio.run(main())

but then, after 10 seconds the main() program will stop, and all tasks will stop.

On the other hand, I didn't understand what you mean: queue movement command in obstacle_detection(queue)

What I meant is this: in the obstacle detection routine, you will need to change the movement of the robot when an obstacle is detected. This could also be done by queueing commands to the movement task using, for example: await queue.put("stop")

12 replies

bixb922 Mar 8, 2024

Use two different queues and put the same command on both.

Queues, by nature, should have only one consumer.

PatLub85 Mar 9, 2024
Author

Hi,
I added a tail as you recommended, and as expected the 2 functions work.
On the other hand, the hcsr function does not loop
When the robot moves forward, the async def hcsr(queue2) function must constantly test for the presence of an obstacle.

@app.route('/reponse',methods = ['POST', 'GET'])
async def command (request):
    command=request.body.decode('ascii')
    if command in ['back', 'stop', 'forward', 'left', 'right']:
        print('cmd2',command)
        await queue1.put(command)
        await queue2.put(command)

async def hcsr(queue2):
    print('HCSR04')
    while True:
        command = await queue2.get()
        if command == "forward":        
            print('Forward Hcsr04')
            '''if obstacle_detected:
                 await stop()'''
        elif command == "back":
            print("Back Hcsr04")
        await asyncio.sleep(1)
``` 
result:
```
1
movement
HCSR04
cmd2 forward
Movement forward
Movement end
Forward
Forward Hcsr04
```
What am I missing?

bixb922 Mar 9, 2024

There is queue.get_nowait()

But then, the loop in hcsr() must have an await asyncio.sleep_ms(), is it clear why?

PatLub85 Mar 9, 2024
Author

Crash! I'm really sorry but Asyncio is relatively new to me, and this is my first time using the Queue library
Initially the queue is empty and I get the message QueueEmpty.
And I can't do a try/except
except QueueEmpty:
gives NameError: Name 'QueueEmpty' is not defined

bixb922 Mar 9, 2024

from queue import QueueEmpty

The nice thing about Python libraries is that you can normally read the source code rather easily

Asyncio and task.cancel #14027

Uh oh!

Replies: 4 comments · 19 replies

Uh oh!

Uh oh!

Uh oh!

dpgeorge Mar 5, 2024 Maintainer

Uh oh!

PatLub85 Mar 5, 2024 Author

Uh oh!

dpgeorge Mar 6, 2024 Maintainer

Uh oh!

PatLub85 Mar 6, 2024 Author

Uh oh!

PatLub85 Mar 6, 2024 Author

Uh oh!

Uh oh!

PatLub85 Mar 6, 2024 Author

Uh oh!

PatLub85 Mar 7, 2024 Author

Uh oh!

Uh oh!

Uh oh!

PatLub85 Mar 9, 2024 Author

Uh oh!

Uh oh!

PatLub85 Mar 9, 2024 Author

Uh oh!

Uh oh!

Replies: 4 comments 19 replies

dpgeorge
Mar 5, 2024
Maintainer

PatLub85 Mar 5, 2024
Author

dpgeorge Mar 6, 2024
Maintainer

PatLub85 Mar 6, 2024
Author

PatLub85 Mar 6, 2024
Author

PatLub85 Mar 6, 2024
Author

PatLub85 Mar 7, 2024
Author

PatLub85 Mar 9, 2024
Author

PatLub85 Mar 9, 2024
Author