ESP32 with async socket to communicate between multiple clients

[rarefeen]

Hi, I've been trying to communicate between esp32's using tcp socket using uasyncio in micropython. but so far I've failed when I do more jobs with the uasyncio tasks.
is it limited to the firmware that I cannot do it for a reason?
or is there any other way?
[i am using uasyncio to get data from sensors too]

[bixb922]

There is no practical limit for creating asyncio tasks. The limit seems to be the available RAM, since each task needs a bit of RAM. I don't know the real limit, but I have run 100 concurrent asyncio tasks on a ESP32 with no problems at all. What problems are you experiencing? Can you show the code that fails?

[Emmanuel789]

But socket.recv() itself is blocking call. So how to deal with reading data then?

[karfas]

Not true. The Linux manpage says

If no messages are available at the socket, the receive calls wait for a message to arrive, unless the socket is nonblocking.

I am pretty sure the micropython sockets behave in a similar way.
Did you look into the Server implementation in uasyncio/stream.py ?

[rarefeen]

@karfas thanks for the stream link. I was using normal socket connection before so the problem was occuring. Now I'm using uasncio start server to create a server. currently It seems to resolve one issue of I can use multiple asyncio jobs while waiting for new connection.
and to resolve the connected client which wait for a message, I think it waits until there's message in the socket connection. I have not seen how to resolve it yet, maybe use timer to check or there is any timeout function that you know of?

[bixb922]

Perhaps this may help, if you haven't seen this:
https://github.com/peterhinch/micropython-async/blob/master/v3/docs/TUTORIAL.md
See section 7.6

[peterhinch]

You might also look at asynchronous MQTT which uses nonblocking sockets with uasyncio.

[karfas]

@rarefeen: There is more than one way to do it.

As far as I understand the Server class, uasyncio.start_server() creates

• a listener task
• one task for each accepted connection (the callback).

a) You could steal(maybe inherit) the general concept of Server, but check for available data yourself using select.poll() .

b) Nothing stops you from creating another task from inside the callback and cancel the "connected" tasks. Asyncio tasks are cheap.
E.g. (Not sure I got this completely correct (and this will need an exception handler for the cancel)).

async def callback(...)
    data_handler = asyncio.create_task(...) # runs in the background, but not longer than 0.5s
    await asyncio.sleep_ms(500)
    asyncio.cancel(data_handler)
    await data_handler

[rarefeen]

@karfas b is an interesting approach. One of the issue that it has is if any incoming data suppose to come then wouldn't it cause issue?
I also faced such issue that since this has no timeout, it get stuck if I try to read bytes and no incoming data is coming from clients. I am looking into workaround for timeout or something similar now.

[karfas]

@rarefeen: Please check the docs regarding select.poll() and the asyncio tutorial.

One of the issue that it has is if any incoming data suppose to come then wouldn't it cause issue?

When you use the Server class from Stream.py, Incoming data will be processed by a handler function you need to create. The handler function is given as parameter to Server.
Where is the issue here ?

it get stuck if I try to read bytes and no incoming data is coming from clients

Nothing should "get stuck" if you use the Stream class in an asyncio task, as this handles the socket in a non-blocking way using select.poll(). When nothing comes in, the task will sit there ("stuck") and wait for more data, yes, but your other async tasks will be running. This is the main reason for the existance of uasyncio/stream.py.

The interface of Stream is obviously aimed at common TCP protocols (e.g. HTTP), where timeouts+errors are handled by TCP.
When you need something non-standard (e.g. disconnect after a few seconds without traffic), you will need to create your own functions to handle this (either using select.poll() or using another async task as a kind of watchdog).
See e.g. https://github.com/peterhinch/micropython-mqtt/blob/master/mqtt_as/mqtt_as_timeout.py for an example.

[rarefeen]

@karfas Thank you, I will look into select.poll().

writer.write(b"hey")
 await writer.drain()
 await asyncio.sleep(0.2)
 writer.close()
 await writer.wait_closed()
 writer.write(b"hey")

according to the documents my last line should give me OSERROR.
but it doesn't, I tested with both serverside callback code and client side open connection code.
the connection still seems to be kept open!!!!

[peterhinch]

I suggest you study the source. The last line will not raise an error because it doesn't access the socket, simply buffering the data. Subsequently issuing

await writer.drain()

probably will, because the socket is closed.

Re uasyncio and sockets there is a fundamental choice to be made. Either use nonblocking sockets or use blocking sockets with select.polll.