Pico W, uselect.select and uselect.poll blocking intermittently when timeout specified

[gmrza]

Hi,

I am running into some issues where I am seeing uselect.select and uselect.poll (or ipoll) blocking even when I specify a timeout in the call. This behaviour is intermittent, with occurrences sometimes minutes apart, but generally hours apart.
What is helping as a workaround is to set the socket as non-blocking as well, by setting a timeout on the socket. (As a rule I am setting the timeout on the socket to be 500ms longer than the timeout in my call to poll or select. I then occasionally get an Exception - ETIMEOUT.)

Has anybody else seen this behaviour?

Regards,
Geoff.

[dpgeorge]

I am running into some issues where I am seeing uselect.select and uselect.poll (or ipoll) blocking even when I specify a timeout in the call.

How long does the call to select/poll block for, beyond the specified timeout? Does it block forever?

Using socket.socket objects in select/poll should never block. It's simply a loop that keeps polling until the socket is ready.

Can you share code that shows the problem?

[gmrza]

I've pulled out the lines of code relating to the socket use.

Initial setup, before we go into the loop is as follows:

addr = socket.getaddrinfo('0.0.0.0', 80)[0][-1]

s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.settimeout(4.5)
s.bind(addr)
s.listen(1)

the main loop runs infinitely (while True).
The socket code is as follows:

First:

        try:
            r, w, x = uselect.select([s], [],[], timeout) # timeout is a value from 1 to 4 seconds
        except OSError as e:
            l.append('Select: {}'.format(e))
            print(e)
            r = [] # Make sure r is defined

Then there is a block which reads a BME280 sensor via I2C

And then we go on to read data from the socket if len(r) > 0:

        if len(r) > 0:
            try:
                cl, addr = r[0].accept()
            except OSError as e:
                l.append('Accept: {}'.format(e))
                print('Accept: {}'.format(e))
            else:
                #print('client connected from', addr)
                try:
                    request = cl.recv(1024)
                except OSError as e:
                    l.append('Recv: {}'.format(e))
                    print('Recv: {}'.format(e))
                else:
                    #If we didn't raise an exception, process the data read

The only line that times out is the recv() call, which shouldn't time out, as it is only called if there is data to be read. As I've noted earlier I am only really seeing this behaviour if my WiFi connectivity is not so great. What is also appearing to be the case is that it is more likely to occur when using a Windows or Android client than when using curl or a browser on Mac OS - I don't have data to confirm that however.

[dpgeorge]

The only line that times out is the recv() call, which shouldn't time out, as it is only called if there is data to be read.

I don't think that's true, there's not necessarily data to read. The select will return readable because the socket s has a waiting, incoming connection. That means the call to accept() will succeed without blocking. But then you need to do another call to select() on the client socket cl before you can call cl.recv(1024) without blocking.

[gmrza]

Thanks. That seems to be making a difference, at least in a good connectivity environment. I'll need to deploy the update outdoors to really test it.

[gmrza]

I think mobile browsers seem to do things a bit differently, which is was also causing part of my problem. I've now modified my code to use a state machine to take it through the steps on accepting and reading from the connection, while keeping my loop to read the sensors running. Once I tidy up my code that will provide a more elegant solution.
Of course once I move to using asyncio and mqtt this will all be moot....

[dpgeorge]

Moving to asyncio would probably make a lot of these problems disappear, because the built-in socket server in asyncio will handle all this for you.

[peterhinch]

The fact that your problem occurs when RSSI is low suggests that you may be encountering brief WiFi outages.

Writing applications capable of handling WiFi outages is quite difficult - see these notes. The approach we used in developing asynchronous MQTT differed from yours in that we used nonblocking sockets rather than select.poll but some of the observations may be relevant.

[gmrza]

I've got my code to a point where it is working reliably enough. I think you may have helped with the next step. My intention was to move to MQTT - I needed to work through the learning curve of learning to work with MicroPython and the Pico W first. I also have an existing InfluxDB/Grafana setup that I needed to fit into. Using MQTT would be a more elegant way to get data into InfluxDB, however. The links you shared will probably be a big help to getting me on my way.