uasyncio server reader.read(len) getting cropped/truncated

[GM-Script-Writer-62850]

Posted this on the raspberry pi forum a few days ago, 0 replies so I am trying here

Documentation: https://docs.micropython.org/en/latest/library/uasyncio.html#tcp-stream-connections
My full code: https://github.com/GM-Script-Writer-62850/PICO_W_Thermostat/blob/main/PICO/main.py#L836

I am sending some json to my pico via a post request (made here)
data length is 693 and it is getting cut off in the middle of a integer/time stamp at 683 chars

async def webUI(reader, writer):
	# stuff 
	data=await reader.readexactly(length) # works as expected returning all 693 chars
	data=await reader.readline() # works as expected returning all 693 chars
	data=await reader.read(length) # gets cut at 683, is this a bug?

# more stuff
uasyncio.create_task(uasyncio.start_server(webUI, "0.0.0.0", 80))

any idea what is going on here

[karfas]

I assume reader.read() follows the semantics of reading from as socket or a C/POSIX read().

read() will read up to the given length but stops when no more data is available; the caller is responsible to call read() again when too less bytes were received. The documentation does not state it will "read everything".

readexactly() reads exactly the given bytes (and waits for more, when a simple read() returns too less).
See the uasyncio source (from https://github.com/micropython/micropython/blob/master/extmod/uasyncio/stream.py):

    async def read(self, n=-1):
        r = b""
        while True:
            yield core._io_queue.queue_read(self.s)
            r2 = self.s.read(n)
            if r2 is not None:
                if n >= 0:
                    return r2
                if not len(r2):
                    return r
                r += r2
    async def readexactly(self, n):
        r = b""
        while n:
            yield core._io_queue.queue_read(self.s)
            r2 = self.s.read(n)
            if r2 is not None:
                if not len(r2):
                    raise EOFError
                r += r2
                n -= len(r2)
        return r

[peterhinch]

This is correct.

[DavesCodeMusings]

Just a thought... are there any multi-byte unicode characters being sent? I seem to recall being bitten by something similar in the past. I think with read it's returning bytes and length would limit the number of bytes which is not necessarily the same as the number of characters.

[karfas]

@DavesCodeMusings: I don't think so, as readexactly() returns the wanted result. With unicode characters, the problem would be the same as with read().

[DavesCodeMusings]

Ah, true.

[straga]

db7682e
That commit change read.
For example, esp32 does not read any data from UART.
But if back to what was before the change. Works and reads data from UART.
Probably, I am not correct. But before work like that. : #8867

[karfas]

@straga: An UART problem is not relevant here, I think. read(n) for sockets(!) seems to behave like documented.

The diff found in db7682e doesn't look like major change of behaviour (OK, it will no longer return None) when read(n) gets called with a positive n.

[straga]

@karfas Yes.
if -1, for UART : that lives in the while loop forever: stream read.