Recommended way for reliable PC-Hub-communication

[Novakasa]

I'm currently trying to understand how to do PC to hub communication reliably. Right now, I am sending my messages from PC to the CityHub with something like this:

async def send_message(self, message):
    if isinstance(message, str):
        message = bytearray(message + "$", encoding="utf8")
    
    for block in chunk(message, 100): #splits the message into chunks of 100
        await self.hub.client.write_gatt_char(NUS_RX_UUID, block, False)
        await asyncio.sleep(0.15)

and receiving them like this:

def update_input(char):
    global input_buffer
    if char == "$":
        input_handler(input_buffer)
        input_buffer = ""
    else:
        input_buffer += char

while running:
    if p.poll(int(1000*delta)):
        char = stdin.read(1)
        if char is not None:
            update_input(char)
    update()

This works quite well when the messages are not too frequent, however when I am sending a large number of messages in a short period of time, the end of a message tends to be cut off, resulting in the terminating character "$" not being received, so the next message will be treated as part of the previous one. This gets better if I increase the delay await asyncio.sleep(0.3), but makes communication kind of slow. In practice, my messages are smaller than 100 characters.

For example, a message might look like this:
rpc::{'func': 'remove_color', 'args': ['red'], 'kwargs': {}}$
but, if a lot of messages come in, they are bunched up like so:
rpc::{'func': 'remove_color', 'args': ['red'], 'kwargrpc::{'func': 'set_color', 'args': ['red', [[0, 84, 48]], 'marker'], 'kwargs': {}}$

The write_gatt_char method has a argument response. I am not too sure what this does, I had hoped it would do some kind of checksum checking, but simply turning it on doesn't seem to change much.

My questions now are:

• Am I reaching the bluetooth limit in this case, causing the messages to be cut off, or might something else be wrong with this implementation?
• What does the response argument on write_gatt_char do?
• If I am reaching the hardware limit, I might need to look at doing some kind of checksum implementation myself. Any recommendations? Pybricksdev seems to rely on lower level firmware code in this case, so it's hard for me to copy the implementation.

Would be glad for any kind of advice on this topic. Thanks!

[dlech]

* Am I reaching the bluetooth limit in this case, causing the messages to be cut off, or might something else be wrong with this implementation?

Probably a firmware limit. There is currently a 100? byte receive buffer, so if stdin.read() is not called fast enough, then data can be lost. Plus, there are additionally buffers in the Bluetooth chip that we don't have any control over, so if data is receive too fast, it can be dropped there too.

What does the response argument on write_gatt_char do?

It selects Write With Response or Write Without Response. This is a Bluetooth Low Energy protocol thing. The response is a BLE packet that acknowledges that the data was, in fact, received. This slows down throughput, but as you have seen, Write Without Response requires a delay anyway, so using Write With Response would eliminate the need for an arbitrary delay.

In terms of Pybricks, Write With Response also means that, if it succeeds, that the data made it through the Bluetooth chip to the MCU (where Pybricks firmware is running). However a buffer overflow in the Pybricks firmware can still occur.

* If I am reaching the hardware limit, I might need to look at doing some kind of checksum implementation myself. Any recommendations? Pybricksdev seems to rely on lower level firmware code in this case, so it's hard for me to copy the implementation.

I would suggest adding something to create backpressure in the pybricksdev program. For example, only send one RPC request at a time, then wait for an ack from the hub that it has processed the message before sending another.

[Novakasa]

Thanks for the explanations, that clears it up a lot. As I am also seeing the issue with response=True, I am guessing that it is the pybricks buffer that creates the issue for me. I will investigate further and probably limit the messages as you suggested.

[laurensvalk]

Not really an answer to your question, but rather a question for you.

Do you perhaps have a few good (small) examples of communicating between the PC and the hub?

This Pybricks user would be quite interested: #470

[Novakasa]

I think I should be able to produce a small example