aioble indicate not working for characteristic.write(msg, send_update=True)?

[iot49]

I'm using aioble essentially as a UART - hence I need to capture all packets. An ESP32-S3 (with 8MB psram) acts as peripheral, Google Chrome (on MacOS) as central.

The rx and tx characteristics are both declared with indicate=True:

self._rx_characteristic = aioble.Characteristic(connector_service, _CONNECTOR_RX, indicate=True, write=True)
self._tx_characteristic = aioble.Characteristic(connector_service, _CONNECTOR_TX, indicate=True, read=True)

To send messages to connected centrals:

self._tx_characteristic.write(msg, send_update=True)

The Micropython doc states

If send_update is True, then any subscribed clients will be notified (or indicated, depending on what they’re subscribed to and which operations the characteristic supports) about this write.

On the central side (Web Bluetooth on Chrome):

const tx = await service.getCharacteristic(_CONNECTOR_TX);
// Note: tx.properties.indicate is true
tx.startNotifications()
tx.addEventListener('characteristicvaluechanged', this.valueChanged.bind(this));

this.valueChanged gets called as expected, but unless I use a delay (100ms?) on the peripheral, messages get dropped.

The other direction (central to peripheral) works correctly without delay between writes (await rx_characteristic.writeValue(data);), suggesting (?) that Chrome correctly waits for the indicate response.

I've tried await self._tx_characteristic(...) in Micropython, but then data gets sent twice and after a while a get a memory error.

So the question is:

How do I use indicate with aioble as the sender?

[jimmo]

@iot49 I'm having a little bit of trouble understanding the WebBluetooth API. Does tx.startNotifications() subscribe to the characteristic? If so, is it subscribing to just notifications or indications? I have no idea what "when navigator.bluetooth is added to the active notification context" means?

Also, what does it mean for a "value changed" event to fire on a characteristic? There is no such thing in a GATT client... only notifications and indications sent by the server.

Anyway...

When you send a notification from a server to a client, there is no guaranteed delivery... it's just fire and forget. If you attempt to send more notifications than can be queued for the underlying L2CAP channel, then they are dropped.

The other direction (central to peripheral) works correctly without delay between writes

I suspect this is due to the above -- your PC/Phone running Chrome has essentially unlimited memory to buffer pending notifications, whereas on MicroPython the queue length is very small.

For indications though, that's why the characteristic.indicate function is async (where as notify isn't). You can safely do:

while ...:
  await x.indicate(...)

with no delays.

However, the problem here is that there's no equivalent awaitable version of char.write(..., send_update=True) for indications.

I've tried await self._tx_characteristic(...) in Micropython, but then data gets sent twice and after a while a get a memory error.

There's a missing function name there... did you mean await self._tx_characteristic.indicate ?

To send messages to connected centrals: self._tx_characteristic.write(msg, send_update=True)

Do you need to use subscription? Could you instead just send direct notifications/indications without using write? This is what the Nordic UART Service does for example.

[iot49]

@jimmo

I'm having a little bit of trouble understanding the WebBluetooth API.

With WebBluetooth API I mean the BLE interface implemented in some browsers, e.g. Chrome.

tx.startNotifications() (javascript in the browser) enables a callback on the tx characteristic whenever a notify or indicate is received (in my case only indicate, as I do not enable notify; WebBluetooth has a single function for both). Confusingly I called this "value changed" event.

My objective is to get "websocket" functionality between Micropython and the the browser. Similar to Nordic UART, except I use "indicate" for both directions to not loose any messages.

I suspect this is due to the above -- your PC/Phone running Chrome has essentially unlimited memory to buffer pending notifications, whereas on MicroPython the queue length is very small.

I am puzzled by this statement.

The direction central (Chrome) to peripheral (Micropython) works correctly without delay or dropping messages. I presume that Chrome uses indicate to achieve this (and, as expected, I need to use await: await this.rx_characteristic.writeValue(x); in WebBluetooth).

The problem is in the direction peripheral (Micropython) to central (Chrome). My code (MicroPython):

self._tx_characteristic.write(msg, send_update=True)
# delay needed for central not to miss any messages sent by the peripheral
await asyncio.sleep_ms(100)
# await self._tx_characteristic.indicate(list(self._active_connections.keys())[0], timeout_ms=1000)

The last line is commented out since with it I get memory errors after a while (and duplicate messages - I do not recall the exact issues since after some fiddling I just gave up on calling await self._tx_characteristic.indicate and instead settled on the delay.

Do you need to use subscription? Could you instead just send direct notifications/indications without using write? This is what the Nordic UART Service does for example.

That's what I am trying to do, but I have not found the code for the Nordic UART Service implemented with aioble, only with the low level bluetooth library, and that example does not use indicate as far as I can tell. It that code exists it would be terrific to add it to https://github.com/micropython/micropython-lib/tree/master/micropython/bluetooth/aioble/examples.

Source of confusion: The doc states that

BLE.gatts_write(value_handle, data, send_update=True)

Writes the local value for this handle, which can be read by a client.
If send_update is True, then any subscribed clients will be notified (or indicated, depending on what they’re subscribed to and which operations the characteristic supports) about this write.

If that's the case, then my code self._tx_characteristic.write(msg, send_update=True) (where _tx_characteristic has been created with aioble and which calls the underlyingBLE.gatts_write should use indicate. I agree there should be an await in front, but that's not how aioble implements write(msg, send_update=True), no matter the value of send_update.

await x.indicate(...) has that await, but how do I tell it WHAT data to send?

[jimmo]

@iot49

With WebBluetooth API I mean the BLE interface implemented in some browsers, e.g. Chrome.

Yep, I realised that, I was just saying that I found their documentation a bit vague and hard to align with the way I understand how a GATT client works.

tx.startNotifications() (javascript in the browser) enables a callback on the tx characteristic whenever a notify or indicate is received (in my case only indicate, as I do not enable notify; WebBluetooth has a single function for both).

Confusingly I called this "value changed" event.

That appears to be what WebBluetooth calls it too :)

(This is my confusion -- in GATT, a notification and indication can include a payload that is independent of the server's value)

My objective is to get "websocket" functionality between Micropython and the the browser. Similar to Nordic UART, except I use "indicate" for both directions to not loose any messages.

It's a shame that WebBluetooth does not suport L2CAP Connection Oriented Channels, which is a far simpler way to achieve what is essentially a socket with flow control. (See the aioble file transfer examples).

I suspect this is due to the above -- your PC/Phone running Chrome has essentially unlimited memory to buffer pending notifications, whereas on MicroPython the queue length is very small.

I am puzzled by this statement.

What I was trying to say is that I would imagine that a server on WebBluetooth/Chrome can essentially queue up as many notifications as it likes and just keep streaming them out the GATT channel. In MicroPython, we can only queue up a couple of notifications.

The same could be true of indications too, but in MIcroPython we require you to get the receipt of an indication before sending another one.

The direction central (Chrome) to peripheral (Micropython) works correctly without delay or dropping messages. I presume that Chrome uses indicate to achieve this (and, as expected, I need to use await: await this.rx_characteristic.writeValue(x); in WebBluetooth).

The problem is in the direction peripheral (Micropython) to central (Chrome). My code (MicroPython):

self._tx_characteristic.write(msg, send_update=True)
# delay needed for central not to miss any messages sent by the peripheral
await asyncio.sleep_ms(100)
# await self._tx_characteristic.indicate(list(self._active_connections.keys())[0], timeout_ms=1000)

This will cause a duplicate message because the write has send_update (which will send a notification and/or indication), and then you immediately explicitly send a standalone indication too.

The last line is commented out since with it I get memory errors after a while (and duplicate messages - I do not recall the exact issues since after some fiddling I just gave up on calling await self._tx_characteristic.indicate and instead settled on the delay.

Do you need to use subscription? Could you instead just send direct notifications/indications without using write? This is what the Nordic UART Service does for example.

That's what I am trying to do, but I have not found the code for the Nordic UART Service implemented with aioble, only with the low level bluetooth library, and that example does not use indicate as far as I can tell. It that code exists it would be terrific to add it to https://github.com/micropython/micropython-lib/tree/master/micropython/bluetooth/aioble/examples.

Essentially you just need to not use write. To send data server->client, do await tx.indicate(data). To send data client->server do await rx.write(data).

Source of confusion: The doc states that

BLE.gatts_write(value_handle, data, send_update=True)

Writes the local value for this handle, which can be read by a client. If send_update is True, then any subscribed clients will be notified (or indicated, depending on what they’re subscribed to and which operations the characteristic supports) about this write.

If that's the case, then my code self._tx_characteristic.write(msg, send_update=True) (where _tx_characteristic has been created with aioble and which calls the underlyingBLE.gatts_write should use indicate. I agree there should be an await in front, but that's not how aioble implements write(msg, send_update=True), no matter the value of send_update.

await x.indicate(...) has that await, but how do I tell it WHAT data to send?

AHHHH. Now I see the problem. Sorry I missed this.

Initially we only supported including a payload for notify, but recently-ish (PR #11239) I fixed this so that indicate also supports a payload. What's missing is that I haven't updated aioble's Characteristic.indicate (in server.py) to support this argument. Sorry about this!

I will send a PR, but you can try patching it yourself. (It basically just needs to work just like the notify method above -- the underlying ble.gatts_notify and ble.gatts_indicate have the same signatures now).

diff --git a/micropython/bluetooth/aioble/aioble/server.py b/micropython/bluetooth/aioble/aioble/server.py
index b6cc4a3..a4162dd 100644
--- a/micropython/bluetooth/aioble/aioble/server.py
+++ b/micropython/bluetooth/aioble/aioble/server.py
@@ -257,7 +257,7 @@ class Characteristic(BaseCharacteristic):
             raise ValueError("Not supported")
         ble.gatts_notify(connection._conn_handle, self._value_handle, data)
 
-    async def indicate(self, connection, timeout_ms=1000):
+    async def indicate(self, connection, timeout_ms=1000, data=None):
         if not (self.flags & _FLAG_INDICATE):
             raise ValueError("Not supported")
         if self._indicate_connection is not None:
@@ -270,7 +270,7 @@ class Characteristic(BaseCharacteristic):
 
         try:
             with connection.timeout(timeout_ms):
-                ble.gatts_indicate(connection._conn_handle, self._value_handle)
+                ble.gatts_indicate(connection._conn_handle, self._value_handle, data)
                 await self._indicate_event.wait()
                 if self._indicate_status != 0:
                     raise GattError(self._indicate_status)

The other workaround is that you can do

tx.write(data, send_update=False)
await tx.indicate()

Which has the effect of only writing the local value, which the no-arg indicate will then use.

But this is a good reminder... I have long been meaning to write a NUS demo (and many others) for aioble

[jimmo]

I will send a PR...

Created micropython/micropython-lib#728 to fix this.

[iot49]

Works reliably with new version