support multi mqtt-sn control packets in one UDP datagram?

[qzhuyan]

I look through the spec but I don't see it mentioned packaging.

I think for efficiency, it will be nice to support MQTT-SN control Packet packaging in one UDP datagram to utlized the MTU.

That is, one UDP datagram sent from one endpoint may contain multiple MQTT-SN control Packet

e.g.

--------------------------
UDP Headers
--------------------------
MQTT-SN control Packet 1
MQTT-SN control Packet 2
MQTT-SN control Packet 3
---END of UDP datagram--

These control packets are no necessary only for new control packets from one endpoint but could also be used for retransmission.

We may add a flag to exchange this capability for the resources constratind devices which are not going to support this.

What do you think?

[simon622]

We have introduced the notion of envelopes quite extensively now, I can definitively see value in this (not necessarily just for UDP). I support this.

@icraggs @lenzarda ?

[lenzarda]

@qzhuyan What is the problem you are trying to solve? Can you please elaborate on the scenario you have in mind?

[qzhuyan]

@lenzarda there are three stuffs in my mind as followings:

1. Give the option to send small payloads in batch to reduce overhead.
   Send large UDP datagram (contains more MQTT-sn control packet) instead of many small ones (One UDP packet per MQTT-sn) , this will save udp hdr, ip hdr overheads, less Acks from peer, lower the workload on the network driver task queue, save network bandwidth and processing units, save power as well.

2. Improve retransmission.

If 5 lost individual (one per datagram) MQTT-SN control packets within one MTU (minus UDP header), they can be retransmitted as a single UDP datagram instead of five separate ones.

3. for encryption efficiency.
   Encryption adds fixed per-packet overhead, efficiency wins with larger datagrams.

there are more, I will update when I recall them.

[updated below]

4. Device sends multi pubilshes for different topics.
   Instead of sending udp datagram per publish topic, put them in one datagram.

5. Publish from local node and Ack the publish msg from remote node in the same datagram.

--------------------------
UDP Headers
--------------------------
MQTT-SN Publish
MQTT-SN Puback
---END of UDP datagram--

6. Request/Response.

--------------------------
UDP Headers
--------------------------
MQTT-SN Puback for  request Publish
MQTT-SN Publish for response.
---END of UDP datagram--

[lenzarda]

@qzhuyan Thanks for the additional details provided. What is the use case in which an IoT device needs to send multiple MQTT-SN packets?

[qzhuyan]

For the usecases that Packets doesn't need to get send out in realtime. they could be batched and send in some pace to save the power of radio transmission.

[qzhuyan]

Another usecase is that a device would like to send MQTT-SN packets for different topics.

[lenzarda]

For the usecases that Packets doesn't need to get send out in realtime. they could be batched and send in some pace to save the power of radio transmission.

In this case the payloads can be grouped to be sent on the same MQTT-SN packet

[lenzarda]

Another usecase is that a device would like to send MQTT-SN packets for different topics.

This is an interesting use case that I never faced in constrained IoT. Can you please provide an example of practical use?

[icraggs]

My first reaction was that I'm not keen on this idea because of the change in implementation assumptions required by receivers in particular. Whereas the TCP/IP programming interfaces are stream oriented, UDP is a datagram interface, the packet is either received completely or not at all. This raises potential complexity - if the datagram containing three packets, and that is lost, do you resend the whole datagram or not? (I presume we would have to treat the three packets separately.)

However, now I'm thinking we don't disallow this in the spec, do we? We talk about delivery of MQTT-SN packets and what happens if they don't arrive, not about how to package the MQTT-SN packets in the underlying transport mechanism.

There are contraints on this packaging. Given that we only allow 1 ouststanding request at a time, I think this proposal can only apply to PUBLISH QoS 0 and PUBWOS. For instance, you can't send a PUBLISH QoS 1 immediately following another PUBLISH QoS 1 without waiting for the PUBACK.

The method of packaging packets is an agreement between the communicating partners which we don't specify. If that statement is true, which I think it is, then this suggests we might want to draw up companion documents (standards or committee notes) to suggest how to bind MQTT-SN to specific underlying transports, to aid interoperability.

An alternative would be to allow more than one topic in a PUBLISH packet, which would allow the same functionality but also support QoS 1 and 2 semantics. Then the question would be whether to allow multiple payloads as well as topics and what the semantics of partial failure would be (publishing to one topic works, but not to another).

I believe having multiple topic destinations in a PUBLISH packet is one of those long-standing MQTT requests.

[qzhuyan]

Another usecase is that a device would like to send MQTT-SN packets for different topics.

This is an interesting use case that I never faced in constrained IoT. Can you please provide an example of practical use?

thanks for asking,

Just random example: A device has two sensors one for temputure, one for lumen, they are two topics and broker will forward these to diffent 'consumer' who is asssnged to handle specific topic.

A real world example from what I have heard:
some sensor devices are powered by water flow in the water pipes, it is charged by water flow, it wakes up to take record of some sensor multi topics, it only emits the last X data points when it get charged enough power as the water is not constant flowing.

[qzhuyan]

My first reaction was that I'm not keen on this idea because of the change in implementation assumptions required by receivers in particular. Whereas the TCP/IP programming interfaces are stream oriented, UDP is a datagram interface, the packet is either received completely or not at all. This raises potential complexity - if the datagram containing three packets, and that is lost, do you resend the whole datagram or not? (I presume we would have to treat the three packets separately.)

However, now I'm thinking we don't disallow this in the spec, do we? We talk about delivery of MQTT-SN packets and what happens if they don't arrive, not about how to package the MQTT-SN packets in the underlying transport mechanism.

There are contraints on this packaging. Given that we only allow 1 ouststanding request at a time, I think this proposal can only apply to PUBLISH QoS 0 and PUBWOS. For instance, you can't send a PUBLISH QoS 1 immediately following another PUBLISH QoS 1 without waiting for the PUBACK.

The method of packaging packets is an agreement between the communicating partners which we don't specify. If that statement is true, which I think it is, then this suggests we might want to draw up companion documents (standards or committee notes) to suggest how to bind MQTT-SN to specific underlying transports, to aid interoperability.

An alternative would be to allow more than one topic in a PUBLISH packet, which would allow the same functionality but also support QoS 1 and 2 semantics. Then the question would be whether to allow multiple payloads as well as topics and what the semantics of partial failure would be (publishing to one topic works, but not to another).

I believe having multiple topic destinations in a PUBLISH packet is one of those long-standing MQTT requests.

I agree it is not necessary to get defined in this spec but it must be standardised (maybe in another doc) as the client and broker mostly come from different vendors, the standardlized org should define the capability detection or negotiation for interoperability. I could image a separate doc describing the methods packaging of UDP datagram, but how do we address this capability detections? I think it is better done in this spec and how to do it.

I just want to clarify that I start this as a question from developer of gateway, broker pespective for performance, it took me a while to find the info how we construct the udp datagram as working with UDP the first question is about packaging on UDP datagram. (e.g. if extra framing is needed) I understand MQTT-SN is not only for UDP, If it is out of scope the MQTT-SN, maybe we could state it it is out of the scope and reserve methods for this capability detections.