[Feature Request]: Message Priority Bit

[swdrumm]

Platform

Linux Native

Description

Summary: Add a priority bit to the message header which modifies the order in which queued messages are (re)transmitted.

Proposal: While messages should always be sent immediately, there's no guarantee that a message is delivered on the first try. When a number of messages are being held in queue (i.e. in the case of retransmissions or when a node requests a download of messages from store-and-forward) it would be desirable to receive the queued messages in priority order - i.e. the most important / urgent messages first.

One way to do this would be to set a priority bit in the header for all messages.

Inspiration: US naval messages are sent with four precedence levels: Routine (R), Priority (P), Immediate (O), and Flash (Z) to indicate urgency.

• Routine (R): This level is for standard day-to-day operations that do not require immediate attention. Messages are expected to be delivered within 6 hours.

• Priority (P): Used for important tasks that require timely coordination and should be actioned before routine messages. These messages are typically delivered within 3 hours.

• Immediate (O): This level indicates that the message requires rapid action or decision. Immediate messages are to be delivered within 30 minutes.

• Flash (Z): The highest level of precedence, reserved for messages of extreme urgency, such as those vital to national survival. Flash messages are to be delivered within 10 minutes.

The symbols R, P, O, and Z are used to designate these precedence levels. The timeline for each level of precedence indicates the maximum time allowed for a message to be delivered after its release by the originating station.

[fifieldt]

Thanks for using Meshtastic! Let's move this out of the bug tracker into a discussion so we can get more eyes on it and have a broader-ranging chat about what we have right now and the potential options.

[swdrumm]

Sounds good! GUVWAF is in the loop - this issue was created based on the thread Message Priority Bit on Discord.

[fifieldt]

Here are the current priorities from the code:

 388  Higher priorities are sent first (when managing the transmit queue).
 389  This field is never sent over the air, it is only used internally inside of a local device node.
 390  API clients (either on the local node or connected directly to the node)
 391  can set this parameter if necessary.
 392  (values must be <= 127 to keep protobuf field to one byte in size.
 393  Detailed background on this field:
 394  I noticed a funny side effect of lora being so slow: Usually when making
 395  a protocol there isn’t much need to use message priority to change the order
 396  of transmission (because interfaces are fairly fast).
 397  But for lora where packets can take a few seconds each, it is very important
 398  to make sure that critical packets are sent ASAP.
 399  In the case of meshtastic that means we want to send protocol acks as soon as possible
 400  (to prevent unneeded retransmissions), we want routing messages to be sent next,
 401  then messages marked as reliable and finally 'background' packets like periodic position updates.
 402  So I bit the bullet and implemented a new (internal - not sent over the air)
 403  field in MeshPacket called 'priority'.
 404  And the transmission queue in the router object is now a priority queue. */
 405 typedef enum _meshtastic_MeshPacket_Priority {
 406     /* Treated as Priority.DEFAULT */
 407     meshtastic_MeshPacket_Priority_UNSET = 0,
 408     /* TODO: REPLACE */
 409     meshtastic_MeshPacket_Priority_MIN = 1,
 410     /* Background position updates are sent with very low priority -
 411  if the link is super congested they might not go out at all */
 412     meshtastic_MeshPacket_Priority_BACKGROUND = 10,
 413     /* This priority is used for most messages that don't have a priority set */
 414     meshtastic_MeshPacket_Priority_DEFAULT = 64,
 415     /* If priority is unset but the message is marked as want_ack,
 416  assume it is important and use a slightly higher priority */
 417     meshtastic_MeshPacket_Priority_RELIABLE = 70,
 418     /* If priority is unset but the packet is a response to a request, we want it to get there relatively quickly.
 419  Furthermore, responses stop relaying packets directed to a node early. */
 420     meshtastic_MeshPacket_Priority_RESPONSE = 80,
 421     /* Higher priority for specific message types (portnums) to distinguish between other reliable packets. */
 422     meshtastic_MeshPacket_Priority_HIGH = 100,
 423     /* Higher priority alert message used for critical alerts which take priority over other reliable packets. */
 424     meshtastic_MeshPacket_Priority_ALERT = 110,
 425     /* Ack/naks are sent with very high priority to ensure that retransmission
 426  stops as soon as possible */
 427     meshtastic_MeshPacket_Priority_ACK = 120,
 428     /* TODO: REPLACE */
 429     meshtastic_MeshPacket_Priority_MAX = 127
 430 } meshtastic_MeshPacket_Priority;

[swdrumm]

Makes sense from the protocol side, but I think there's value in priority flags from the user-side as well. "Road blocked by fire - Do not attempt to follow" is a lot more important than "Can you pickup lunch?".

[swdrumm]

Some follow-up context. My team is located in rural Montana, USA in fairly rugged and mountainous terrain. While we have VHF/UHF voice comms coverage, we are intending to use Meshtastic for message + location traffic during volunteer / qualified-amateur coordination of fire-fighting, search-and-rescue, avalanche rescue, etc.

[fifieldt]

Thanks, yes! I was mainly sharing to help myself understand what we have right now that we might be able to align to.

[fifieldt]

@swdrumm , may I ask a bit about the desired behaviour? Imagine the scenario where higher priority message is send and received by the intended recipient, but the ACK is not received by the sender. Should the sender keep sending that message as a priority even if it means no other or fewer (lower priority) messages are sent?

[swdrumm]

Two other thoughts.... The Naval Message structure is just an example. The Amateur Radio Relay League (ARRL) uses EMERGENCY, PRIORITY, WELFARE and ROUTINE. Pick your favorite. Precedence should also default to the LOWEST level and attempts to use other, higher levels should require positive approval by the user on the app UI via some type of warning dialog to prevent abuse.

[GUVWAF]

I was originally confused about this request, but I think the main request is actually an automatic re-try functionality. Whether this should be using a message priority bit or e.g. just by setting the timeout is a different question. You would need to keep track of the "time left" anyway, which depends on when the original packet was sent as well as the priority.

Considering this will require additional memory, it would likely need to be implemented as a feature of the Store and Forward module, using either PSRAM or on Linux native devices.

Something that makes this difficult is that 1) acknowledgments are not retransmitted, so it's more likely the ACK doesn't come back even when the message is received and 2) it's hard to determine a timeout for when the ACK should have come back as it depends on the amount of hops towards and from the destination, as well as queuing delays depending on how busy it is anywhere on the path.

[swdrumm]

Thinking about this along with the Read Receipts discussion that I just moved over from Discord to GH. Maybe there should be an entire settings category that can be enabled for "mission critical" operations. Underneath that category would be message precedence / auto re-try, ACK retransmits, read receipts, etc.

Capabilities that may not be needed....until they're really needed. 🤔

[GUVWAF]

By the way, some of these functionalities you might already find using this BBS: https://github.com/SpudGunMan/meshing-around

[swdrumm]

Another rationale for the importance of this use case... sparse or non-existing mesh.

I just finished up range testing some Seeed Studios T-1000E's. There were no other users or nodes in the area so the "mesh" was just two devices point-to-point. As expected, I experienced a fair amount of packet loss (sequence #'s) due to intervening terrain as I walked around the neighborhood. If this had been a real scenario, the desired behavior would be for the sender queue up and to continue retrying messages (based on message precedence levels) in the hopes that the receiving node would come back into range.