Proposed - October 29, 2025
In the development of the IOT Network for IOT projects, the communication layer must support:
- Reliable command and telemetry exchange.
- Low-latency message delivery.
- Easy integration with Python.
- Operability in constrained networks (e.g., Wi-Fi mesh or LTE).
- Autonomy if the hub connection is lost (nodes continue their mission and coordinate locally).
- Centralized Broker Model: All nodes communicate through a central broker (e.g., MQTT broker). This model simplifies message routing and management but introduces a single point of failure.
- Decentralized Peer-to-Peer Model: Nodes communicate directly with each other without a central broker (e.g., using ZMTP/ZeroMQ). This model enhances resilience and autonomy but requires more complex message routing and management.
- Hybrid Model: Combines both centralized and decentralized approaches, where nodes can communicate through a central broker but also have the capability to communicate directly with each other when necessary.
- Hierarchical Model: Nodes are organized in a hierarchy, where lower-level nodes communicate with higher-level nodes, which then communicate with the central broker. This model can help manage large networks but may introduce latency and complexity.
- HTTP/REST: While widely used, HTTP/REST is not ideal for real-time communication due to its request-response nature and higher overhead.
- GRPC: Although more efficient than HTTP/REST, GRPC is better suited for microservices rather than IoT devices with constrained resources.
- WebSockets: Provides full-duplex communication channels over a single TCP connection. It is better suited for real-time applications than for IoT as it can't scale well as each upstream connection requires a socket.
- MQTT: A lightweight messaging protocol designed for low-bandwidth, high-latency networks, making it suitable for IoT applications. Versions 3.1.1 and 5.0 are widely supported. Protocol 5.0 introduces enhanced features like improved error reporting and user properties, required for IoT applications deployed on cloud platforms, and cases that require more reliability and error handling to implement mission-critical messaging. Include features like Last Will and Testament (LWT) or shared subscriptions. Defacto standard for IoT messaging.
- CoAP (Constrained Application Protocol): A specialized web transfer protocol for use with constrained nodes and networks in the Internet of Things. It is designed to easily translate to HTTP for simplified integration with the web while being lightweight enough for low-power devices. Underlying protocol is UDP, low overhead, secure connection with DTLS, and asynchronous message exchanges. No pub-sub natively.
- MQTT-SN: A variant of MQTT designed for sensor networks and constrained devices, with reduced packet overhead and without the need of a permanent TCP/IP connection, it uses UDP instead. Similar to COAP but with pub-sub natively and in process of being standardized.
- ZMTP (ZeroMQ Message Transport Protocol): A high-performance asynchronous messaging protocol that allows for peer-to-peer communication without a central broker. It is suitable for decentralized systems.
- AMQP (Advanced Message Queuing Protocol): A robust messaging protocol that supports complex routing and queuing mechanisms. While powerful, it may be too heavyweight for constrained IoT devices as it was originally designed for enterprise messaging systems.
- JSON: A lightweight data interchange format that is easy to read and write.
- Protocol Buffers: A language-agnostic binary serialization format developed by Google. It is more efficient than JSON in terms of both size and speed, making it suitable for IoT applications with limited bandwidth and processing power.
- Custom Formats: Plain Text could be used for very simple data structures, but it lacks the flexibility and efficiency of more structured formats like JSON or Protocol Buffers.
Paho MQTT client can work with any data serialization format as it can publish and subscribe to byte arrays or strings. ZeroMQ also supports sending and receiving raw byte arrays, allowing for the same flexibility.
The MQTT architecture follows a publish-subscribe model where clients (nodes) can publish messages to topics and subscribe to topics to receive messages. A central MQTT broker manages the distribution of messages between clients. This architecture is suitable for IoT applications due to its lightweight nature and support for various Quality of Service (QoS) levels.
MQTT brokers can be configured to persist messages, ensuring that messages are not lost in case of broker failure or network issues. This is particularly important for mission-critical applications where data integrity is essential. There are three persistence options:
- No Persistence: This is the default option in MQTT. In this mode, messages are not stored on the server and are lost if the server or network fails. This mode is suitable for situations where messages are not critical and can be easily regenerated.
- Queued persistent: In this mode, messages are stored on the server until they are delivered to the subscriber. If the subscriber is not available, messages are queued until the subscriber reconnects. Queued persistence is useful when the subscriber is not always connected to the network, or if the subscriber needs to receive all messages, even if they are sent when the subscriber is offline.
- Persistent with ACK acknowledgment: This mode provides the highest level of message persistence. In this mode, messages are stored on the server until they are delivered to the subscriber, and the subscriber must acknowledge receipt of the message. If the subscriber does not acknowledge receipt, the message is re-sent until the subscriber acknowledges receipt. This mode is useful when it is critical to ensure that messages are received and processed by the subscriber.
MQTT supports various security mechanisms to ensure secure communication between clients and the broker. These include TLS/SSL encryption for secure data transmission, role-based access control (RBAC), and access control lists (ACLs) to restrict client access to specific topics.
- Eclipse Mosquitto: A popular open-source MQTT broker that is lightweight and suitable for IoT applications. It can be used together with Paho MQTT client library for Python. It supports persistent sessions, QoS levels, and is easy to set up using Docker. Protocol supports versions 3.1 and 5.0. Configurable via mosquitto.conf file. Single threaded design.
- NanoMQ: A high-performance MQTT broker written in C, designed for scalability and low resource consumption. It is suitable for large-scale IoT deployments. Configurable via environment variables and configuration files. Limited support of authentication mechanisms compared to Mosquitto. Less community support and documentation. Protocol supports versions 3.1.1 and 5.0. Include a MQQT broker and a lightweight messaging bus for SDV (Software Defined Vehicle) applications. Fully asynchronous and multi-threaded architecture.
Overall Mosquitto (single-core) can help you limit CPU and memory usage while performance is not priority. If you need high throughput and low latency, NanoMQ(multi-core) is a better choice. Comparison reference
- Eclipse Paho MQTT: An open-source MQTT client library for Python that is easy to use and integrates well with Eclipse Mosquitto. Defacto standard for MQTT clients in Python. Synchronous and callback-based for event handling.
The ZMTP architecture allows for direct peer-to-peer communication between nodes without the need for a central broker. Interestingly, as ZeroMQ is considered brokerless, a number of the solutions do introduce a central node of one kind or another, such as this to help with discovery of publishers and subscribers:
ZeroMQ flexibility comes with complexity as developers must implement many features themselves, from message routing to session management.
- ZeroMQ: A high-performance asynchronous messaging library that allows for peer-to-peer communication without a central broker. ZeroMQ offers sockets that carry data/messages via multiple transports: in-process, inter-process, TCP, UDP, TIPC, and multicast. It also provides a collection of messaging patterns such as fan-out, pub/sub, push-pull (pipeline), request-reply(sync), and pair/pair (exclusive pair). ZeroMQ supports several different authentication methods, including NULL, PLAIN, and CURVE.
- Nanomsg: A simple and easy-to-use messaging library that provides a subset of ZeroMQ's functionality. It is suitable for lightweight applications but lacks some advanced features of ZeroMQ. Still under active development and has a smaller community compared to ZeroMQ.
- PyZMQ: The official Python binding for ZeroMQ, providing a Pythonic interface to the ZeroMQ library. It is well-documented and widely used in the Python community.
- nanomsg-python: Python bindings for the Nanomsg library, providing a simple interface for messaging. However, it is less popular and has limited documentation compared to PyZMQ.
Using only an MQTT broker would introduce a single point of failure, making it unsuitable for fully decentralized systems. Instead, ZeroMQ will be used for inter-node communication, enabling peer-to-peer messaging without a central broker. This design enhances autonomy, fault tolerance, and resilience, allowing nodes to continue operating even if the central hub or cloud connection is unavailable.
The system could start with a pure ZeroMQ-based architecture to ensure simplicity and robustness in early stages. If future requirements demand cloud integration or IoT interoperability, a hybrid architecture can be introduced — where ZeroMQ handles high-speed, low-latency communication within the local or edge network, and MQTT bridges data exchange with cloud services or external systems.
1 MQTT Authorization
2 Eclipse Mosquitto MQTT Broker
3 NanoMQ MQTT Broker
4 Eclipse Paho MQTT Python Client
5 ZeroMQ Messaging Library
6 Nanomsg Messaging Library
7 ZeroMQ Python Bindings
8 Nanomsg Python Bindings
1 Lavrenko, Anastasia & Barry, Z & Norman, R & Frazer, C & Ma, Y & Woodward, Graeme & Pawson, Stephen. (2021). Autonomous Swarm of UAVs for Tracking of Flying Insects with Harmonic Radar. 10.1109/VTC2021-Spring51267.2021.9449074.
2 Matternet Enables Real-time Monitoring of Autonomous Drones
3 Secure Drone Communications using MQTT protocol
4 Using Protobuf for MQTT Message and QoS 2 Stored Messages in Python
5 Introducing MQTT
6 MQTT Basics Course
7 ZeroMQ Guide