Skip to content

Commit 1f20007

Browse files
doc: zbus: add documentation for multi-domain zbus
Add documentation for the multi-domain zbus feature, and multi-domain zbus samples. Signed-off-by: Trond F. Christiansen <[email protected]>
1 parent 0b776a7 commit 1f20007

File tree

4 files changed

+333
-1
lines changed

4 files changed

+333
-1
lines changed
100 KB
Loading

doc/services/zbus/index.rst

Lines changed: 161 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -878,6 +878,134 @@ illustrates the runtime registration usage.
878878
the channel observer it was first associated with through :c:func:`zbus_chan_rm_obs`.
879879

880880

881+
Multi-domain Communication
882+
***************************
883+
884+
ZBus supports multi-domain communication, enabling message passing between different execution
885+
domains such as CPU cores or separate devices. This is achieved through proxy agents that
886+
forward messages between domains.
887+
888+
.. figure:: images/zbus_proxy_agent.png
889+
:alt: ZBus publish processing detail
890+
:width: 75%
891+
892+
..
893+
Temporary image illustrating zbus multi-domain communication with proxy agents.
894+
895+
Concepts
896+
========
897+
898+
Multi-domain zbus introduces several key concepts:
899+
900+
* **Master channels**: Channels where messages are published locally within a domain
901+
* **Shadow channels**: Read-only channels that mirror master channels from other domains
902+
* **Proxy agents**: Background services that synchronize channel data between domains
903+
* **Transport backends**: Communication mechanisms (IPC, UART) used by proxy agents
904+
905+
The :c:macro:`ZBUS_MULTIDOMAIN_CHAN_DEFINE` macro enables conditional compilation to create
906+
master channels in one domain and corresponding shadow channels in other domains.
907+
908+
Transport Backends
909+
==================
910+
911+
ZBus multi-domain communication relies on transport backends to forward messages between different
912+
execution domains.
913+
914+
IPC Backend
915+
-----------
916+
917+
The IPC backend facilitates communication between CPU cores within the same system using
918+
Inter-Process Communication mechanisms.
919+
920+
See the :zephyr:code-sample:`zbus-ipc-forwarder` sample for a complete implementation.
921+
922+
UART Backend
923+
------------
924+
925+
The UART backend enables communication between physically separate devices over serial connections.
926+
This backend extends zbus messaging across device boundaries, allowing distributed systems to
927+
maintain a unified message bus architecture.
928+
929+
See the :zephyr:code-sample:`zbus-uart-forwarder` sample for a complete implementation.
930+
931+
Usage
932+
=====
933+
934+
Multi-domain zbus requires defining shared channels and setting up proxy agents. Here's a typical setup:
935+
936+
**common.h** - Shared channel definitions:
937+
938+
.. code-block:: c
939+
940+
#include <zephyr/zbus/zbus.h>
941+
#include <zephyr/zbus/multidomain/zbus_multidomain.h>
942+
943+
struct request_data {
944+
// ...
945+
};
946+
947+
struct response_data {
948+
// ...
949+
};
950+
951+
ZBUS_MULTIDOMAIN_CHAN_DEFINE(request_channel, struct request_data,
952+
NULL, NULL, ZBUS_OBSERVERS_EMPTY,
953+
ZBUS_MSG_INIT(0),
954+
IS_ENABLED(DOMAIN_A), /* Master on domain A */
955+
1 /* Include on both domains */);
956+
957+
ZBUS_MULTIDOMAIN_CHAN_DEFINE(response_channel, struct response_data,
958+
NULL, NULL, ZBUS_OBSERVERS_EMPTY,
959+
ZBUS_MSG_INIT(0),
960+
IS_ENABLED(DOMAIN_B), /* Master on domain B */
961+
1 /* Include on both domains */);
962+
963+
**Domain A** - Requester setup:
964+
965+
.. code-block:: c
966+
967+
#define DOMAIN_A 1
968+
#include "common.h"
969+
970+
/* Set up proxy agent and add channels for forwarding */
971+
ZBUS_PROXY_AGENT_DEFINE(proxy_agent, ZBUS_MULTIDOMAIN_TYPE_IPC, ipc_instance);
972+
ZBUS_PROXY_ADD_CHANNEL(proxy_agent, request_channel);
973+
974+
void response_listener_cb(const struct zbus_channel *chan) {
975+
const struct response_data *resp = zbus_chan_const_msg(chan);
976+
LOG_INF("Received response: ...");
977+
}
978+
ZBUS_LISTENER_DEFINE(response_listener, response_listener_cb);
979+
ZBUS_CHAN_ADD_OBS(response_channel, response_listener, 0);
980+
981+
int main(void)
982+
{
983+
struct request_data req = { /* ... */ };
984+
zbus_chan_pub(&request_channel, &req, K_MSEC(100));
985+
return 0;
986+
}
987+
988+
**Domain B** - Responder setup:
989+
990+
.. code-block:: c
991+
992+
#define DOMAIN_B 1
993+
#include "common.h"
994+
995+
/* Set up proxy agent and add channels for forwarding */
996+
ZBUS_PROXY_AGENT_DEFINE(proxy_agent, ZBUS_MULTIDOMAIN_TYPE_IPC, ipc_instance);
997+
ZBUS_PROXY_ADD_CHANNEL(proxy_agent, response_channel);
998+
999+
/* Observe requests and publish responses */
1000+
void request_listener_cb(const struct zbus_channel *chan) {
1001+
const struct request_data *req = zbus_chan_const_msg(chan);
1002+
struct response_data resp = { /* ... */ };
1003+
zbus_chan_pub(&response_channel, &resp, K_MSEC(100));
1004+
LOG_INF("Processed request ...");
1005+
}
1006+
ZBUS_LISTENER_DEFINE(request_listener, request_listener_cb);
1007+
ZBUS_CHAN_ADD_OBS(request_channel, request_listener, 0);
1008+
8811009
Samples
8821010
*******
8831011

@@ -901,7 +1029,8 @@ available:
9011029
observer registration feature;
9021030
* :zephyr:code-sample:`zbus-confirmed-channel` implements a way of implement confirmed channel only
9031031
with subscribers;
904-
* :zephyr:code-sample:`zbus-benchmark` implements a benchmark with different combinations of inputs.
1032+
* :zephyr:code-sample:`zbus-ipc-forwarder` demonstrates multi-core communication using IPC forwarders;
1033+
* :zephyr:code-sample:`zbus-uart-forwarder` demonstrates inter-device communication using UART forwarders.
9051034

9061035
Suggested Uses
9071036
**************
@@ -913,6 +1042,13 @@ subscribers (if you need a thread) or listeners (if you need to be lean and fast
9131042
the listener, another asynchronous message processing mechanism (like :ref:`message queues
9141043
<message_queues_v2>`) may be necessary to retain the pending message until it gets processed.
9151044

1045+
For multi-domain scenarios, use zbus to enable communication across execution boundaries:
1046+
1047+
* **Multi-core systems**: Use IPC backend to coordinate between application and network processors,
1048+
or distribute workloads across multiple CPU cores.
1049+
* **Distributed devices**: Use UART backend to create distributed applications where multiple
1050+
connected devices participate in the same logical message bus over serial connections.
1051+
9161052
.. note::
9171053
ZBus can be used to transfer streams from the producer to the consumer. However, this can
9181054
increase zbus' communication latency. So maybe consider a Pipe a good alternative for this
@@ -954,6 +1090,30 @@ Related configuration options:
9541090
observers to statically allocate.
9551091
* :kconfig:option:`CONFIG_ZBUS_RUNTIME_OBSERVERS_NODE_ALLOC_NONE` use user-provided runtime
9561092
observers nodes;
1093+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN` enable multi-domain communication support.
1094+
1095+
Multi-domain Configuration Options
1096+
==================================
1097+
1098+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_IPC` enable IPC backend for multi-domain communication;
1099+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_UART` enable UART backend for multi-domain communication;
1100+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_LOG_LEVEL` set the log level for multi-domain operations;
1101+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_MESSAGE_SIZE` maximum message size for multi-domain
1102+
channels;
1103+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_CHANNEL_NAME_SIZE` maximum size of channel names in
1104+
multi-domain communication;
1105+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_PROXY_STACK_SIZE` stack size for proxy agent threads;
1106+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_PROXY_PRIORITY` priority of proxy agent threads;
1107+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_SENT_MSG_POOL_SIZE` number of sent messages to track for
1108+
acknowledgments;
1109+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_MAX_TRANSMIT_ATTEMPTS` maximum retry attempts for message
1110+
transmission;
1111+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_SENT_MSG_ACK_TIMEOUT` initial acknowledgment timeout for
1112+
sent messages;
1113+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_SENT_MSG_ACK_TIMEOUT_MAX` maximum acknowledgment timeout
1114+
for exponential backoff;
1115+
* :kconfig:option:`CONFIG_ZBUS_MULTIDOMAIN_UART_BUF_COUNT` number of UART buffers for multi-domain
1116+
communication;
9571117

9581118
API Reference
9591119
*************
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
.. zephyr:code-sample:: zbus-ipc-forwarder
2+
:name: zbus Proxy agent - IPC forwarder
3+
:relevant-api: zbus_apis
4+
5+
Forward zbus messages between different domains using IPC.
6+
7+
Overview
8+
********
9+
This sample demonstrates zbus inter-domain communication on multi-core platforms.
10+
The sample implements a request-response pattern where:
11+
12+
- The **application CPU** acts as a requester, publishing requests on a master channel and receiving responses on a shadow channel
13+
- The **remote CPU** acts as a responder, receiving requests on a shadow channel and publishing responses on a master channel
14+
- **IPC forwarders** automatically synchronize channel data between CPU domains using the zbus proxy functionality
15+
16+
The ``common/common.h`` file defines shared channels using :c:macro:`ZBUS_MULTIDOMAIN_CHAN_DEFINE` with conditional
17+
compilation to create master channels on one domain and shadow channels on the other.
18+
19+
This architecture enables message passing between different CPU cores while maintaining zbus's
20+
publish-subscribe structure across domain boundaries.
21+
22+
Building and Running
23+
********************
24+
25+
Use sysbuild to build this sample for multi-core platforms:
26+
27+
For nRF54H20 with CPURAD:
28+
29+
.. zephyr-app-commands::
30+
:zephyr-app: samples/subsys/zbus/ipc_forwarder
31+
:board: nrf54h20dk/nrf54h20/cpuapp
32+
:goals: build
33+
:west-args: --sysbuild -S nordic-log-stm
34+
35+
It can also be built for the other cores in the nRF54H20 using sysbuild configurations:
36+
37+
.. code-block::
38+
39+
SB_CONFIG_REMOTE_BOARD_NRF54H20_CPURAD=y # Default
40+
SB_CONFIG_REMOTE_BOARD_NRF54H20_CPUPPR=y
41+
SB_CONFIG_REMOTE_BOARD_NRF54H20_CPUPPR_XIP=y
42+
SB_CONFIG_REMOTE_BOARD_NRF54H20_CPUFLPR_XIP=y
43+
44+
For nRF5340:
45+
46+
.. zephyr-app-commands::
47+
:zephyr-app: samples/subsys/zbus/ipc_forwarder
48+
:board: nrf5340dk/nrf5340/cpuapp
49+
:goals: build
50+
:west-args: --sysbuild
51+
52+
Sample Output
53+
*************
54+
55+
Application CPU Output
56+
======================
57+
58+
.. code-block:: console
59+
60+
*** Booting Zephyr OS build v4.2.0-1157-gaaa3626d8206 ***
61+
<inf> main: ZBUS Multidomain IPC Forwarder Sample Application
62+
<inf> main: Channel request_channel is a master channel
63+
<inf> main: Channel response_channel is a shadow channel
64+
<inf> main: Published on channel request_channel. Request ID=1, Min=-1, Max=1
65+
<inf> main: Received message on channel response_channel
66+
<inf> main: Response ID: 1, Value: 0
67+
<inf> main: Published on channel request_channel. Request ID=2, Min=-2, Max=2
68+
<inf> main: Received message on channel response_channel
69+
<inf> main: Response ID: 2, Value: -1
70+
71+
Remote CPU Output
72+
=================
73+
74+
.. code-block:: console
75+
76+
*** Booting Zephyr OS build v4.2.0-1157-gaaa3626d8206 ***
77+
<inf> main: ZBUS Multidomain IPC Forwarder Sample Application
78+
<inf> main: Channel request_channel is a shadow channel
79+
<inf> main: Channel response_channel is a master channel
80+
<inf> main: Received message on channel request_channel
81+
<inf> main: Request ID: 1, Min: -1, Max: 1
82+
<inf> main: Sending response: ID=1, Value=0
83+
<inf> main: Response published on channel response_channel
84+
<inf> main: Received message on channel request_channel
85+
<inf> main: Request ID: 2, Min: -2, Max: 2
86+
<inf> main: Sending response: ID=2, Value=-1
87+
<inf> main: Response published on channel response_channel
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
.. zephyr:code-sample:: zbus-uart-forwarder
2+
:name: zbus Proxy agent - UART forwarder
3+
:relevant-api: zbus_apis
4+
5+
Forward zbus messages between different devices using UART.
6+
7+
Overview
8+
********
9+
This sample demonstrates zbus inter-device communication using UART forwarders between separate devices.
10+
The sample implements a request-response pattern where:
11+
12+
- **Device A** acts as a requester, publishing requests on a master channel and receiving responses on a shadow channel
13+
- **Device B** acts as a responder, receiving requests on a shadow channel and publishing responses on a master channel
14+
- **UART forwarders** automatically synchronize channel data between devices using the zbus proxy functionality over UART
15+
16+
The ``common/common.h`` file defines shared channels using :c:macro:`ZBUS_MULTIDOMAIN_CHAN_DEFINE` with conditional
17+
compilation to create master channels on one device and shadow channels on the other.
18+
19+
This architecture enables message passing between different devices while maintaining zbus's
20+
publish-subscribe structure across device boundaries.
21+
22+
Building and Running
23+
********************
24+
25+
This sample requires two separate devices connected via UART. Each device runs a different application:
26+
27+
For Device A (Requester):
28+
29+
.. zephyr-app-commands::
30+
:zephyr-app: samples/subsys/zbus/uart_forwarder/dev_a
31+
:board: nrf5340dk/nrf5340/cpuapp
32+
:goals: build flash
33+
34+
For Device B (Responder):
35+
36+
.. zephyr-app-commands::
37+
:zephyr-app: samples/subsys/zbus/uart_forwarder/dev_b
38+
:board: nrf5340dk/nrf5340/cpuapp
39+
:goals: build flash
40+
41+
Hardware Setup
42+
==============
43+
44+
Connect the two devices via UART:
45+
46+
- Device A TX → Device B RX
47+
- Device A RX → Device B TX
48+
- Connect GND between devices
49+
50+
Sample Output
51+
=============
52+
53+
Device A Output (Requester)
54+
****************************
55+
56+
.. code-block:: console
57+
58+
*** Booting Zephyr OS build v4.2.0-1157-g7bf51d719a31 ***
59+
<inf> main: ZBUS Multidomain UART Forwarder Sample Application
60+
<inf> main: Channel request_channel is a master channel
61+
<inf> main: Channel response_channel is a shadow channel
62+
<inf> main: Published on channel request_channel. Request ID=1, Min=-1, Max=1
63+
<inf> main: Received message on channel response_channel
64+
<inf> main: Response ID: 1, Value: 1
65+
<inf> main: Published on channel request_channel. Request ID=2, Min=-2, Max=2
66+
<inf> main: Received message on channel response_channel
67+
<inf> main: Response ID: 2, Value: 0
68+
69+
Device B Output (Responder)
70+
****************************
71+
72+
.. code-block:: console
73+
74+
*** Booting Zephyr OS build v4.2.0-1157-g7bf51d719a31 ***
75+
<inf> main: ZBUS Multidomain UART Forwarder Sample Application
76+
<inf> main: Channel request_channel is a shadow channel
77+
<inf> main: Channel response_channel is a master channel
78+
<inf> main: Received message on channel request_channel
79+
<inf> main: Request ID: 1, Min: -1, Max: 1
80+
<inf> main: Sending response: ID=1, Value=1
81+
<inf> main: Response published on channel response_channel
82+
<inf> main: Received message on channel request_channel
83+
<inf> main: Request ID: 2, Min: -2, Max: 2
84+
<inf> main: Sending response: ID=2, Value=-1
85+
<inf> main: Response published on channel response_channel

0 commit comments

Comments
 (0)