Skip to content

Commit b61bb95

Browse files
jlianPat Altimore
jlian
authored and
Pat Altimore
committed
Persistence docs draft
1 parent def5090 commit b61bb95

File tree

1 file changed

+399
-0
lines changed

1 file changed

+399
-0
lines changed

‎articles/iot-operations/manage-mqtt-broker/howto-broker-persistence.md

Lines changed: 399 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,399 @@
1+
---
2+
title: Data persistence for the Azure IoT Operations MQTT broker (preview)
3+
description: Learn how to configure the data persistence feature for the Azure IoT Operations MQTT broker for data durability.
4+
author: PatAltimore
5+
ms.author: patricka
6+
ms.topic: how-to
7+
ms.service: azure-iot-operations
8+
ms.subservice: azure-mqtt-broker
9+
ms.date: 07/22/2025
10+
11+
---
12+
13+
# Configure MQTT broker persistence (preview)
14+
15+
The data persistence feature is designed as a complementary mechanism to the replication system. While the broker replicates data across multiple nodes, a cluster-wide shutdown can still result in data loss.
16+
17+
To mitigate this risk, the MQTT broker supports persistent storage, which allows critical data to be written to disk and preserved across restarts. This data persistence feature is different from the [Disk-backed message buffer](./howto-disk-backed-message-buffer.md), which uses disk as an extension of memory but is ephemeral and doesn't provide durability guarantees.
18+
19+
Storing data on disk introduces a performance cost. The impact varies depending on the type and speed of the underlying storage medium.
20+
21+
You can configure data persistence during initial deployment by using the Azure portal or Azure CLI. Some persistence-related options can also be modified after deployment.
22+
23+
## Configuration methods
24+
25+
To configure data persistence for the MQTT broker, you can use either:
26+
27+
- **Azure portal**: Configure persistence settings through the graphical user interface during IoT Operations deployment.
28+
- **Azure CLI**: Use a JSON configuration file with the `--broker-config-file` flag when you deploy IoT Operations using the `az iot ops create` command.
29+
30+
For the complete list of available settings, see the Azure IoT Operations API documentation.
31+
32+
## Deployment-time configuration options
33+
34+
These settings must be configured at deployment time and can't be changed later.
35+
36+
> [!IMPORTANT]
37+
> Persistence must be configured at deployment time and can't be disabled afterward. However, some persistence-related options can be modified after deployment.
38+
39+
### Volume and volume size
40+
41+
To persist data on disk, the MQTT broker uses a Persistent Volume (PV). Two settings control how this volume is provisioned:
42+
43+
- **`maxSize`** *(required)*: Specifies the maximum size of the persistent volume used for storing broker data. This field is always required, even if a custom volume claim is provided. The value must be above 100 MB.
44+
45+
**Example:** `10GiB`
46+
47+
- **`persistentVolumeClaimSpec`** *(optional)*: Defines a custom PersistentVolumeClaim (PVC) template to control how the persistent volume is provisioned. If you don't specify this setting, a default PVC is automatically created using the specified `maxSize` and the default storage class, which may result in suboptimal performance if the default class isn't backed by a local path provisioner.
48+
49+
> [!IMPORTANT]
50+
> When you specify `persistentVolumeClaimSpec`, the access mode must be set to `ReadWriteOncePod`.
51+
52+
# [Azure portal](#tab/portal)
53+
54+
[Screenshot placeholder: Volume and volume size configuration in Azure portal]
55+
56+
To configure volume settings in the Azure portal:
57+
58+
1. During IoT Operations deployment, navigate to the **MQTT Broker** configuration section.
59+
2. In the **Data Persistence** settings:
60+
- Set the **Maximum Size** for the persistent volume (required).
61+
- Optionally configure **Persistent Volume Claim Spec** settings for custom storage class requirements.
62+
63+
# [Azure CLI](#tab/azurecli)
64+
65+
To configure volume settings using Azure CLI, prepare a Broker configuration file in JSON format:
66+
67+
```json
68+
{
69+
"persistence": {
70+
"maxSize": "10GiB",
71+
"persistentVolumeClaimSpec": {
72+
"storageClassName": "example-storage-class",
73+
"accessModes": [
74+
"ReadWriteOncePod"
75+
]
76+
}
77+
}
78+
}
79+
```
80+
81+
Then deploy IoT Operations using the `az iot ops create` command with the `--broker-config-file` flag:
82+
83+
```azurecli
84+
az iot ops create ... --broker-config-file <FILE>.json
85+
```
86+
87+
---
88+
89+
### Encryption
90+
91+
To protect data, the MQTT broker encrypts all persistence data on disk by default using strong AES-256-GCM encryption. This ensures that even if an attacker gains access to the underlying volume, sensitive broker state or session data remains protected.
92+
93+
Encryption is optional and is enabled automatically. However, you can disable encryption if needed. Note that encryption protects data at rest only and data in memory isn't encrypted. The performance cost of using encryption should be minimal, but key rotation isn't supported yet.
94+
95+
# [Azure portal](#tab/portal)
96+
97+
[Screenshot placeholder: Encryption configuration in Azure portal]
98+
99+
To configure encryption settings in the Azure portal:
100+
101+
1. During IoT Operations deployment, navigate to the **MQTT Broker** configuration section.
102+
2. In the **Data Persistence** settings:
103+
- Toggle **Encryption** to enable or disable data encryption.
104+
- By default, encryption is enabled using AES-256-GCM.
105+
106+
# [Azure CLI](#tab/azurecli)
107+
108+
To disable encryption using Azure CLI, add the following to your Broker configuration file:
109+
110+
```json
111+
{
112+
"persistence": {
113+
"encryption": {
114+
"mode": "Disabled"
115+
}
116+
}
117+
}
118+
```
119+
120+
---
121+
122+
## Runtime configuration options
123+
124+
These options can be updated after you deploy Azure IoT Operations MQTT broker.
125+
126+
### Retained messages persistence
127+
128+
Retained messages are MQTT messages that the broker stores and delivers to new subscribers when they connect to a topic. These messages help ensure that subscribers receive the most recent data even if they weren't connected when the message was originally published. This is particularly useful for status updates, configuration data, or last-known values that new subscribers need immediately upon connection.
129+
130+
Persisting retained messages to disk ensures that these important messages survive broker restarts and aren't lost during maintenance or unexpected shutdowns. Without persistence, retained messages exist only in memory and are lost when the broker restarts, which could leave new subscribers without critical initial data.
131+
132+
This setting controls which retained messages are persisted to disk.
133+
134+
- **`mode`** *(required if `retain` is set)*: Options are `None`, `All`, or `Custom`.
135+
- `None`: No retained messages are persisted.
136+
- `All`: All retained messages are persisted.
137+
- `Custom`: Only the topics listed in `retainSettings.topics` are persisted.
138+
139+
- If you select `Custom`:
140+
- **`retainSettings.topics`** *(optional)*: List of topic patterns to persist. Wildcards `#` and `+` are supported.
141+
142+
- **`retainSettings.dynamic.mode`** *(optional, default: `Enabled`)*: Allows MQTT clients to request disk persistence for retained messages using an MQTT v5 user property.
143+
144+
# [Azure portal](#tab/portal)
145+
146+
[Screenshot placeholder: Retained messages persistence configuration in Azure portal]
147+
148+
To configure retained messages persistence in the Azure portal:
149+
150+
1. Navigate to your IoT Operations instance.
151+
2. Go to **MQTT Broker** > **Data Persistence**.
152+
3. In the **Retained Messages** section:
153+
- Select the **Mode**: None, All, or Custom.
154+
- If Custom is selected, specify topic patterns and dynamic mode settings.
155+
156+
# [Azure CLI](#tab/azurecli)
157+
158+
To configure retained messages persistence using Azure CLI, add the following to your Broker configuration file:
159+
160+
```json
161+
{
162+
"persistence": {
163+
"retain": {
164+
"mode": "Custom",
165+
"retainSettings": {
166+
"topics": ["my/+/topics"],
167+
"dynamic": {
168+
"mode": "Enabled"
169+
}
170+
}
171+
}
172+
}
173+
}
174+
```
175+
176+
---
177+
178+
### Subscriber queue persistence
179+
180+
Subscriber queues hold messages that are waiting to be delivered to MQTT clients with Quality of Service (QoS) 1 subscriptions. When a client subscribes with QoS 1, the broker guarantees message delivery by queuing messages until the client acknowledges receipt. These queues are especially important for clients that may be temporarily disconnected or processing messages slowly.
181+
182+
Persisting subscriber queues to disk ensures that messages waiting for delivery aren't lost during broker restarts. This is critical for IoT scenarios where devices may have intermittent connectivity, slow processing capabilities, or persistent sessions that need to maintain message delivery guarantees across broker restarts. Without persistence, queued messages would be lost, potentially causing data loss for important device communications.
183+
184+
For more information about subscriber queues and message delivery, see [Configure broker MQTT client options](./howto-broker-mqtt-client-options.md#subscriber-queue-limit).
185+
186+
This setting controls which subscriber message queues are persisted to disk. Session state metadata is always persisted regardless of these settings.
187+
188+
- **`mode`** *(required if `subscriberQueue` is set)*: Options are `None`, `All`, or `Custom`.
189+
190+
- If you select `Custom`:
191+
- **`subscriberQueueSettings.subscriberClientIds`** *(optional)*: List of subscriber client IDs to persist. Wildcards `*` are supported.
192+
193+
- **`subscriberQueueSettings.dynamic.mode`** *(optional, default: `Enabled`)*: Enables MQTT clients to request persistence dynamically.
194+
195+
# [Azure portal](#tab/portal)
196+
197+
[Screenshot placeholder: Subscriber queue persistence configuration in Azure portal]
198+
199+
To configure subscriber queue persistence in the Azure portal:
200+
201+
1. Navigate to your IoT Operations instance.
202+
2. Go to **MQTT Broker** > **Data Persistence**.
203+
3. In the **Subscriber Queue** section:
204+
- Select the **Mode**: None, All, or Custom.
205+
- If Custom is selected, specify subscriber client IDs and dynamic mode settings.
206+
207+
# [Azure CLI](#tab/azurecli)
208+
209+
To configure subscriber queue persistence using Azure CLI, add the following to your Broker configuration file:
210+
211+
```json
212+
{
213+
"persistence": {
214+
"subscriberQueue": {
215+
"mode": "Custom",
216+
"subscriberQueueSettings": {
217+
"subscriberClientIds": ["hello-client-id", "my-other-client-*"],
218+
"dynamic": {
219+
"mode": "Enabled"
220+
}
221+
}
222+
}
223+
}
224+
}
225+
```
226+
227+
---
228+
229+
> [!IMPORTANT]
230+
> A client that wasn't previously persisted can become persisted at any time. However, only the publishes received after persistence is enabled for that specific client are stored on disk. If persistence is later disabled for the client, that change won't take effect until the client reconnects with the MQTT clean start = true flag.
231+
232+
#### Session expiry and subscriber queue persistence
233+
234+
For subscribers to have their subscriber message queues persisted to disk, both the session expiry interval and the broker's persistence configuration must align properly. Specifically:
235+
236+
- MQTTv5 clients: Can specify session expiry interval using the Session Expiry Interval property of CONNECT or DISCONNECT packets. They can also request disk persistence behavior with a specified user property.
237+
238+
- MQTTv3 clients: If the Clean Session flag is true, then the session expiry interval is set to 0. Otherwise, the value of `maxSessionExpirySeconds` configuration in the broker is used.
239+
240+
The broker handles subscriber queue persistence differently based on the configuration mode and session expiry interval:
241+
242+
When mode is set to `None`:
243+
- All subscriber queues remain in-memory only, regardless of session expiry interval
244+
245+
When mode is set to `All`:
246+
- If session expiry interval = 0: Queues remain in-memory
247+
- If session expiry interval > 0: Queues are persisted to disk for the duration of the interval
248+
249+
When mode is set to `Custom`:
250+
- If session expiry interval = 0: Queues remain in-memory
251+
- If session expiry interval > 0: Queues are persisted to disk for the duration of the interval only if:
252+
- The client ID matches the configured list in `subscriberQueueSettings.subscriberClientIds`, OR
253+
- Dynamic mode is enabled and an MQTTv5 client provided the matching user property in their CONNECT packet
254+
255+
To ensure that subscriber queues are persisted to disk, remember the following key points:
256+
257+
- Subscriber queues are only persisted when the session expiry interval is greater than 0
258+
- For `Custom` mode, either the client ID must be explicitly listed or dynamic persistence must be enabled with the correct user property
259+
- MQTTv5 clients can use dynamic persistence by including the configured user property (default: `aio-persistence=true`) in their CONNECT packet
260+
261+
### State store persistence
262+
263+
The state store is an internal component of the MQTT broker that maintains various types of operational data and metadata beyond standard MQTT messages. This includes broker configuration state, session information, subscription details, and other internal data structures that the broker uses to manage client connections and message routing efficiently.
264+
265+
Persisting state store data to disk ensures that the broker can maintain operational continuity across restarts. This is particularly important for complex IoT deployments where losing internal state could result in connection issues, subscription losses, or performance degradation as the broker rebuilds its internal data structures from scratch.
266+
267+
State store persistence is especially valuable in production environments where minimizing recovery time and maintaining service consistency are critical. Without persistence, the broker must rebuild all internal state when it restarts, which can cause temporary service disruptions and performance impacts.
268+
269+
For more information about the state store, see [Learn about the MQTT broker state store protocol](../create-edge-apps/concept-about-state-store-protocol.md).
270+
271+
This setting controls which keys in the internal state store are persisted.
272+
273+
- **`mode`** *(required if stateStore is set)*: Options are `None`, `All`, or `Custom`.
274+
275+
- If you select `Custom`:
276+
- **`stateStoreSettings.stateStoreResources`** *(optional)*: List of key types and keys to persist.
277+
278+
- **`keyType`**: Pattern, String, or Binary
279+
280+
- **`keys`**: List of keys/patterns to persist.
281+
282+
- **`stateStoreSettings.dynamic.mode`** *(optional, default: `Enabled`)*: Enables MQTT clients to request persistence dynamically.
283+
284+
# [Azure portal](#tab/portal)
285+
286+
[Screenshot placeholder: State store persistence configuration in Azure portal]
287+
288+
To configure state store persistence in the Azure portal:
289+
290+
1. Navigate to your IoT Operations instance.
291+
2. Go to **MQTT Broker** > **Data Persistence**.
292+
3. In the **State Store** section:
293+
- Select the **Mode**: None, All, or Custom.
294+
- If Custom is selected, specify state store resources with key types, keys, and dynamic mode settings.
295+
296+
# [Azure CLI](#tab/azurecli)
297+
298+
To configure state store persistence using Azure CLI, add the following to your Broker configuration file:
299+
300+
```json
301+
{
302+
"persistence": {
303+
"stateStore": {
304+
"mode": "Custom",
305+
"stateStoreSettings": {
306+
"stateStoreResources": [
307+
{
308+
"keyType": "Pattern",
309+
"keys": ["cats", "topics-*"]
310+
},
311+
{
312+
"keyType": "Pattern",
313+
"keys": ["dogs", "weather-*"]
314+
}
315+
],
316+
"dynamic": {
317+
"mode": "Enabled"
318+
}
319+
}
320+
}
321+
}
322+
}
323+
```
324+
325+
---
326+
327+
### Request persistence from a client using dynamic persistence setting
328+
329+
Clients can request persistence for their data by sending an MQTT v5 user property in their messages. This allows clients to dynamically enable persistence for their messages, subscriber queues, or state store entries without requiring broker configuration changes.
330+
331+
When a client requests persistence, the broker checks its current persistence settings and applies them accordingly. If the requested persistence mode is enabled and set to allow dynamic requests, the broker persists the client's data as specified in the configuration.
332+
333+
The dynamic persistence setting can be enabled or disabled for each data type (retained messages, subscriber queues, and state store entries) by setting the `dynamic.mode` to `Enabled` in the respective configuration sections. The MQTT user property key and value used for dynamic persistence requests are configured separately at the broker level.
334+
335+
# [Azure portal](#tab/portal)
336+
337+
[Screenshot placeholder: Dynamic persistence configuration in Azure portal]
338+
339+
To configure dynamic persistence settings in the Azure portal:
340+
341+
1. Navigate to your IoT Operations instance.
342+
2. Go to **MQTT Broker** > **Data Persistence**.
343+
3. Configure the global MQTT user property settings:
344+
- Set the **User Property Key** (default: `aio-persistence`)
345+
- Set the **User Property Value** (default: `true`)
346+
4. In each persistence section (Retained Messages, Subscriber Queue, State Store):
347+
- Set **Dynamic Mode** to Enabled to allow clients to request persistence for that data type.
348+
349+
# [Azure CLI](#tab/azurecli)
350+
351+
To configure dynamic persistence using Azure CLI, add the MQTT user property settings at the broker level and enable dynamic mode for specific data types:
352+
353+
```json
354+
{
355+
"persistence": {
356+
"dynamicSettings": {
357+
"userPropertyKey": "custom-persistence-key",
358+
"userPropertyValue": "true"
359+
},
360+
"retain": {
361+
"mode": "Custom",
362+
"retainSettings": {
363+
"topics": ["my/+/topics"],
364+
"dynamic": {
365+
"mode": "Enabled"
366+
}
367+
}
368+
},
369+
"subscriberQueue": {
370+
"mode": "Custom",
371+
"subscriberQueueSettings": {
372+
"subscriberClientIds": ["client-*"],
373+
"dynamic": {
374+
"mode": "Enabled"
375+
}
376+
}
377+
},
378+
"stateStore": {
379+
"mode": "Custom",
380+
"stateStoreSettings": {
381+
"stateStoreResources": [
382+
{
383+
"keyType": "Pattern",
384+
"keys": ["important-*"]
385+
}
386+
],
387+
"dynamic": {
388+
"mode": "Enabled"
389+
}
390+
}
391+
}
392+
}
393+
}
394+
```
395+
396+
---
397+
398+
To learn more about Azure CLI support for advanced MQTT broker configuration, see [Azure CLI support for advanced MQTT broker configuration](https://aka.ms/aziotops-broker-config).
399+

0 commit comments

Comments
 (0)