Skip to content

Commit c9261a5

Browse files
JamesJBarnettJamesJBarnett
authored
Merge pull request #303631 from spelluru/egridhttppublishandretain
Event Grid MQTT Broker - HTTP Publish and Retain
2 parents ec76f6e + b756d4f commit c9261a5

11 files changed

+434
-22
lines changed

‎articles/event-grid/authenticate-with-namespaces-using-webhook-authentication.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,18 @@ description: This article shows you how to authenticate with Azure Event Grid na
44
ms.topic: how-to
55
ms.custom:
66
- build-2025
7-
ms.date: 04/30/2025
7+
ms.date: 07/30/2025
88
author: Connected-Seth
99
ms.author: seshanmugam
1010
---
1111

12-
# Authenticate with MQTT broker using custom webhook authentication (Preview)
12+
# Authenticate with MQTT broker using custom webhook authentication
1313
This article shows how to authenticate with Azure Event Grid namespace using webhook or Azure function.
1414

1515
Webhook authentication allows external HTTP endpoints (webhooks or functions) to authenticate MQTT connections dynamically. This method uses Microsoft Entra ID JWT validation to ensure secure access.
1616

1717
When a client attempts to connect, the broker invokes a user-defined HTTP endpoint that validates credentials such SAS tokens, usernames/passwords, or even performs Certificate Revocation List (CRL) checks. The webhook evaluates the request and returns a decision to allow or deny the connection, along with optional metadata for fine-grained authorization. This approach supports flexible and centralized authentication policies across diverse device fleets and use cases.
1818

19-
> [!NOTE]
20-
> This feature is currently in preview.
2119

2220
## Prerequisites
2321

@@ -203,7 +201,7 @@ Content-Type: application/json
203201
| expiration | Expiration date in Unix time format. (Optional when Decision is set to allow) |
204202
| errorReason | Error message if decision is set to deny. This error is logged. (Optional when Decision is set to deny) |
205203

206-
### Examples of supported attribute types:
204+
### Examples of supported attribute types
207205

208206
```json
209207
"num_attr_pos": 1,

‎articles/event-grid/includes/limits.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,16 @@ The following limits apply to MQTT in Azure Event Grid namespace resource.
4848
| Inbound MQTT bandwidth per session | 1 MB per second |
4949
| Inbound in-flight MQTT messages* | 1000 messages |
5050
| Inbound in-flight MQTT bandwidth* | 64 KB |
51+
| Inbound HTTP publishing requests per Event Grid namespace | 500 messages per second per TU |
52+
| Inbound HTTP bandwidth per Event Grid namespace | 512 KB per second per TU |
53+
| Inbound HTTP publishing requests per session | 500 messages per second |
54+
| Inbound HTTP bandwidth per session | 512 KB per second |
55+
| Inbound in-flight HTTP messages* | 500 messages |
56+
| Maximum Retain Message Size** | 64 KB |
57+
| Maximum Retain Messages per Throughput Unit (TU) | 10,000 messages or 640 MB (whichever is reached first) |
58+
| Total Retain Storage per TU | 640 MB |
59+
| Retain Message Expiry (MQTT 3.1.1) | 365 days (default) |
60+
| Retain Message Expiry (MQTT 5.0) | Configurable using Message Expiry Interval, range: 0 to 31,536,000 seconds (365 days) |
5161
| Outbound MQTT publishing requests per Event Grid namespace | 1,000 messages per second per TU |
5262
| Outbound MQTT bandwidth per Event Grid namespace | 1 MB per second per TU |
5363
| Outbound MQTT publishing requests per session| 100 messages per second |
@@ -79,8 +89,11 @@ The following limits apply to MQTT in Azure Event Grid namespace resource.
7989
| Topic templates | 10 per topic space |
8090
| Permission bindings | 100 |
8191

92+
8293
\* For MQTTv5, learn more about [flow control support](../mqtt-support.md#flow-control).
8394

95+
\** Retain messages count against the total MQTT storage quota for the namespace. When the quota is reached, publishing new retain messages will fail until existing messages are expired or deleted (e.g., via empty payload publish).
96+
8497
## Events limits in Event Grid namespace
8598

8699
The following limits apply to events in Azure Event Grid namespace resource.

‎articles/event-grid/mqtt-how-to-http-publish.md

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
---
2+
title: How to Publish MQTT Messages via HTTP with Azure Event Grid
3+
description: Explains how to publish MQTT messages via HTTP with Azure Event Grid for scalable server-to-device communication. Learn how to use the HTTP Publish API effectively.
4+
#customer intent: As a backend developer, I want to publish MQTT messages via HTTP so that I can integrate with Azure Event Grid without maintaining persistent MQTT sessions.
5+
author: spelluru
6+
contributors: null
7+
ms.topic: how-to
8+
ms.date: 07/30/2025
9+
ms.author: spelluru
10+
ms.reviewer: spelluru
11+
ms.custom:
12+
- ai-gen-docs-bap
13+
- ai-gen-title
14+
- ai-seo-date:07/30/2025
15+
- ai-gen-description
16+
---
17+
18+
# How to Publish MQTT Messages via HTTP with Azure Event Grid (preview)
19+
Azure Event Grid now supports publishing MQTT messages via HTTP, enabling backend systems to send messages to devices without maintaining persistent MQTT connections. This approach simplifies integration for applications that prefer stateless communication, uses secure authentication with Entra ID, and provides scalable, reliable delivery to MQTT clients. In this article, you learn how to use the HTTP Publish API, obtain the necessary credentials, and verify message delivery using popular tools like Bruno, and MQTTX.
20+
21+
> [!NOTE]
22+
> This feature is currently in preview.
23+
24+
This article explains how to publish MQTT messages via HTTP with Azure Event Grid.
25+
26+
## Get your connection details
27+
28+
- Namespace fully qualified domain name (FQDN). Example: `contoso.westus3-1.ts.eventgrid.azure.net`.
29+
- Topic. Example: `devices/CXa-23112/prompt`.
30+
- Entra ID client credentials.
31+
32+
## Get a bearer token
33+
Run the following Azure CLI command to get a bearer token.
34+
35+
```bash
36+
az account get-access-token --resource=https://<namespaceFQDN> --query accessToken -o tsv
37+
```
38+
39+
Save this token to use in the `Authorization: Bearer <TOKEN>` header.
40+
41+
## Publish messages using HTTP
42+
43+
### [Curl](#tab/curl)
44+
45+
Here’s an example cURL command to simulate the HTTP Publish.
46+
47+
```http
48+
curl -X POST "https://contoso.westus3-1.ts.eventgrid.azure.net/mqtt/messages?topic=devices%2XXXX-0000%2Fprompt&api-version=2025-08-01-preview" \
49+
-H "Authorization: Bearer <ENTRA_TOKEN_HERE>" \
50+
-H "mqtt-qos: 1" \
51+
-H "mqtt-retain: 0" \
52+
-H "mqtt-response-topic: devices%2XXXX-00000%2Freply" \
53+
-H "mqtt-correlation-data: XXXXXXX" \
54+
-H "mqtt-user-properties: XXXXXXXXXXXX" \
55+
-H "Content-Type: text/plain;charset=UTF-8" \
56+
--data-raw "Please accept terms of licensing and agreement"
57+
```
58+
59+
In this sample command:
60+
61+
- Topic is percent-encoded.
62+
- Add optional headers for QoS, retain flag, response topic, user properties.
63+
- Payload goes in the request body.
64+
65+
### [Bruno](#tab/brno)
66+
67+
### Import to Bruno
68+
1. Open Bruno.
69+
1. Select **Import Collection**, and then select `EventGrid_HTTP_Publish_Postman_Collection.json`. Here's the content for the JSON file:
70+
71+
```json
72+
{
73+
"info": {
74+
"name": "Event Grid MQTT Broker - HTTP Publish",
75+
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
76+
},
77+
"item": [
78+
{
79+
"name": "HTTP Publish to MQTT Broker",
80+
"request": {
81+
"method": "POST",
82+
"header": [
83+
{
84+
"key": "Authorization",
85+
"value": "Bearer {{entra_token}}"
86+
},
87+
{
88+
"key": "mqtt-qos",
89+
"value": "1"
90+
},
91+
{
92+
"key": "mqtt-retain",
93+
"value": "0"
94+
},
95+
{
96+
"key": "mqtt-response-topic",
97+
"value": "devices%2FCXa-23112%2Freply"
98+
},
99+
{
100+
"key": "mqtt-correlation-data",
101+
"value": "PlXCscK2wrbCuy8="
102+
},
103+
{
104+
"key": "mqtt-user-properties",
105+
"value": "[{\"Urgency\":\"alert\"},{\"RequestId\":\"55f4a7ee-b0b4-4d7f-8eb5-2edba2ced5d7\"}]"
106+
},
107+
{
108+
"key": "Content-Type",
109+
"value": "text/plain;charset=UTF-8"
110+
}
111+
],
112+
"url": {
113+
"raw": "https://{{namespace}}/mqtt/messages?topic={{topic}}&api-version=2025-08-01-preview",
114+
"host": [
115+
"{{namespace}}"
116+
],
117+
"path": [
118+
"mqtt",
119+
"messages"
120+
],
121+
"query": [
122+
{
123+
"key": "topic",
124+
"value": "{{topic}}"
125+
},
126+
{
127+
"key": "api-version",
128+
"value": "2025-02-15-preview"
129+
}
130+
]
131+
},
132+
"body": {
133+
"mode": "raw",
134+
"raw": "Please accept terms of licensing and agreement"
135+
}
136+
}
137+
}
138+
],
139+
"variable": [
140+
{
141+
"key": "namespace",
142+
"value": "contoso.westus3-1.ts.eventgrid.azure.net"
143+
},
144+
{
145+
"key": "topic",
146+
"value": "devices/CXa-23112/prompt"
147+
},
148+
{
149+
"key": "entra_token",
150+
"value": "<ENTRA_TOKEN_HERE>"
151+
}
152+
]
153+
}
154+
```
155+
1. Go to **Variables** tab:
156+
- Replace `{{namespace}}` with your namespace FQDN.
157+
- Replace `{{topic}}` with your MQTT topic.
158+
- Replace `{{entra_token}}` with your token from Step 2.
159+
1. Select **Send**. You should get 202 or 204.
160+
161+
---
162+
163+
## Verify in MQTTX
164+
Use MQTTX or any MQTT library (like paho-mqtt Python) to subscribe to the same topic to confirm delivery.
165+
166+
1. Create a new connection in MQTTX:
167+
- `Host: contoso.westus3-1.ts.eventgrid.azure.net`
168+
- `Port: 8883 (TLS)`
169+
- `Client ID: same as your Entra Object ID`
170+
- `Username/Password: N/A — use certificate or token auth if configured`
171+
1. Subscribe to the topic you used in the HTTP POST command.
172+
1. Run the **HTTP Publish** and watch for the message in MQTTX. You should see your payload appear.
173+
174+
If the publish **succeeds**, you see:
175+
176+
- **HTTP Response**: 204 No Content or 202 Accepted (depending on routing rules).
177+
- MQTT client sees the message instantly.
178+
179+
## Troubleshoot
180+
181+
- **401 Unauthorized?** — If the token is missing or expired: You get 401 Unauthorized. Refresh your token.
182+
- **403 Forbidden?** — If the topic is invalid or you don’t have rights: 403 Forbidden. Check your topic or permissions.
183+
- **500 Internal Server Error** - If routing fails internally, check metrics and diagnostic Logs for your Event Grid namespace.
184+
- **Message doesn’t appear?** — Ensure topic is percent-encoded in the URL, check broker routing config, and verify you’re using the same namespace.
185+
186+
187+
## Related content
188+
For an overview of this feature, see [HTTP Publish of MQTT messages with Azure Event Grid](mqtt-http-publish.md).

‎articles/event-grid/mqtt-http-publish.md

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
---
2+
title: HTTP Publish of MQTT messages with Azure Event Grid
3+
description: Publish MQTT messages via HTTP with Azure Event Grid for scalable server-to-device communication. Learn how to use the HTTP Publish API effectively.
4+
#customer intent: As a backend developer, I want to publish MQTT messages via HTTP so that I can integrate with Azure Event Grid without maintaining persistent MQTT sessions.
5+
author: spelluru
6+
contributors: null
7+
ms.topic: concept-article
8+
ms.date: 07/30/2025
9+
ms.author: spelluru
10+
ms.reviewer: spelluru
11+
ms.custom:
12+
- ai-gen-docs-bap
13+
- ai-gen-title
14+
- ai-seo-date:07/30/2025
15+
- ai-gen-description
16+
---
17+
18+
# HTTP Publish of MQTT messages with Azure Event Grid (preview)
19+
20+
The Azure Event Grid MQTT Broker HTTP Publish API empowers customers to publish MQTT messages using standard HTTP requests. This complements direct MQTT client connections, providing a simple and scalable option for server-side systems that prefer HTTP for server-to-device command-and-control, updates, or retained message management.
21+
22+
> [!NOTE]
23+
> This feature is currently in preview.
24+
25+
Key benefits:
26+
27+
- Allows backend services to send MQTT messages without keeping persistent MQTT sessions open.
28+
- Helps protect broker stability by limiting per-client MQTT sessions.
29+
- Ensures consistent message processing for both MQTT and HTTP-originated messages.
30+
31+
## When to Use HTTP Publish
32+
33+
Consider HTTP Publish when:
34+
35+
- Your backend services are HTTP-native and need to send device commands or updates over MQTT.
36+
- You want to manage retained messages without opening an MQTT connection.
37+
- You need to scale up publish capacity without exhausting session limits.
38+
39+
## How It Works
40+
41+
1. HTTP clients issue an HTTP POST request with MQTT publish details.
42+
1. Event Grid maps HTTP request parts to standard MQTT PUBLISH packet properties.
43+
1. Messages flow through the Event Grid routing and enrichment pipeline, ensuring delivery guarantees and applying any enrichment or transformation.
44+
45+
## Example: MQTT publish equivalent
46+
47+
```http
48+
PUBLISH Topic Name: devices/CXa-23112/prompt
49+
QoS: 1
50+
RETAIN: 0
51+
Response Topic: devices/CXa-23112/reply
52+
Correlation Data: >U±¶¶»/
53+
User Property: Urgency = alert
54+
User Property: RequestId = 55f4a7ee-b0b4-4d7f-8eb5-2edba2ced5d7
55+
Payload: Please accept terms of licensing and agreement
56+
```
57+
58+
## Example: HTTP publish request
59+
60+
```http
61+
POST /mqtt/messages?topic=devices%2FCXa-23112%2Fprompt&api-version=2025-02-15-preview HTTP/1.1
62+
Host: nsname.westus3-1.ts.eventgrid.azure.net
63+
Authorization: Bearer <ENTRA_TOKEN_HERE>
64+
mqtt-qos: 1
65+
mqtt-retain: 0
66+
mqtt-response-topic: devices%2FCXa-23112%2Freply
67+
mqtt-correlation-data: PlXCscK2wrbCuy8=
68+
mqtt-user-properties: W3siVXJnZW5jeSI6ImFsZXJ0In0seyJSZXF1ZXN0SWQiOiI1NWY0YTdlZS1iMGI0LTRkN2YtOGViNS0yZWRiYTJjZWQ1ZDcifV0=
69+
Content-Type: text/plain;charset=UTF-8
70+
Date: Sun, 06 Nov 1994 08:49:37 GMT
71+
Content-Length: 46
72+
73+
Please accept terms of licensing and agreement
74+
```
75+
76+
77+
78+
## Request parameters
79+
80+
This table describes how HTTP request parts map to MQTT PUBLISH packet properties. Refer to the original documentation for full details.
81+
82+
| **MQTT PUBLISH Part** | **Type/Values** | **Location** | **Required** | **Description** |
83+
|--------------------------|------------------------|--------------------------------------------|--------------------|--------------------------------|
84+
| Topic name | Percent-encoded string | Query `topic` | Yes | MQTT topic to publish to. |
85+
| QoS | 0 or 1 | Query `qos` or header `mqtt-qos` | No [default = 1] | Quality of Service level. |
86+
| RETAIN flag | 0 or 1 | Query `retain` or header `mqtt-retain` | No [default = 0] | Whether to retain the message. |
87+
| Response Topic | Percent-encoded string | Header `mqtt-response-topic` | No | Response topic if needed. |
88+
| Correlation Data | Base64 string | Header `mqtt-correlation-data` | No | Extra data for tracking. |
89+
| User Properties | Base64 JSON Array | Header `mqtt-user-properties` | No | Custom user properties. |
90+
| Content Type | String | Header `content-type` | No | Payload type. |
91+
| Message Expiry Interval | Unsigned integer | Header `mqtt-message-expiry` | No | Retention period in seconds. |
92+
| Payload Format Indicator | 0 or 1 | Header `mqtt-payload-format-indicator` | No [default = 0] | Format indicator. |
93+
| Payload | bytes | HTTP body | No | Message body. |
94+
95+
**Notes:**
96+
97+
- Query parameter values override header values if both are present.
98+
- Percent-encoding is required for topic and response topic.
99+
- Correlation Data must be base64-encoded.
100+
101+
## High-level steps for using HTTP Publish
102+
103+
- Step 1: Prepare your Entra ID bearer token for authentication.
104+
- Step 2: Construct your HTTP POST request to your Event Grid MQTT broker endpoint.
105+
- Step 3: Include required query parameters like topic.
106+
- Step 4: Add optional headers for QoS, RETAIN, response topics, user properties, etc.
107+
- Step 5: Add your payload as the HTTP body.
108+
- Step 6: Send the request.
109+
- Step 7: Confirm delivery via logs and metrics in the Event Grid portal.
110+
111+
## Authentication & authorization
112+
113+
- HTTP Publish uses Microsoft Entra ID for authentication.
114+
- Add a Bearer token to the Authorization header.
115+
- The Microsoft Entra Object ID becomes the MQTT clientId.
116+
- The AuthN/AuthZ model aligns with standard MQTT connections.
117+
118+
## Routing and observability
119+
120+
- Metrics and logs include:
121+
- Protocol: http-publish
122+
- Request ID
123+
- Topic
124+
- Source IP
125+
- Auth principal
126+
127+
## Best practices
128+
129+
- Use lowercase header keys where possible. HTTP/2 header keys are case-insensitive.
130+
- Monitor throughput — HTTP messages tend to be larger than direct MQTT messages.
131+
- HTTP Publish shares throughput limits with direct MQTT publishes.
132+
133+
## Throttling
134+
135+
Be aware that HTTP Publish counts towards your overall MQTT throughput quota. Monitor your usage to avoid exceeding limits.
136+
137+
## Next step
138+
See [How to publish MQTT messages using HTTP with Azure Event Grid](mqtt-how-to-http-publish.md).

0 commit comments

Comments
 (0)