Merge pull request #281411 from Y-Sindo/mqtt

Add MQTT for WebPubSub

Author: Jill Grant

articles/azure-web-pubsub/howto-connect-mqtt-websocket-client.md

@@ -0,0 +1,149 @@
+---
+title: How to connect MQTT clients to Azure Web PubSub
+description: How to connect MQTT clients to Azure Web PubSub
+author: Y-Sindo
+ms.author: zityang
+ms.service: azure-web-pubsub
+ms.date: 06/17/2024
+ms.topic: how-to
+---
+
+# How to connect MQTT clients to Azure Web PubSub
+
+MQTT is a lightweight pub/sub messaging protocol designed for devices with constrained resources.
+
+In this article, we introduce how to connect MQTT clients to the service, so that the clients can publish and subscribe messages.
+
+## Connection parameters
+
+WebSocket connection URI: `wss://{serviceName}.webpubsub.azure.com/clients/mqtt/hubs/{hub}?access_token={token}`.
+
+* {hub} is a mandatory parameter that provides isolation for different applications.
+* {token} is required by default. Alternatively, you can include the token in the `Authorization` header in the format `Bearer {token}`. You can bypass the token requirement by enabling anonymous access to the hub. <!--TODO MQTT allow anonymous access to the hub-->
+
+If client library doesn't accept a URI, then you probably need to split the information in the URI into multiple parameters:
+
+* Host: `{serviceName}.webpubsub.azure.com`
+* Path: `/clients/mqtt/hubs/{hub}?access_token={token}`
+* Port: 443
+* Transport: WebSockets with [TLS](https://wikipedia.org/wiki/Transport_Layer_Security).
+
+[!INCLUDE [MQTT-Connection-parameters](includes/mqtt-connection-parameters.md)]
+
+By default MQTT clients don't have any permissions to publish or subscribe to any topics. You need to grant [permissions](#permissions) to MQTT clients.
+
+## Permissions
+
+A client can publish to other clients only when it's *authorized* to do so. A client's permissions can be granted when it's being connected or during the lifetime of the connection.
+
+| Role | Permission |
+|---|---|
+| Not specified | The client can send event requests. |
+| `webpubsub.joinLeaveGroup` | The client can join or leave any group. |
+| `webpubsub.sendToGroup` | The client can publish messages to any group. |
+| `webpubsub.joinLeaveGroup.<group>` | The client can join or leave group `<group>`. |
+| `webpubsub.sendToGroup.<group>` | The client can publish messages to group `<group>`. |
+| | |
+
+## Authentication and authorization
+
+There are two workflows supported by Web PubSub to authenticate and authorize MQTT clients, so that they have proper permissions.
+
+These workflows can be used individually or in combination. If they're used in together, the auth result in the latter workflow would be honored by the service.
+
+### 1. JWT workflow
+
+This is the default workflow, shown as follows:
+
+![Diagram of MQTT auth workflow with JWT.](./media/howto-connect-mqtt-websocket-client/mqtt-jwt-auth-workflow.png)
+
+1. The client negotiates with your auth server. The auth server contains the authorization middleware, which handles the client request and signs a JWT for the client to connect to the service.
+1. The auth server returns the JWT to the client.
+1. The client tries to connect to the Web PubSub service with the JWT token returned from the auth server. The token can be in either the query string, as `/clients/mqtt/hubs/{hub}?access_token={token}`, or the `Authorization` header, as `Authorization: Bearer {token}`.
+
+#### Supported claims
+You could also configure properties for the client connection when generating the access token by specifying special claims inside the JWT token:
+
+| Description | Claim type | Claim value | Notes |
+| --- | --- | --- | --- |
+| The [permissions](#permissions) the client connection initially has | `role` | the role value defined in [permissions](#permissions) | Specify multiple `role` claims if the client has multiple permissions. |
+| The lifetime of the token | `exp` | the expiration time | The `exp` (expiration time) claim identifies the expiration time on or after which the token MUST NOT be accepted for processing. |
+| The initial groups that the client connection joins once it connects to Azure Web PubSub | `group` | the group to join | Specify multiple `group` claims if the client joins multiple groups. |
+| The `userId` for the client connection | `sub` | the userId | Only one `sub` claim is allowed. |
+
+You could also add custom claims into the access token, and these values are preserved as the `claims` property in [connect upstream request body](./reference-mqtt-cloud-events.md#system-connect-event).
+
+[Server SDKs](./howto-generate-client-access-url.md#generate-from-service-sdk) provides APIs to generate the access token for MQTT clients. Note that you must specify the client protocol to `Mqtt`.
+
+# [JavaScript](#tab/javascript)
+
+1. Follow [Getting started with server SDK](./reference-server-sdk-js.md#getting-started) to create a `WebPubSubServiceClient` object `service`
+
+> [!NOTE]
+> Generating MQTT client access URL is supported since [version 1.1.3](https://www.npmjs.com/package/@azure/web-pubsub/v/1.1.3?activeTab=versions).
+
+2. Generate Client Access URL by calling `WebPubSubServiceClient.getClientAccessToken`:
+
+     ```js
+     let token = await serviceClient.getClientAccessToken({ clientProtocol: "mqtt" });
+     ```
+
+# [C#](#tab/csharp)
+
+1. Follow [Getting started with server SDK](./reference-server-sdk-csharp.md#getting-started) to create a `WebPubSubServiceClient` object `service`
+
+> [!NOTE]
+> Generating MQTT client access URL is supported since [version 1.4.0](https://www.nuget.org/packages/Azure.Messaging.WebPubSub/1.4.0).
+
+2. Generate Client Access URL by calling `WebPubSubServiceClient.GetClientAccessUri`:
+
+     ```csharp
+     var url = service.GetClientAccessUri(clientProtocol: WebPubSubClientProtocol.Mqtt);
+     ```
+
+# [Python](#tab/python)
+
+1. Follow [Getting started with server SDK](./reference-server-sdk-python.md#install-the-package) to create a `WebPubSubServiceClient` object `service`
+
+> [!NOTE]
+> Generating MQTT client access URL is supported since [version 1.2.0](https://pypi.org/project/azure-messaging-webpubsubservice/1.2.0/).
+
+2. Generate Client Access URL by calling `WebPubSubServiceClient.get_client_access_token`:
+
+     ```python
+     token = service.get_client_access_token(client_protocol="MQTT")
+     ```
+
+# [Java](#tab/java)
+
+1. Follow [Getting started with server SDK](./reference-server-sdk-java.md#getting-started) to create a `WebPubSubServiceClient` object `service`
+
+> [!NOTE]
+> Generating MQTT client access URL is supported since [version 1.3.0](https://central.sonatype.com/artifact/com.azure/azure-messaging-webpubsub/1.3.0).
+
+
+2. Generate Client Access URL by calling `WebPubSubServiceClient.getClientAccessToken`:
+
+     ```java
+     GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();
+     option.setWebPubSubClientProtocol(WebPubSubClientProtocol.MQTT);
+     WebPubSubClientAccessToken token = service.getClientAccessToken(option);
+     ```
+---
+
+### 2. Upstream server workflow
+
+The MQTT client sends an MQTT CONNECT packet after it establishes a WebSocket connection with the service, then the service calls an API in the upstream server. The upstream server can auth the client according to the username and password fields in the MQTT connection request, and the TLS certificate from the client.
+
+![Diagram of MQTT auth workflow with upstream server](./media/howto-connect-mqtt-websocket-client/mqtt-upstream-auth-workflow.png)
+
+This workflow needs explicit configuration.
+* [Tutorial - Authenticate and authorize MQTT clients based on client certificates](./tutorial-upstream-auth-mqtt-client.md)
+* For details about how to use upstream server to auth the clients, see [How to configure event handler](./howto-develop-eventhandler.md)
+
+## Troubleshooting
+
+If you're experiencing failure of connection, or unable to publish or subscribe messages, please check the reason code / return code from the service, or see [How to troubleshoot with resource logs](./howto-troubleshoot-resource-logs.md).
+
+
+

articles/azure-web-pubsub/howto-develop-event-listener.md

@@ -12,6 +12,7 @@ ms.date: 09/30/2022
 
 > [!NOTE]
 > Event listener feature is in preview.
+> Sending MQTT client events to event listener is not supported yet.
 
 ## Overview
 
@@ -76,7 +77,7 @@ Find your Azure Web PubSub service from **Azure portal**. Navigate to **Identity
 
 In this article, you learned how event listeners work and how to configure an event listener with an event hub endpoint. To learn the data format sent to Event Hubs, read the following specification.
 
-> [!div class="nextstepaction"] 
+> [!div class="nextstepaction"]
 > [Specification: CloudEvents AMQP extension for Azure Web PubSub](./reference-cloud-events-amqp.md)
 
 <!--TODO: Add demo-->

articles/azure-web-pubsub/howto-generate-client-access-url.md

@@ -10,16 +10,24 @@ ms.topic: how-to
 
 # How to generate client access URL for the clients
 
-A client, be it a browser 💻, a mobile app 📱, or an IoT device 💡, uses a **Client Access URL** to connect and authenticate with your resource. This URL follows a pattern of `wss://<service_name>.webpubsub.azure.com/client/hubs/<hub_name>?access_token=<token>`. This article shows you several ways to get the Client Access URL.
+A client, be it a browser 💻, a mobile app 📱, or an IoT device 💡, uses a **Client Access URL** to connect and authenticate with your resource.
+
+The URL follows the below pattern:
+* For MQTT clients, it's `wss://<service_name>.webpubsub.azure.com/clients/mqtt/hubs/<hub_name>?access_token=<token>`.
+* For all other clients, it's `wss://<service_name>.webpubsub.azure.com/client/hubs/<hub_name>?access_token=<token>`.
+
+This article shows you several ways to get the Client Access URL.
 
 - For quick start, copy one from the Azure portal
-- For development, generate the value using [Web PubSub server SDK](./reference-server-sdk-js.md)
+- For development, generate the value using [Web PubSub server SDK](#generate-from-service-sdk)
 - If you're using Microsoft Entra ID, you can also invoke the [Generate Client Token REST API](/rest/api/webpubsub/dataplane/web-pub-sub/generate-client-token)
 
 ## Copy from the Azure portal
 
 In the Keys tab in Azure portal, there's a Client URL Generator tool to quickly generate a Client Access URL for you, as shown in the following diagram. Values input here aren't stored.
 
+Note that for MQTT clients, you should select "MQTT Client" in the dropdown menu in front of the "Client Access URL" text box.
+
 :::image type="content" source="./media/howto-websocket-connect/generate-client-url.png" alt-text="Screenshot of the Web PubSub Client URL Generator.":::
 
 ## Generate from service SDK
@@ -32,6 +40,12 @@ The same Client Access URL can be generated by using the Web PubSub server SDK.
 
 2. Generate Client Access URL by calling `WebPubSubServiceClient.getClientAccessToken`:
 
+   - Generate MQTT client access token
+
+     ```js
+     let token = await serviceClient.getClientAccessToken({ clientProtocol: "mqtt" });
+     ```
+
    - Configure user ID
 
      ```js
@@ -76,6 +90,12 @@ The same Client Access URL can be generated by using the Web PubSub server SDK.
 
 2. Generate Client Access URL by calling `WebPubSubServiceClient.GetClientAccessUri`:
 
+   - Generate MQTT client access token
+
+     ```csharp
+     var url = service.GetClientAccessUri(clientProtocol: WebPubSubClientProtocol.Mqtt);
+     ```
+
    - Configure user ID
 
      ```csharp
@@ -112,6 +132,12 @@ The same Client Access URL can be generated by using the Web PubSub server SDK.
 
 2. Generate Client Access URL by calling `WebPubSubServiceClient.get_client_access_token`:
 
+   - Generate MQTT client access token
+
+     ```python
+     token = service.get_client_access_token(client_protocol="MQTT")
+     ```
+
    - Configure user ID
 
      ```python
@@ -148,7 +174,15 @@ The same Client Access URL can be generated by using the Web PubSub server SDK.
 
 2. Generate Client Access URL by calling `WebPubSubServiceClient.getClientAccessToken`:
 
-   - Configure user ID
+   - Generate MQTT client access token
+
+     ```java
+     GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();
+     option.setWebPubSubClientProtocol(WebPubSubClientProtocol.MQTT);
+     WebPubSubClientAccessToken token = service.getClientAccessToken(option);
+     ```
+
+    - Configure user ID
 
      ```java
      GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();
@@ -192,18 +226,18 @@ The same Client Access URL can be generated by using the Web PubSub server SDK.
 
 In real-world code, we usually have a server side to host the logic generating the Client Access URL. When a client request comes in, the server side can use the general authentication/authorization workflow to validate the client request. Only valid client requests can get the Client Access URL back.
 
-## Invoke the Generate Client Token REST API
+## Invoke the "Generate Client Token" REST API
 
 You can enable Microsoft Entra ID in your service and use the Microsoft Entra token to invoke [Generate Client Token rest API](/rest/api/webpubsub/dataplane/web-pub-sub/generate-client-token) to get the token for the client to use.
 
 1. Follow [Authorize from application](./howto-authorize-from-application.md) to enable Microsoft Entra ID.
 2. Follow [Get Microsoft Entra token](./howto-authorize-from-application.md#use-postman-to-get-the-microsoft-entra-token) to get the Microsoft Entra token with Postman.
 3. Use the Microsoft Entra token to invoke `:generateToken` with Postman:
-   
+
    > [!NOTE]
    > Please use the latest version of Postman. Old versions of Postman have [some issue](https://github.com/postmanlabs/postman-app-support/issues/3994#issuecomment-893453089) supporting colon `:` in path.
 
-   1. For the URI, enter `https://{Endpoint}/api/hubs/{hub}/:generateToken?api-version=2022-11-01`
+   1. For the URI, enter `https://{Endpoint}/api/hubs/{hub}/:generateToken?api-version=2024-01-01`. If you'd like to generate token for MQTT clients, append query parameter `&clientType=mqtt` to the URL.
    2. On the **Auth** tab, select **Bearer Token** and paste the Microsoft Entra token fetched in the previous step
    3. Select **Send** and you see the Client Access Token in the response:
 
@@ -212,5 +246,3 @@ You can enable Microsoft Entra ID in your service and use the Microsoft Entra to
         "token": "ABCDEFG.ABC.ABC"
       }
       ```
-
-5. The Client Access URI is in the format of `wss://<endpoint>/client/hubs/<hub_name>?access_token=<token>`

articles/azure-web-pubsub/includes/mqtt-connection-parameters.md

@@ -0,0 +1,16 @@
+---
+author: Y-Sindo
+ms.author: zityang
+ms.service: azure-web-pubsub
+ms.topic: include
+ms.date: 06/22/2024
+---
+
+There are some limitations you should follow in your [MQTT](https://mqtt.org/mqtt-specification/) clients, otherwise the connection will be rejected. These MQTT protocol parameters include:
+* Protocol versions: 3.1.1 or 5.0.
+* Client ID format
+    * Allowed characters: 0-9, a-z, A-Z
+    * Length is between 1 and 128
+* Keep-alive interval for MQTT 3.1.1: 1 - 180 seconds
+* Last-will Topic format: Not empty, and contains at least one nonwhitespace character. The max length is 1024.
+* Last-will message size: up to 2,000 bytes

articles/azure-web-pubsub/includes/mqtt-term-mappings.md

@@ -0,0 +1,15 @@
+---
+author: Y-Sindo
+ms.author: zityang
+ms.service: azure-web-pubsub
+ms.topic: include
+ms.date: 07/19/2024
+---
+
+| MQTT terms| Corresponding Web PubSub terms | Relationship |
+| --- | --- | --- |
+| Server/MQTT Broker | Web PubSub Service  |  Web PubSub service work as MQTT brokers to serve MQTT connections. Please note that we usually use the term *server* to refer to the upstream server instead of the MQTT brokers in the documents. |
+| Session | Connection  | *Connection* in Web PubSub is a logical concept that represents a stateful relationship between the client and service, and one *Connection* is corresponding to one *Session*. Usually these two words are interchangeable.  |
+| Subscribe To A Topic | Join A Group | These two actions have the same effect: the client will receive messages from that topic or group. Topic name is the group name.  |
+| Publish Message To A Topic | Send Message To A Group |  These two actions have the same effect: the client who subscribes to that topic or belong to that group will receive the message |
+| Client ID | Connection ID | *Connection ID* identifies a *Connection* to Web PubSub. We use the *Client ID* as the *Connection ID* of MQTT connections in Web PubSub. |