Skip to content

Commit f5f8799

Browse files
PMEds28PMEds28
authored
Merge pull request #182881 from zackliu/exactlyonce
[Web PubSub] Add exactly-once message delivery related document
2 parents 3a7b33a + 21d2f93 commit f5f8799

11 files changed

+1041
-515
lines changed

‎articles/azure-web-pubsub/howto-develop-reliable-clients.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
---
2+
title: Create reliable Websocket clients
3+
description: How to create reliable Websocket clients
4+
author: chenyl
5+
ms.author: chenyl
6+
ms.service: azure-web-pubsub
7+
ms.topic: reference
8+
ms.date: 12/15/2021
9+
---
10+
11+
# Create reliable Websocket with subprotocol
12+
13+
Websocket client connections may drop due to intermittent network issue and when connections drop, messages will also be lost. In a pubsub system, publishers are decoupled from subscribers, so publishers hard to detect subscribers' drop or message loss. It's crucial for clients to overcome intermittent network issue and keep the reliability of message delivery. To achieve that, you can create a reliable Websocket client with the help of reliable subprotocols.
14+
15+
> [!NOTE]
16+
> Reliable protocols are still in preview. Some changes are expected in future.
17+
18+
## Reliable Protocol
19+
20+
Service supports two reliable subprotocols `json.reliable.webpubsub.azure.v1` and `protobuf.reliable.webpubsub.azure.v1`. Clients must follow the protocol, mainly including the part of reconnection, publisher and subscriber to achieve the reliability, or the message delivery may not work as expected or the service may terminate the client as it violates the protocol spec.
21+
22+
## Initialization
23+
24+
To use reliable subprotocols, you must set subprotocol when constructing Websocket connections. In JavaScript, you can use as following:
25+
26+
- Use Json reliable subprotocol
27+
```js
28+
var pubsub = new WebSocket('wss://test.webpubsub.azure.com/client/hubs/hub1', 'json.reliable.webpubsub.azure.v1');
29+
```
30+
31+
- Use Protobuf reliable subprotocol
32+
```js
33+
var pubsub = new WebSocket('wss://test.webpubsub.azure.com/client/hubs/hub1', 'protobuf.reliable.webpubsub.azure.v1');
34+
```
35+
36+
## Reconnection
37+
38+
Websocket connections relay on TCP, so if the connection doesn't drop, all messages should be lossless and in order. When facing network issue and connections drop, all the status such as group and message info are kept by the service and wait for reconnection to recover. A Websocket connection owns a session in the service and the identifier is `connectionId`. Reconnection is the basis of achieving reliability and must be implemented. When a new connection connects to the service using reliable subprotocols, the connection will receive a `Connected` message contains `connectionId` and `reconnectionToken`.
39+
40+
```json
41+
{
42+
"type":"system",
43+
"event":"connected",
44+
"connectionId": "<connection_id>",
45+
"reconnectionToken": "<reconnection_token>"
46+
}
47+
```
48+
49+
Once the WebSocket connection dropped, the client should first try to reconnect with the same `connectionId` to keep the session. Clients don't need to negotiate with server and obtain the `access_token`. Instead, reconnection should make a websocket connect request to service directly with `connection_id` and `reconnection_token` with the following uri:
50+
51+
```
52+
wss://<service-endpoint>/client/hubs/<hub>?awps_connection_id=<connection_id>&awps_reconnection_token=<reconnection_token>
53+
```
54+
55+
Reconnection may fail as network issue hasn't been recovered yet. Client should keep retrying reconnecting until
56+
1. Websocket connection closed with status code 1008. The status code means the connectionId has been removed from the service.
57+
2. Reconnection failure keeps more than 1 minute.
58+
59+
## Publisher
60+
61+
Clients who send events to event handler or publish message to other clients are called publishers in the document. Publishers should set `ackId` to the message to get acknowledged from the service about whether the message publishing success or not. The `ackId` in message is the identifier of the message, which means different messages should use different `ackId`s, while resending message should keep the same `ackId` for the service to de-duplicate.
62+
63+
A sample group send message:
64+
```json
65+
{
66+
"type": "sendToGroup",
67+
"group": "group1",
68+
"dataType" : "text",
69+
"data": "text data",
70+
"ackId": 1
71+
}
72+
```
73+
74+
A sample ack response:
75+
```json
76+
{
77+
"type": "ack",
78+
"ackId": 1,
79+
"success": true
80+
}
81+
```
82+
83+
If the service returns ack with `success: true`, the message has been processed by the service and the client can expect the message will be delivered to all subscribers.
84+
85+
However, In some cases, Service meets some transient internal error and the message can't be sent to subscriber. In such case, publisher will receive an ack like following and should resend message with the same `ackId` if it's necessary based on business logic.
86+
87+
```json
88+
{
89+
"type": "ack",
90+
"ackId": 1,
91+
"success": false,
92+
"error": {
93+
"name": "InternalServerError",
94+
"message": "Internal server error"
95+
}
96+
}
97+
```
98+
99+
![Message Failure](./media/howto-develop-reliable-clients/message-failed.png)
100+
101+
Service's ack may be dropped because of WebSockets connection dropped. So, publishers should get notified when Websocket connection drops and resend message with the same `ackId` after reconnection. If the message has actually processed by the service, it will response ack with `Duplicate` and publishers should stop resending this message again.
102+
103+
```json
104+
{
105+
"type": "ack",
106+
"ackId": 1,
107+
"success": false,
108+
"error": {
109+
"name": "Duplicate",
110+
"message": "Message with ack-id: 1 has been processed"
111+
}
112+
}
113+
```
114+
115+
![Message duplicated](./media/howto-develop-reliable-clients/message-duplicated.png)
116+
117+
## Subscriber
118+
119+
Clients who receive messages sent from event handlers or publishers are called subscriber in the document. When connections drop by network issues, the service doesn't know how many messages are actually sent to subscribers. So subscribers should tell the service which message has been received. Data Messages contains `sequenceId` and subscribers must ack the sequence-id with sequence ack message:
120+
121+
A sample sequence ack:
122+
```json
123+
{
124+
"type": "sequenceAck",
125+
"sequenceId": 1
126+
}
127+
```
128+
129+
The sequence-id is a uint64 incremental number with-in a connection-id session. Subscribers should record the largest sequence-id it ever received and accept all messages with larger sequence-id and drop all messages with smaller or equal sequence-id. The sequence ack supports cumulative ack, which means if you ack `sequence-id=5`, the service will treat all messages with sequence-id smaller than 5 have already been received by subscribers. Subscribers should ack with the largest sequence-id it recorded, so that the service can skip redelivering messages that subscribers have already received.
130+
131+
All messages are delivered to subscribers in order until the WebSockets connection drops. With sequence-id, the service can have the knowledge about how many messages subscribers have actually received across WebSockets connections with-in a connection-id session. After a WebSockets connection drop, the service will redeliver messages it should deliver but not ack-ed by the subscriber. The service hold messages that are not ack-ed with limit, if messages exceed the limit, the service will close the WebSockets connection and remove the connection-id session. Thus, subscribers should ack the sequence-id as soon as possible.

‎articles/azure-web-pubsub/includes/reference-json-requests.md

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
---
2+
author: vicancy
3+
ms.author: lianwei
4+
ms.service: azure-web-pubsub
5+
ms.topic: include
6+
ms.date: 08/06/2021
7+
---
8+
9+
### Join groups
10+
11+
Format:
12+
13+
```json
14+
{
15+
"type": "joinGroup",
16+
"group": "<group_name>",
17+
"ackId" : 1
18+
}
19+
```
20+
21+
* `ackId` is the identity of each request and should be unique. The service sends a [ack response message](#ack-response) to notify the process result of the request. More details can be found at [AckId and Ack Response](../concept-client-protocols.md#ackid-and-ack-response)
22+
23+
### Leave groups
24+
25+
Format:
26+
27+
```json
28+
{
29+
"type": "leaveGroup",
30+
"group": "<group_name>",
31+
"ackId" : 1
32+
}
33+
```
34+
35+
* `ackId` is the identity of each request and should be unique. The service sends a [ack response message](#ack-response) to notify the process result of the request. More details can be found at [AckId and Ack Response](../concept-client-protocols.md#ackid-and-ack-response)
36+
37+
### Publish messages
38+
39+
Format:
40+
41+
```json
42+
{
43+
"type": "sendToGroup",
44+
"group": "<group_name>",
45+
"ackId" : 1,
46+
"noEcho": true|false,
47+
"dataType" : "json|text|binary",
48+
"data": {}, // data can be string or valid json token depending on the dataType
49+
}
50+
```
51+
52+
* `ackId` is the identity of each request and should be unique. The service sends a [ack response message](#ack-response) to notify the process result of the request. More details can be found at [AckId and Ack Response](../concept-client-protocols.md#ackid-and-ack-response)
53+
* `noEcho` is optional. If set to true, this message is not echoed back to the same connection. If not set, the default value is false.
54+
* `dataType` can be one of `json`, `text`, or `binary`:
55+
* `json`: `data` can be any type that JSON supports and will be published as what it is; If `dataType` isn't specified, it defaults to `json`.
56+
* `text`: `data` should be in string format, and the string data will be published;
57+
* `binary`: `data` should be in base64 format, and the binary data will be published;
58+
59+
#### Case 1: publish text data:
60+
```json
61+
{
62+
"type": "sendToGroup",
63+
"group": "<group_name>",
64+
"dataType" : "text",
65+
"data": "text data",
66+
"ackId": 1
67+
}
68+
```
69+
70+
* What subprotocol client in this group `<group_name>` receives:
71+
```json
72+
{
73+
"type": "message",
74+
"from": "group",
75+
"group": "<group_name>",
76+
"dataType" : "text",
77+
"data" : "text data"
78+
}
79+
```
80+
* What the raw client in this group `<group_name>` receives is string data `text data`.
81+
82+
#### Case 2: publish JSON data:
83+
```json
84+
{
85+
"type": "sendToGroup",
86+
"group": "<group_name>",
87+
"dataType" : "json",
88+
"data": {
89+
"hello": "world"
90+
}
91+
}
92+
```
93+
94+
* What subprotocol client in this group `<group_name>` receives:
95+
```json
96+
{
97+
"type": "message",
98+
"from": "group",
99+
"group": "<group_name>",
100+
"dataType" : "json",
101+
"data" : {
102+
"hello": "world"
103+
}
104+
}
105+
```
106+
* What the raw client in this group `<group_name>` receives is serialized string data `{"hello": "world"}`.
107+
108+
109+
#### Case 3: publish binary data:
110+
```json
111+
{
112+
"type": "sendToGroup",
113+
"group": "<group_name>",
114+
"dataType" : "binary",
115+
"data": "<base64_binary>",
116+
"ackId": 1
117+
}
118+
```
119+
120+
* What subprotocol client in this group `<group_name>` receives:
121+
```json
122+
{
123+
"type": "message",
124+
"from": "group",
125+
"group": "<group_name>",
126+
"dataType" : "binary",
127+
"data" : "<base64_binary>",
128+
}
129+
```
130+
* What the raw client in this group `<group_name>` receives is the **binary** data in the binary frame.
131+
132+
### Send custom events
133+
134+
Format:
135+
136+
```json
137+
{
138+
"type": "event",
139+
"event": "<event_name>",
140+
"ackId": 1,
141+
"dataType" : "json|text|binary",
142+
"data": {}, // data can be string or valid json token depending on the dataType
143+
}
144+
```
145+
146+
* `ackId` is the identity of each request and should be unique. The service sends a [ack response message](#ack-response) to notify the process result of the request. More details can be found at [AckId and Ack Response](../concept-client-protocols.md#ackid-and-ack-response)
147+
148+
`dataType` can be one of `text`, `binary`, or `json`:
149+
* `json`: data can be any type json supports and will be published as what it is; If `dataType` is not specified, it defaults to `json`.
150+
* `text`: data should be in string format, and the string data will be published;
151+
* `binary`: data should be in base64 format, and the binary data will be published;
152+
153+
#### Case 1: send event with text data:
154+
```json
155+
{
156+
"type": "event",
157+
"event": "<event_name>",
158+
"ackId": 1,
159+
"dataType" : "text",
160+
"data": "text data",
161+
}
162+
```
163+
164+
What the upstream event handler receives like below, the `Content-Type` for the CloudEvents HTTP request is `text/plain` for `dataType`=`text`
165+
166+
```HTTP
167+
POST /upstream HTTP/1.1
168+
Host: xxxxxx
169+
WebHook-Request-Origin: xxx.webpubsub.azure.com
170+
Content-Type: text/plain
171+
Content-Length: nnnn
172+
ce-specversion: 1.0
173+
ce-type: azure.webpubsub.user.<event_name>
174+
ce-source: /client/{connectionId}
175+
ce-id: {eventId}
176+
ce-time: 2021-01-01T00:00:00Z
177+
ce-signature: sha256={connection-id-hash-primary},sha256={connection-id-hash-secondary}
178+
ce-userId: {userId}
179+
ce-connectionId: {connectionId}
180+
ce-hub: {hub_name}
181+
ce-eventName: <event_name>
182+
183+
text data
184+
185+
```
186+
187+
#### Case 2: send event with JSON data:
188+
```json
189+
{
190+
"type": "event",
191+
"event": "<event_name>",
192+
"ackId": 1,
193+
"dataType" : "json",
194+
"data": {
195+
"hello": "world"
196+
},
197+
}
198+
```
199+
200+
What the upstream event handler receives like below, the `Content-Type` for the CloudEvents HTTP request is `application/json` for `dataType`=`json`
201+
202+
```HTTP
203+
POST /upstream HTTP/1.1
204+
Host: xxxxxx
205+
WebHook-Request-Origin: xxx.webpubsub.azure.com
206+
Content-Type: application/json
207+
Content-Length: nnnn
208+
ce-specversion: 1.0
209+
ce-type: azure.webpubsub.user.<event_name>
210+
ce-source: /client/{connectionId}
211+
ce-id: {eventId}
212+
ce-time: 2021-01-01T00:00:00Z
213+
ce-signature: sha256={connection-id-hash-primary},sha256={connection-id-hash-secondary}
214+
ce-userId: {userId}
215+
ce-connectionId: {connectionId}
216+
ce-hub: {hub_name}
217+
ce-eventName: <event_name>
218+
219+
{
220+
"hello": "world"
221+
}
222+
223+
```
224+
225+
#### Case 3: send event with binary data:
226+
```json
227+
{
228+
"type": "event",
229+
"event": "<event_name>",
230+
"ackId": 1,
231+
"dataType" : "binary",
232+
"data": "base64_binary",
233+
}
234+
```
235+
236+
What the upstream event handler receives like below, the `Content-Type` for the CloudEvents HTTP request is `application/octet-stream` for `dataType`=`binary`
237+
238+
```HTTP
239+
POST /upstream HTTP/1.1
240+
Host: xxxxxx
241+
WebHook-Request-Origin: xxx.webpubsub.azure.com
242+
Content-Type: application/octet-stream
243+
Content-Length: nnnn
244+
ce-specversion: 1.0
245+
ce-type: azure.webpubsub.user.<event_name>
246+
ce-source: /client/{connectionId}
247+
ce-id: {eventId}
248+
ce-time: 2021-01-01T00:00:00Z
249+
ce-signature: sha256={connection-id-hash-primary},sha256={connection-id-hash-secondary}
250+
ce-userId: {userId}
251+
ce-connectionId: {connectionId}
252+
ce-hub: {hub_name}
253+
ce-eventName: <event_name>
254+
255+
binary
256+
257+
```
258+
259+
The WebSocket frame can be `text` format for text message frames or UTF8 encoded binaries for `binary` message frames.
260+
261+
Service declines the client if the message does not match the described format.

0 commit comments

Comments
 (0)