Skip to content

Commit 0ab0e1e

Browse files
davidbrochartjasongroutblink1073
authored
Add WebSocket wire protocol documentation (#693)
Co-authored-by: Jason Grout <[email protected]> Co-authored-by: Steven Silvester <[email protected]>
1 parent f973883 commit 0ab0e1e

File tree

2 files changed

+156
-0
lines changed

2 files changed

+156
-0
lines changed

‎docs/source/developers/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,4 @@ These pages target people writing Jupyter Web applications and server extensions
1313
extensions
1414
savehooks
1515
contents
16+
websocket-protocols

‎docs/source/developers/websocket-protocols.rst

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
.. _websocket_protocols:
2+
3+
WebSocket kernel wire protocols
4+
===============================
5+
6+
The Jupyter Server needs to pass messages between kernels and the Jupyter web application. Kernels use ZeroMQ sockets, and the web application uses a WebSocket.
7+
8+
ZeroMQ wire protocol
9+
--------------------
10+
11+
The kernel wire protocol over ZeroMQ takes advantage of multipart messages, allowing to decompose a message into parts and to send and receive them unmerged. The following table shows the message format (the beginning has been omitted for clarity):
12+
13+
.. list-table:: Format of a kernel message over ZeroMQ socket (indices refer to parts, not bytes)
14+
:header-rows: 1
15+
16+
* - ...
17+
- 0
18+
- 1
19+
- 2
20+
- 3
21+
- 4
22+
- 5
23+
- ...
24+
* - ...
25+
- header
26+
- parent_header
27+
- metadata
28+
- content
29+
- buffer_0
30+
- buffer_1
31+
- ...
32+
33+
See also the `Jupyter Client documentation <https://jupyter-client.readthedocs.io/en/stable/messaging.html#the-wire-protocol>`_.
34+
35+
Note that a set of ZeroMQ sockets, one for each channel (shell, iopub, etc.), are multiplexed into one WebSocket. Thus, the channel name must be encoded in WebSocket messages.
36+
37+
WebSocket protocol negotiation
38+
------------------------------
39+
40+
When opening a WebSocket, the Jupyter web application can optionally provide a list of subprotocols it supports (see e.g. the `MDN documentation <https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers#subprotocols>`_). If nothing is provided (empty list), then the Jupyter Server assumes the default protocol will be used. Otherwise, the Jupyter Server must select one of the provided subprotocols, or none of them. If none of them is selected, the Jupyter Server must reply with an empty string, which means that the default protocol will be used.
41+
42+
Default WebSocket protocol
43+
--------------------------
44+
45+
The Jupyter Server must support the default protocol, in which a kernel message is serialized over WebSocket as follows:
46+
47+
.. list-table:: Format of a kernel message over WebSocket (indices refer to bytes)
48+
:header-rows: 1
49+
50+
* - 0
51+
- 4
52+
- 8
53+
- ...
54+
- offset_0
55+
- offset_1
56+
- offset_2
57+
- ...
58+
* - offset_0
59+
- offset_1
60+
- offset_2
61+
- ...
62+
- msg
63+
- buffer_0
64+
- buffer_1
65+
- ...
66+
67+
Where:
68+
69+
* ``offset_0`` is the position of the kernel message (``msg``) from the beginning of this message, in bytes.
70+
* ``offset_1`` is the position of the first binary buffer (``buffer_0``) from the beginning of this message, in bytes (optional).
71+
* ``offset_2`` is the position of the second binary buffer (``buffer_1``) from the beginning of this message, in bytes (optional).
72+
* ``msg`` is the kernel message, excluding binary buffers and including the channel name, as a UTF8-encoded stringified JSON.
73+
* ``buffer_0`` is the first binary buffer (optional).
74+
* ``buffer_1`` is the second binary buffer (optional).
75+
76+
The message can be deserialized by parsing ``msg`` as a JSON object (after decoding it to a string):
77+
78+
.. code-block:: python
79+
80+
msg = {
81+
'channel': channel,
82+
'header': header,
83+
'parent_header': parent_header,
84+
'metadata': metadata,
85+
'content': content
86+
}
87+
88+
Then retrieving the channel name, and updating with the buffers, if any:
89+
90+
.. code-block:: python
91+
92+
buffers = {
93+
[
94+
buffer_0,
95+
buffer_1
96+
# ...
97+
]
98+
}
99+
100+
``v1.kernel.websocket.jupyter.org`` protocol
101+
--------------------------------------------
102+
103+
The Jupyter Server can optionally support the ``v1.kernel.websocket.jupyter.org`` protocol, in which a kernel message is serialized over WebSocket as follows:
104+
105+
.. list-table:: Format of a kernel message over WebSocket (indices refer to bytes)
106+
:header-rows: 1
107+
108+
* - 0
109+
- 8
110+
- 16
111+
- ...
112+
- 8*offset_number
113+
- offset_0
114+
- offset_1
115+
- offset_2
116+
- offset_3
117+
- offset_4
118+
- offset_5
119+
- offset_6
120+
- ...
121+
* - offset_number
122+
- offset_0
123+
- offset_1
124+
- ...
125+
- offset_n
126+
- channel
127+
- header
128+
- parent_header
129+
- metadata
130+
- content
131+
- buffer_0
132+
- buffer_1
133+
- ...
134+
135+
Where:
136+
137+
* ``offset_number`` is a 64-bit (little endian) unsigned integer.
138+
* ``offset_0`` to ``offset_n`` are 64-bit (little endian) unsigned integers (with ``n=offset_number-1``).
139+
* ``channel`` is a UTF-8 encoded string containing the channel for the message (shell, iopub, etc.).
140+
* ``header``, ``parent_header``, ``metadata``, and ``content`` are UTF-8 encoded JSON text representing the given part of a message in the Jupyter message protocol.
141+
* ``offset_n`` is the number of bytes in the message.
142+
* The message can be deserialized from the ``bin_msg`` serialized message as follows (Python code):
143+
144+
.. code-block:: python
145+
146+
import json
147+
channel = bin_msg[offset_0:offset_1].decode('utf-8')
148+
header = json.loads(bin_msg[offset_1:offset_2])
149+
parent_header = json.loads(bin_msg[offset_2:offset_3])
150+
metadata = json.loads(bin_msg[offset_3:offset_4])
151+
content = json.loads(bin_msg[offset_4:offset_5])
152+
buffer_0 = bin_msg[offset_5:offset_6]
153+
buffer_1 = bin_msg[offset_6:offset_7]
154+
# ...
155+
last_buffer = bin_msg[offset_n_minus_1:offset_n]

0 commit comments

Comments
 (0)