Skip to content
This repository was archived by the owner on Aug 15, 2025. It is now read-only.

Commit 195b041

Browse files
bitfl0werbitfl0wer
committed
2 parents 3f7ec48 + e3e5224 commit 195b041

File tree

2 files changed

+169
-22
lines changed

2 files changed

+169
-22
lines changed

‎docs/Protocol Specifications/core.md‎

Lines changed: 168 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -34,13 +34,13 @@ weight: 0
3434
- [3.2.3.3 Service channels](#3233-service-channels)
3535
- [3.2.3.4 New session event](#3234-new-session-event)
3636
- [3.2.3.5 Actor certificate invalidation event](#3235-actor-certificate-invalidation-event)
37-
- [3.2.3.6 Resume event](#3236-resume-event)
37+
- [3.2.3.6 "Resume" event and "resumed" event](#3236-resume-event-and-resumed-event)
3838
- [3.2.3.7 Server certificate change event](#3237-server-certificate-change-event)
3939
- [3.2.3.8 Heartbeat and heartbeat ACK events](#3238-heartbeat-and-heartbeat-ack-events)
4040
- [3.2.4 Establishing a connection](#324-establishing-a-connection)
4141
- [3.2.5 Closing a connection](#325-closing-a-connection)
4242
- [3.2.6 Guaranteed delivery of gateway messages through package acknowledgement](#326-guaranteed-delivery-of-gateway-messages-through-package-acknowledgement)
43-
- [3.3 Events over REST](#33-events-over-rest)
43+
- [3.3 Events over events](#33-events-over-events)
4444
- [3.4 HTTP](#34-http)
4545
- [3.5 Internet Protocol (IP)](#35-internet-protocol-ip)
4646
- [3.6 Compression](#36-compression)
@@ -271,11 +271,12 @@ The following opcodes are defined by the `core` namespace:
271271
| `2` | Identify | Actor Send | Identify to the server. |
272272
| `3` | New Session | Actor Receive | Received by all sessions except the new one. |
273273
| `4` | Actor Certificate Invalidation | Actor Send/Receive | An actor certificate has been invalidated. Sent *to* server when an actor invalidates one of their certificates. |
274-
| `5` | Resume | Actor Send/Receive | Replay events after re-connecting. |
274+
| `5` | Resume | Actor Send | Request the replaying events after re-connecting. |
275275
| `6` | Server Certificate Change | Actor Receive | Received when the server's certificate changed. |
276276
| `7` | Heartbeat ACK | Actor Receive | Acknowledgement of a heartbeat |
277277
| `8` | Service Channel | Actor Send/Receive | Open or close a service channel. |
278278
| `9` | Service Channel ACK | Actor Receive | Acknowledgement of a service channel event. |
279+
| `10` | Resumed | Actor Receive | Replayed events. |
279280

280281
##### 3.2.1.3 Sequence numbers `s`
281282

@@ -335,7 +336,25 @@ in milliseconds at which the client should send heartbeat events to the server.
335336
The "identify" event is sent by the client to the server to let the server know which actor the
336337
client is.
337338

338-
TODO
339+
!!! example "Example identify event payload"
340+
341+
```json
342+
{
343+
"n": "core",
344+
"op": 2,
345+
"d": {
346+
"token": "a9144379a161e1fcf6b07801b70db6d6c481..."
347+
}
348+
}
349+
```
350+
351+
| Field | Type | Description |
352+
| ------- | ------ | --------------------------------------------------------------------------------------------------------------------- |
353+
| `token` | string | A [session token](#41-authentication) issued by the server, identifying the session the client wants to connect with. |
354+
355+
!!! info
356+
357+
This event may be extended in a backwards-compatible manner in future versions of polyproto.
339358

340359
##### 3.2.3.3 Service channels
341360

@@ -461,12 +480,94 @@ occurs.
461480
| `invalidSince` | uint64 | UNIX timestamp of the point in time where this ID-Cert became invalid on |
462481
| `signature` | string | Signature of a string concatenation of the `invalidSince` timestamp and the `serial` number, in that order. Clients must verify this signature, verifying that the signature was generated by the private key of the revoked certificate. |
463482

464-
##### 3.2.3.6 Resume event
483+
##### 3.2.3.6 "Resume" event and "resumed" event
465484

466485
When a client re-connects to a polyproto WebSocket gateway server, the client may send a resume event
467-
to the server instead of identifying.
486+
to the server instead of identifying. The resumed event sent by the server informs the client
487+
about everything the client has missed since their last active connection to the gateway.
468488

469-
TODO
489+
TODO: Must the resume event not contain a token?
490+
491+
!!! example "Example resume event structure"
492+
493+
```json
494+
{
495+
"n": "core",
496+
"op": 5,
497+
"d": {
498+
"s": 12
499+
}
500+
}
501+
```
502+
503+
| Field | Type | Description |
504+
| ----- | ------ | --------------------------------------------------------------------------------------- |
505+
| `s` | uint64 | Sequence number of the last event received by the client; aka. "Where to receive from". |
506+
507+
!!! example "Example "resumed" event"
508+
509+
```json
510+
{
511+
"n": "core",
512+
"op": 10,
513+
"d": {
514+
[
515+
{
516+
"n": "core",
517+
"op": 6,
518+
"d": {
519+
"payload data"
520+
}
521+
},
522+
{
523+
"n": "core",
524+
"op": 9,
525+
"d": {
526+
...
527+
}
528+
},
529+
...
530+
]
531+
},
532+
}
533+
```
534+
535+
As touched on earlier, the "resumed" event contains all relevant events the client has missed.
536+
537+
"Missed events" are events with
538+
539+
$$
540+
s_{event} \gt s_{resume}
541+
$$
542+
543+
where $s_{resume}$ is equal to the parameter `s` supplied by the client in the resume event.
544+
545+
A set of "relevant events" is a set of events which meet both of the following conditions:
546+
547+
1. Each event in the set is intended to be received by the client
548+
2. The set must contain the lowest possible amount of events necessary for the client to be informed
549+
about everything that happened while they were disconnected.
550+
551+
!!! example "Example for condition #2"
552+
553+
Assume, that an event "total number of messages sent" exists. The value of this event
554+
payload is a number, representing the total number of messages sent on the entire server. Under
555+
normal circumstances, each client receives this imaginary event every time this state changes.
556+
557+
For the client to resume, the server should not send each individual update of this value to the
558+
client as part of the "resumed" event. Instead, it would be sufficient to send the most
559+
up-to-date value of this event as part of the "resumed" payload, since how many times this event
560+
has been fired and what previous values of this event were, has no impact
561+
on the validity or state of other events.
562+
563+
Certificate change events are an example of events, where all intermediary values of the event
564+
are important as well. This is because a client could have sent a message where the signature was
565+
generated using a revoked certificate. In other words, intermediary values of this event type
566+
affect the validity or state of other events.
567+
568+
Servers may reject a clients' wish to resume, if the number of events that would need to be replayed
569+
is too high for the server to process. In this case, the request to resume is met with a close code
570+
of `4010` by the server and the connection is terminated.
470571

471572
##### 3.2.3.7 Server certificate change event
472573

@@ -495,7 +596,7 @@ of this event contains the ASCII-PEM encoded ID-Cert of the server.
495596
##### 3.2.3.8 Heartbeat and heartbeat ACK events
496597

497598
The heartbeat event is sent by the client to the server to keep the WebSocket connection alive.
498-
The payload for the heartbeat event is a "minified number list". Minified number lists are a JSON
599+
The payload for the heartbeat event is a minified number list. Minified number lists are a JSON
499600
object with the fields `from`, `to`, and `except`. The `from` and `to` fields are strings representing
500601
a range of numbers. The `except` field is an array of strings representing numbers that are not
501602
included in the range.
@@ -534,40 +635,78 @@ $$
534635
{
535636
"n": "core",
536637
"op": 0,
537-
"d": {},
638+
"d": {
639+
"from": "213",
640+
"to": "219"
641+
"except": ["214", "216"]
642+
},
538643
"s": 1
539644
}
540645
```
541646

647+
A heartbeat ACK contains events that the client has re-requested as part of their heartbeat message.
648+
542649
!!! example "Example heartbeat ACK event payload"
543650

544651
```json
545652
{
546653
"n": "core",
547654
"op": 7,
548-
"d": {},
655+
"d": {
656+
[
657+
{
658+
"n": "core",
659+
"op": 6,
660+
"d": {
661+
"payload data"
662+
}
663+
},
664+
{
665+
"n": "core",
666+
"op": 9,
667+
"d": {
668+
...
669+
}
670+
},
671+
...
672+
]
673+
},
549674
"s": 1
550675
}
551676
```
552677

678+
As such, the field `d` in a heartbeat ack is optional (`?`) and, if present, contains an array of
679+
other gateway events. Heartbeat ACK payloads must not be present in this array, making recursion
680+
impossible.
681+
553682
#### 3.2.4 Establishing a connection
554683

555684
TODO
556685

557686
#### 3.2.5 Closing a connection
558687

559-
TODO
688+
At any time during the connection, the server or client may wish to terminate the session in an
689+
orderly fashion. This is being done by sending a [WebSocket close code](https://www.rfc-editor.org/rfc/rfc6455.html#section-7.1.5)
690+
to the recipient. In addition to the pre-defined status codes in [IETF RFC #6455](https://www.rfc-editor.org/rfc/rfc6455.html),
691+
polyproto servers and clients must know of and use the following status codes in their appropriate
692+
situations:
693+
694+
| Code | Description | Explanation | Eligible for `RESUME`? | Sent by server? | Sent by client? |
695+
| ------ | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ---------------------- | --------------- | --------------- |
696+
| `4000` | Unknown error | An unknown error has occurred and the connection was terminated. | x | x | x |
697+
| `4001` | Unknown opcode | The client has sent a message with an unknown [opcode](#3212-opcodes-op) to the server. | x | x | |
698+
| `4002` | Invalid payload | The client has sent a message with an invalid payload to the server. | x | x | |
699+
| `4003` | Not authenticated | The server has received a message from the client before the client identified itself, or the clients' session has been invalidated. | | x | |
700+
| `4004` | Invalid authentication | The authentication token received by the server as part of the identify payload is invalid. | | x | |
701+
| `4005` | Already authenticated | The client has sent an identify payload even though it has already identified successfully. | | x | |
702+
| `4006` | Reserved | 4006 is a reserved value and has no function in polyproto v1.0. The specific meaning may be defined in the future. | | | |
703+
| `4007` | Invalid sequence number(s) | The client has sent a heartbeat containing sequence numbers that were invalid. | x | x | |
704+
| `4008` | Rate limited | The client has sent payloads too quickly. | | x | |
705+
| `4009` | Timeout | The session has been deemed to be timed out. This can happen if a heartbeat or heartbeat ACK has not been sent in due time. | x (If sent by server) | x | x |
706+
| `4010` | Unresumeable | The server has determined that this session cannot be resumed. The client should initiate a new, fresh connection to the gateway instead. | | x | |
560707

561708
#### 3.2.6 Guaranteed delivery of gateway messages through package acknowledgement
562709

563-
TODO
564-
565-
!!! bug "TODO"
566-
567-
This section will explain how [heartbeats](#322-heartbeats) and [sequence numbers](#3213-sequence-numbers-s)
568-
come together to form an application-layer package acknowledgement mechanism, and how that
569-
mechanism works.
570-
571710
polyproto implements an application-level guaranteed delivery mechanism. This ensures that all gateway
572711
messages sent from a home server to a client are received by the client in the order they were sent
573712
in – especially when network conditions are suboptimal. This mechanism is based on the use of
@@ -584,9 +723,16 @@ in – especially when network conditions are suboptimal. This mechanism is base
584723
retransmitted, preserving the integrity and completeness of communication between the client
585724
and server.
586725

726+
The [heartbeat payload](#3238-heartbeat-and-heartbeat-ack-events) defines a payload parameter `except`.
727+
728+
If `except` was present and contained entries in the heartbeat payload sent by a client, the server
729+
must re-send these events in the `d` part of the heartbeat ACK response. How this `d` payload is to be
730+
formatted is also defined in [section 3.2.3.8](#3238-heartbeat-and-heartbeat-ack-events).
587731

732+
The server must prioritize sending these "missed" events over other events. The server should expect
733+
that a client requests these events yet another time.
588734

589-
#### 3.3 Events over REST
735+
#### 3.3 Events over events
590736

591737
For some implementation contexts, a constant WebSocket connection might not be wanted. A client can
592738
instead opt to query an API endpoint to receive events, which would normally be sent through the WebSocket
@@ -649,7 +795,8 @@ TODO
649795

650796
!!! bug "TODO; Here's a TL;DR:"
651797

652-
- zstd level 5-13 recommended
798+
- zstd level 5-13 recommended for realtime
799+
- higher zstd levels recommended for events such as resumed etc.
653800
- zstd must be offered on gateway server
654801
- zstd must be offered on http api
655802
- uncompressed available (but why would you do that)

‎requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ paginate==0.5.7
2222
pathspec==0.12.1
2323
platformdirs==4.3.6
2424
Pygments==2.19.1
25-
pymdown-extensions==10.14
25+
pymdown-extensions==10.14.1
2626
python-dateutil==2.9.0.post0
2727
pytz==2024.2
2828
PyYAML==6.0.2

0 commit comments

Comments
 (0)