From d330ba91df63bdd664feec822a97ffd40f0f81d7 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 15 Apr 2022 14:04:02 -0400 Subject: [PATCH 01/29] Add initial MSC for read receipts for threads. --- proposals/3771-read-receipts-for-threads.md | 154 ++++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 proposals/3771-read-receipts-for-threads.md diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md new file mode 100644 index 00000000000..6eb629131c6 --- /dev/null +++ b/proposals/3771-read-receipts-for-threads.md @@ -0,0 +1,154 @@ +# MSC3771: Read receipts for threads + +Currently, each room has a single read receipt per user. This is used to sync the +read status of a room across clients and calculate the number of unread messages. + +Unfortunately a client displaying threads may show a subset of a room's messages +at a time, causing the read receipt to be misleading. + +This might better be described by an example. Given a room with the following +DAG of events (note that the dotted lines are a thread relation, as specified by +[MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440)): + +```mermaid +flowchart RL + F-->E + E-->D + D-->C + C-->B + B-->A + D-.->A + F-.->A +``` + +A client might interpret this as: + +```mermaid +flowchart RL + subgraph Main timeline + C-->B + B-->A + F-->C + end + subgraph Thread timeline + D-->A + E-->D + end + + style A fill:cyan,stroke:#333,stroke-width:2px + style E fill:orange,stroke:#333,stroke-width:2px +``` + +While viewing the "main" timeline of the room, a client might move the read +receipt from event `A` to event `E` without ever showing events `D` and `F`. The +user then reads the thread, the client has no way to mark `F` as read. + +## Proposal + +This MSC proposes adding a new receipt type of `m.read.thread.private` is to be +added. This receipt type is used for clients to sync the read status of each +thread in a room. + +The `/receipt` endpoint gains a new optional path part and becomes: + +`POST /_matrix/client/v3/rooms/{roomId}/receipt/{receiptType}/{eventId}/{context}` + +The `context` contains the thread that the read receipts belongs to (i.e. it should +match the `event_id` contained within the `m.relates_to` of the event represented +by `eventId`). This updates the unique tuple for receipts from +`(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, context)`. +For backwards compatibility, a missing `context` is equivalent to an empty context. + +Given a threaded message: + +```json +{ + "event_id": "$thread_reply", + "room_id": "!room:example.org", + "content": { + "m.relates_to": { + "rel_type": "m.thread", + "event_id": "$thread_root" + } + } +} +``` + +A client could mark this as read by sending a request: + +``` +POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read.thread.private/$thread_reply/$thread_root + +{} +``` + +Similarly to the hidden read receipts from [MSC2285](https://github.com/matrix-org/matrix-spec-proposals/pull/2285), +homeservers are not to send the receipt to any other users except the sender nor +over federation. + +## Potential issues + +For long-lived rooms or rooms with many threads there could be a significant number +of receipts. This has a few downsides: + +* The size of the `/sync` response would increase without bound. +* The effort to generate and process the receipts for each room would increase + without bound. + +Due to both of the above, this proposal is limited to a *private* read receipt for +threads. This limits the impact to a user's own read receipts for threads, but +does not completely solve the issue. + +[MSC2285](https://github.com/matrix-org/matrix-spec-proposals/pull/2285) suggests +that the `/read_markers` endpoint should become a generic bulk receipt endpoint. +This is not compatible with the additional `context` parameter in this MSC. + +## Alternatives + +Instead of adding the thread ID as a new path part, it could be added to the body +of the receipt. There may be a small backwards compatibility benefit to this, but +it seems clearer to put it as part of the URL. + +Similarly, instead of adding a new receipt type, the read status of each thread in +a room could be added to the body of the `m.read.private` receipt. This could +cause data integrity issues if multiple clients attempt to update the receipt +without first reading it. + +Instead of adding the thread ID as a new path part, it could be encoded as a suffix +on the receipt, e.g. `m.thread.read.private-$abcd`. This has the benefit of not +changing the current receipt mechanisms, but seems sub-par and requires additional +parsing of opaque identifiers. + +## Security considerations + +There is potential for abuse by allowing clients to specify a unique `context`. +A mitigation could be to ensure that it is the related event of the thread, ensuring +that each thread only has a single context. + +## Future extensions + +Future extensions will tackle how the thread read receipts impact notification counts. + +## Unstable prefix + +During implementation the receipt type shall be `org.matrix.mscXXXX`. + +To avoid receipts that will never be updated, after stabilization, homeservers are +expected to reject the unstable receipt type and purge all receipts using the +unstable receipt type. + +To detect server support, clients can either rely on the spec version (when stable) +or the presence of a `org.matrix.msc3771` flag in `unstable_features` on `/versions`. + +## Dependencies + +This MSC depends on the following MSCs, which at the time of writing have not yet +been accepted into the spec: + +* [MSC2285](https://github.com/matrix-org/matrix-spec-proposals/pull/2285): Hidden read receipts + +As well as the following MSCs, which have been accepted into the spec, but have +yet to be released: + +* [MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674): Event Relationships +* [MSC3440](https://github.com/matrix-org/matrix-spec-proposals/pull/3440): Threading via `m.thread` relation From 42d7daffb68bb9f77351f369fe46c341d5e8a15e Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 20 Apr 2022 13:05:56 -0400 Subject: [PATCH 02/29] Fix events in diagram. --- proposals/3771-read-receipts-for-threads.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 6eb629131c6..64c501bfed6 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -17,8 +17,8 @@ flowchart RL D-->C C-->B B-->A - D-.->A - F-.->A + C-.->A + E-.->A ``` A client might interpret this as: @@ -26,22 +26,22 @@ A client might interpret this as: ```mermaid flowchart RL subgraph Main timeline - C-->B + D-->B B-->A - F-->C + F-->D end subgraph Thread timeline - D-->A - E-->D + C-->A + E-->C end style A fill:cyan,stroke:#333,stroke-width:2px - style E fill:orange,stroke:#333,stroke-width:2px + style F fill:orange,stroke:#333,stroke-width:2px ``` While viewing the "main" timeline of the room, a client might move the read -receipt from event `A` to event `E` without ever showing events `D` and `F`. The -user then reads the thread, the client has no way to mark `F` as read. +receipt from event `A` to event `F` without ever showing events `C` and `E`. The +user then reads the thread, the client has no way to mark `E` as read. ## Proposal From 4c5382dbe292dbbf6c85341eedf32854918c5ffd Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 20 Apr 2022 13:43:48 -0400 Subject: [PATCH 03/29] Add sync response. --- proposals/3771-read-receipts-for-threads.md | 22 +++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 64c501bfed6..e5d2870ca57 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -86,6 +86,28 @@ Similarly to the hidden read receipts from [MSC2285](https://github.com/matrix-o homeservers are not to send the receipt to any other users except the sender nor over federation. +This would then come down `/sync` for the user with other receipts: + +```json +{ + "content": { + "$thread_reply": { + "m.read.thread.private": { + "@rikj:jki.re": { + "ts": 1436451550453, + "context": "$thread_root" + } + } + } + }, + "room_id": "!jEsUZKDJdhlrceRyVU:example.org", + "type": "m.receipt" +} +``` + +Since this is not shared, only the own user's matrix ID would be expected to +have this kind of receipt. + ## Potential issues For long-lived rooms or rooms with many threads there could be a significant number From b5dac99d0e1ea75919a718db710d630eb8e3983a Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 26 Apr 2022 11:20:56 -0400 Subject: [PATCH 04/29] Link to the spec. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3771-read-receipts-for-threads.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index e5d2870ca57..30560f755ae 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -49,7 +49,8 @@ This MSC proposes adding a new receipt type of `m.read.thread.private` is to be added. This receipt type is used for clients to sync the read status of each thread in a room. -The `/receipt` endpoint gains a new optional path part and becomes: +The [`/receipt`](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) +endpoint gains a new optional path part and becomes: `POST /_matrix/client/v3/rooms/{roomId}/receipt/{receiptType}/{eventId}/{context}` From 94450495afc90d7834dff6180c71490767041e70 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 26 Apr 2022 11:30:19 -0400 Subject: [PATCH 05/29] Clarify sentence. --- proposals/3771-read-receipts-for-threads.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 30560f755ae..ddc38ba9077 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -45,9 +45,8 @@ user then reads the thread, the client has no way to mark `E` as read. ## Proposal -This MSC proposes adding a new receipt type of `m.read.thread.private` is to be -added. This receipt type is used for clients to sync the read status of each -thread in a room. +This MSC proposes adding a new receipt type of `m.read.thread.private`, which is +used for clients to sync the read status of each thread in a room. The [`/receipt`](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) endpoint gains a new optional path part and becomes: From 0e42ec3ec10a94646df26cdf36cad69d1ba8ca90 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 24 May 2022 09:24:29 -0400 Subject: [PATCH 06/29] Some clarifications. --- proposals/3771-read-receipts-for-threads.md | 29 ++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index ddc38ba9077..acf13b7f5a9 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -45,19 +45,33 @@ user then reads the thread, the client has no way to mark `E` as read. ## Proposal -This MSC proposes adding a new receipt type of `m.read.thread.private`, which is -used for clients to sync the read status of each thread in a room. +### Multiple receipts in a room + +This MSC proposes allowing the same receipt type to exist multiple times in a room +by adding a `context` parameter to reach receipt. The [`/receipt`](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) endpoint gains a new optional path part and becomes: `POST /_matrix/client/v3/rooms/{roomId}/receipt/{receiptType}/{eventId}/{context}` +For backwards compatibility, a missing `context` is equivalent to an empty context. +It is up to receipt types to define whether it is valid for them to have a `context`, +it is invalid to provide a `context` for all currently known receipt types (`m.read`, +`m.fully_read`, `m.read.private`). + +### A thread receipt type + +This MSC also proposes a new receipt type of `m.read.thread.private`, which is +used for clients to sync the read status of each thread in a room. + The `context` contains the thread that the read receipts belongs to (i.e. it should match the `event_id` contained within the `m.relates_to` of the event represented by `eventId`). This updates the unique tuple for receipts from `(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, context)`. -For backwards compatibility, a missing `context` is equivalent to an empty context. +A missing (or empty) `context` refers to the "main" timeline (or events which are +not part of a thread). A client which understands threads is expected to only +set the `m.read.thread.private` receipt type. Given a threaded message: @@ -108,6 +122,15 @@ This would then come down `/sync` for the user with other receipts: Since this is not shared, only the own user's matrix ID would be expected to have this kind of receipt. +## Compatibility with unthreaded clients + +When a user has clients which are both "unthreaded" and "threaded" then the read +receipt and threaded read receipts would get out of sync. It is desirable to +sync these two sets of read receipts and minimize duplicate notifications that +occur from these two sets of read receipts. + + + ## Potential issues For long-lived rooms or rooms with many threads there could be a significant number From e74c989d4fc4bd3ff8fa88044498c0fafef60d12 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 6 Jul 2022 13:46:20 -0400 Subject: [PATCH 07/29] Simplification. --- proposals/3771-read-receipts-for-threads.md | 99 ++++++--------------- 1 file changed, 29 insertions(+), 70 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index acf13b7f5a9..a19f6a56582 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -45,33 +45,22 @@ user then reads the thread, the client has no way to mark `E` as read. ## Proposal -### Multiple receipts in a room - This MSC proposes allowing the same receipt type to exist multiple times in a room -by adding a `context` parameter to reach receipt. +by adding a `threadId` parameter to reach receipt. The [`/receipt`](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) endpoint gains a new optional path part and becomes: -`POST /_matrix/client/v3/rooms/{roomId}/receipt/{receiptType}/{eventId}/{context}` - -For backwards compatibility, a missing `context` is equivalent to an empty context. -It is up to receipt types to define whether it is valid for them to have a `context`, -it is invalid to provide a `context` for all currently known receipt types (`m.read`, -`m.fully_read`, `m.read.private`). - -### A thread receipt type +`POST /_matrix/client/v3/rooms/{roomId}/receipt/{receiptType}/{eventId}/{threadId}` -This MSC also proposes a new receipt type of `m.read.thread.private`, which is -used for clients to sync the read status of each thread in a room. - -The `context` contains the thread that the read receipts belongs to (i.e. it should +The `threadId` contains the thread that the read receipts belongs to (i.e. it should match the `event_id` contained within the `m.relates_to` of the event represented -by `eventId`). This updates the unique tuple for receipts from -`(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, context)`. -A missing (or empty) `context` refers to the "main" timeline (or events which are -not part of a thread). A client which understands threads is expected to only -set the `m.read.thread.private` receipt type. +by `eventId`). + +This updates the unique tuple for receipts from +`(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, threadId)`. +A missing (or empty) `threadId` refers to the "main" timeline (or events which are +not part of a thread). Given a threaded message: @@ -91,16 +80,13 @@ Given a threaded message: A client could mark this as read by sending a request: ``` -POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read.thread.private/$thread_reply/$thread_root +POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply/$thread_root {} ``` -Similarly to the hidden read receipts from [MSC2285](https://github.com/matrix-org/matrix-spec-proposals/pull/2285), -homeservers are not to send the receipt to any other users except the sender nor -over federation. - -This would then come down `/sync` for the user with other receipts: +This would then come down `/sync` for the user with other receipts, but with an +additional property in the body containing the thread ID: ```json { @@ -109,7 +95,7 @@ This would then come down `/sync` for the user with other receipts: "m.read.thread.private": { "@rikj:jki.re": { "ts": 1436451550453, - "context": "$thread_root" + "threadId": "$thread_root" } } } @@ -119,18 +105,6 @@ This would then come down `/sync` for the user with other receipts: } ``` -Since this is not shared, only the own user's matrix ID would be expected to -have this kind of receipt. - -## Compatibility with unthreaded clients - -When a user has clients which are both "unthreaded" and "threaded" then the read -receipt and threaded read receipts would get out of sync. It is desirable to -sync these two sets of read receipts and minimize duplicate notifications that -occur from these two sets of read receipts. - - - ## Potential issues For long-lived rooms or rooms with many threads there could be a significant number @@ -140,13 +114,15 @@ of receipts. This has a few downsides: * The effort to generate and process the receipts for each room would increase without bound. -Due to both of the above, this proposal is limited to a *private* read receipt for -threads. This limits the impact to a user's own read receipts for threads, but -does not completely solve the issue. - [MSC2285](https://github.com/matrix-org/matrix-spec-proposals/pull/2285) suggests that the `/read_markers` endpoint should become a generic bulk receipt endpoint. -This is not compatible with the additional `context` parameter in this MSC. +This is not compatible with the additional `threadId` parameter in this MSC. + +### Compatibility with unthreaded clients + +When a user has both a client which is "unthreaded" and "threaded" then there +is a possibility for read receipts to be misrepresented when switching between +clients. Solutions to this problem are deemed out of scope of this MSC. ## Alternatives @@ -154,21 +130,16 @@ Instead of adding the thread ID as a new path part, it could be added to the bod of the receipt. There may be a small backwards compatibility benefit to this, but it seems clearer to put it as part of the URL. -Similarly, instead of adding a new receipt type, the read status of each thread in -a room could be added to the body of the `m.read.private` receipt. This could -cause data integrity issues if multiple clients attempt to update the receipt -without first reading it. - -Instead of adding the thread ID as a new path part, it could be encoded as a suffix -on the receipt, e.g. `m.thread.read.private-$abcd`. This has the benefit of not -changing the current receipt mechanisms, but seems sub-par and requires additional -parsing of opaque identifiers. +Instead of encoding the thread ID as an integral part of the receipt, the read +threads could be added to the body of the receipt. This could cause data +integrity issues if multiple clients attempt to update the receipt without first +reading it. ## Security considerations -There is potential for abuse by allowing clients to specify a unique `context`. -A mitigation could be to ensure that it is the related event of the thread, ensuring -that each thread only has a single context. +There is potential for abuse by allowing clients to specify a unique `threadId`. +A mitigation could be to ensure that the receipt is related to an event of the +thread, ensuring that each thread only has a single receipt. ## Future extensions @@ -176,24 +147,12 @@ Future extensions will tackle how the thread read receipts impact notification c ## Unstable prefix -During implementation the receipt type shall be `org.matrix.mscXXXX`. - -To avoid receipts that will never be updated, after stabilization, homeservers are -expected to reject the unstable receipt type and purge all receipts using the -unstable receipt type. - To detect server support, clients can either rely on the spec version (when stable) or the presence of a `org.matrix.msc3771` flag in `unstable_features` on `/versions`. ## Dependencies -This MSC depends on the following MSCs, which at the time of writing have not yet -been accepted into the spec: - -* [MSC2285](https://github.com/matrix-org/matrix-spec-proposals/pull/2285): Hidden read receipts - -As well as the following MSCs, which have been accepted into the spec, but have -yet to be released: +This MSC depends on the following MSCs, which have been accepted into the spec, +but have yet to be released: -* [MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674): Event Relationships * [MSC3440](https://github.com/matrix-org/matrix-spec-proposals/pull/3440): Threading via `m.thread` relation From 65fa00965002a6a58eea06a584e0c9d3e19fc661 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 22 Jul 2022 09:44:00 -0400 Subject: [PATCH 08/29] Fix JSON key format. Co-authored-by: Tulir Asokan --- proposals/3771-read-receipts-for-threads.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index a19f6a56582..52a6044fc2c 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -95,7 +95,7 @@ additional property in the body containing the thread ID: "m.read.thread.private": { "@rikj:jki.re": { "ts": 1436451550453, - "threadId": "$thread_root" + "thread_id": "$thread_root" } } } From 1f0d6dd6b5bb59c220da782d1e17b651f7e6a692 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 22 Jul 2022 10:00:52 -0400 Subject: [PATCH 09/29] Add information on clearing notifications. --- proposals/3771-read-receipts-for-threads.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 52a6044fc2c..623acf6463d 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -45,6 +45,8 @@ user then reads the thread, the client has no way to mark `E` as read. ## Proposal +### Sending a threaded read receipt + This MSC proposes allowing the same receipt type to exist multiple times in a room by adding a `threadId` parameter to reach receipt. @@ -85,6 +87,8 @@ POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply/$th {} ``` +### Received threaded read receipts + This would then come down `/sync` for the user with other receipts, but with an additional property in the body containing the thread ID: @@ -105,6 +109,14 @@ additional property in the body containing the thread ID: } ``` +### Notifications + +[MSC3773](https://github.com/matrix-org/matrix-spec-proposals/pull/3773) discusses +how notifications for threads are created and returned to the client, but does +not provide a way to clear threaded notifications. A threaded read receipt should +clear notifications for the matching thread following the [same rules](https://spec.matrix.org/latest/client-server-api/#receiving-notifications) +as notifications which are not part of a thread. + ## Potential issues For long-lived rooms or rooms with many threads there could be a significant number @@ -155,4 +167,4 @@ or the presence of a `org.matrix.msc3771` flag in `unstable_features` on `/versi This MSC depends on the following MSCs, which have been accepted into the spec, but have yet to be released: -* [MSC3440](https://github.com/matrix-org/matrix-spec-proposals/pull/3440): Threading via `m.thread` relation +* [MSC3773](https://github.com/matrix-org/matrix-spec-proposals/pull/3773): Notifications for threads From 03ac0e0039b6b6e0d0e21b1d7c10ce74fba1050c Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 3 Aug 2022 09:49:13 -0400 Subject: [PATCH 10/29] Fix example. --- proposals/3771-read-receipts-for-threads.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 623acf6463d..c067bc3e12e 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -96,7 +96,7 @@ additional property in the body containing the thread ID: { "content": { "$thread_reply": { - "m.read.thread.private": { + "m.read": { "@rikj:jki.re": { "ts": 1436451550453, "thread_id": "$thread_root" From e3e677a4d77611695c07068d093ef04e3e024e76 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 8 Aug 2022 15:24:55 -0400 Subject: [PATCH 11/29] Update with current understanding. --- proposals/3771-read-receipts-for-threads.md | 29 ++++++++++++++++++--- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index c067bc3e12e..c479e8cee8d 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -109,6 +109,9 @@ additional property in the body containing the thread ID: } ``` +A missing (or empty) `thread_id` refers to the "main" timeline (or events which are +not part of a thread). + ### Notifications [MSC3773](https://github.com/matrix-org/matrix-spec-proposals/pull/3773) discusses @@ -138,14 +141,32 @@ clients. Solutions to this problem are deemed out of scope of this MSC. ## Alternatives +### Thread ID location + Instead of adding the thread ID as a new path part, it could be added to the body of the receipt. There may be a small backwards compatibility benefit to this, but it seems clearer to put it as part of the URL. -Instead of encoding the thread ID as an integral part of the receipt, the read -threads could be added to the body of the receipt. This could cause data -integrity issues if multiple clients attempt to update the receipt without first -reading it. +Instead of encoding the thread ID as an integral part of the receipt, all of the +read threads could be added to the body of the single receipt. This could cause +data integrity issues if multiple clients attempt to update the receipt without +first reading it. + +### Receipt type + +To potentially improve compatibility it could make sense to use a separate receipt +type (e.g. `m.read.thread`) as the read receipt for threads. Without some syncing +mechanism between unthreaded and threaded receipts this seems likely to cause +users to re-read the same notifications on threaded and unthreaded clients. + +While it is possible to map from an unthreaded read receipt to multiple threaded +read receipts, the opposite is not possible (to the author's knowledge). In short, +it seems the [compatibility issues discussed above](#compatibility-with-unthreaded-clients) +would not be solved by adding more receipt types. + +This also gets more complcated with the addition of the `m.read.private` receipt -- +would there additionally be an `m.read.private.thread`? How do you map between +all of these? ## Security considerations From 554025247f51b0680512defb339bf421228fcc8b Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 16 Aug 2022 14:13:40 -0400 Subject: [PATCH 12/29] Clarify introduction. --- proposals/3771-read-receipts-for-threads.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index c479e8cee8d..e364e621687 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -1,10 +1,12 @@ # MSC3771: Read receipts for threads -Currently, each room has a single read receipt per user. This is used to sync the -read status of a room across clients and calculate the number of unread messages. +Currently, each room can only have a single receipt of each type per user. The +read receipt (`m.read` or `m.read.private`) is used to sync the read status of a +room across clients, to share with other users which events have been read and +is used by the homeserver to calculate the number of unread messages. Unfortunately a client displaying threads may show a subset of a room's messages -at a time, causing the read receipt to be misleading. +at a time, causing a user's read receipt to be misleading. This might better be described by an example. Given a room with the following DAG of events (note that the dotted lines are a thread relation, as specified by @@ -45,10 +47,10 @@ user then reads the thread, the client has no way to mark `E` as read. ## Proposal -### Sending a threaded read receipt +### Threaded read receipts -This MSC proposes allowing the same receipt type to exist multiple times in a room -by adding a `threadId` parameter to reach receipt. +This MSC proposes allowing the same receipt type to exist multiple times in a room, +once per thread. The [`/receipt`](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) endpoint gains a new optional path part and becomes: @@ -87,7 +89,7 @@ POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply/$th {} ``` -### Received threaded read receipts +### Receiving threaded read receipts This would then come down `/sync` for the user with other receipts, but with an additional property in the body containing the thread ID: From bbdb23e591ab52caede76b29c1736c5b59b2c15e Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 16 Aug 2022 14:15:28 -0400 Subject: [PATCH 13/29] MSC3773 is not yet accepted. --- proposals/3771-read-receipts-for-threads.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index e364e621687..82e42532d01 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -187,7 +187,7 @@ or the presence of a `org.matrix.msc3771` flag in `unstable_features` on `/versi ## Dependencies -This MSC depends on the following MSCs, which have been accepted into the spec, -but have yet to be released: +This MSC depends on the following MSCs, which have not yet been accepted into +the spec: * [MSC3773](https://github.com/matrix-org/matrix-spec-proposals/pull/3773): Notifications for threads From 2678f3c0a94a3cc8cc44bc8c9c91bff624f56962 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 16 Aug 2022 14:51:43 -0400 Subject: [PATCH 14/29] Updates from feedback. --- proposals/3771-read-receipts-for-threads.md | 61 ++++++++++++++------- 1 file changed, 41 insertions(+), 20 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 82e42532d01..94db8850dbf 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -47,24 +47,23 @@ user then reads the thread, the client has no way to mark `E` as read. ## Proposal -### Threaded read receipts +### Threaded receipts This MSC proposes allowing the same receipt type to exist multiple times in a room, once per thread. -The [`/receipt`](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) -endpoint gains a new optional path part and becomes: +To denote that a receipt belongs to a thread, the body of the receipt can include +a `thread_id` property when calling the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid). -`POST /_matrix/client/v3/rooms/{roomId}/receipt/{receiptType}/{eventId}/{threadId}` - -The `threadId` contains the thread that the read receipts belongs to (i.e. it should +The `thread_id` contains the thread that the receipt belongs to (i.e. it should match the `event_id` contained within the `m.relates_to` of the event represented -by `eventId`). +by `eventId`). Omitting the `thread_id` corresponds to the receipt being for the +"main" timeline (or events which are not part of a thread). A non-string `thread_id` +(or empty) `thread_id` field is an error and should be rejected with a `400` error +with `errcode` of `M_INVALID_PARAM`. This updates the unique tuple for receipts from -`(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, threadId)`. -A missing (or empty) `threadId` refers to the "main" timeline (or events which are -not part of a thread). +`(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, thread ID)`. Given a threaded message: @@ -84,15 +83,18 @@ Given a threaded message: A client could mark this as read by sending a request: ``` -POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply/$thread_root +POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply -{} +{ + "thread_id": "$thread_root" +} ``` -### Receiving threaded read receipts +The `thread_id` property is not valid for `m.fully_read` receipts. -This would then come down `/sync` for the user with other receipts, but with an -additional property in the body containing the thread ID: +### Receiving threaded receipts + +This would then come down `/sync` for the user with other receipts: ```json { @@ -111,17 +113,23 @@ additional property in the body containing the thread ID: } ``` -A missing (or empty) `thread_id` refers to the "main" timeline (or events which are -not part of a thread). +Since [event bodies must be treated as untrusted](https://spec.matrix.org/latest/client-server-api/#room-event-format) +the `thread_id` field may be of an invalid form. It should be treated as missing +if the field is not a non-empty string. ### Notifications [MSC3773](https://github.com/matrix-org/matrix-spec-proposals/pull/3773) discusses how notifications for threads are created and returned to the client, but does -not provide a way to clear threaded notifications. A threaded read receipt should -clear notifications for the matching thread following the [same rules](https://spec.matrix.org/latest/client-server-api/#receiving-notifications) +not provide a way to clear threaded notifications. + +A threaded read receipt (i.e. a `m.read` or `m.read.private` receipt with a `thread_id` +property) should clear notifications for the matching thread following the +[same rules](https://spec.matrix.org/latest/client-server-api/#receiving-notifications) as notifications which are not part of a thread. +XXX Add an example here. + ## Potential issues For long-lived rooms or rooms with many threads there could be a significant number @@ -141,6 +149,10 @@ When a user has both a client which is "unthreaded" and "threaded" then there is a possibility for read receipts to be misrepresented when switching between clients. Solutions to this problem are deemed out of scope of this MSC. +### Second-order relations + +XXX Is this valid? + ## Alternatives ### Thread ID location @@ -178,7 +190,16 @@ thread, ensuring that each thread only has a single receipt. ## Future extensions -Future extensions will tackle how the thread read receipts impact notification counts. +### Threaded fully read markers + +The `m.fully_read` marker is not supported in threads, a future MSC could expand +support to this pseudo-receipt. + +### Setting threaded receipts using the `/read_markers` endpoint + +This MSC does not propose expanding the `/read_markers` endpoint to support threaded +receipts. A future MSC might expand this to support an object per receipt with +an event ID and thread ID or some other way of setting multiple receipts at once. ## Unstable prefix From ea77632bd5eef4663f8f88dc2ff85e6e1d6a5327 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 31 Aug 2022 13:34:08 -0400 Subject: [PATCH 15/29] Update from learnings from the proof of concept. --- proposals/3771-read-receipts-for-threads.md | 92 +++++++++++++++------ 1 file changed, 66 insertions(+), 26 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 94db8850dbf..95be0ef05cd 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -50,17 +50,24 @@ user then reads the thread, the client has no way to mark `E` as read. ### Threaded receipts This MSC proposes allowing the same receipt type to exist multiple times in a room, -once per thread. +once per thread (and once for the "main" timeline of the room). -To denote that a receipt belongs to a thread, the body of the receipt can include -a `thread_id` property when calling the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid). +The body of request to the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) +gains the following fields: -The `thread_id` contains the thread that the receipt belongs to (i.e. it should -match the `event_id` contained within the `m.relates_to` of the event represented -by `eventId`). Omitting the `thread_id` corresponds to the receipt being for the -"main" timeline (or events which are not part of a thread). A non-string `thread_id` -(or empty) `thread_id` field is an error and should be rejected with a `400` error -with `errcode` of `M_INVALID_PARAM`. + +* `thread_id` (`string`): The thread that the receipt belongs to (i.e. the + `event_id` contained within the `m.relates_to` of the event represented by + `eventId`). + + A special value of `"main"` corresponds to the receipt being for the "main" + timeline (i.e. events which are not part of a thread). + +The following conditions are errors and should be rejected with a `400` error +with `errcode` of `M_INVALID_PARAM`: + +* A non-string `thread_id` (or empty) `thread_id` field. +* Providing the `thread_id` properties for a receipt of type `m.fully_read`. This updates the unique tuple for receipts from `(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, thread ID)`. @@ -90,20 +97,28 @@ POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply } ``` -The `thread_id` property is not valid for `m.fully_read` receipts. +And to send a receipt on the "main" timeline (e.g. on the root event): + +``` +POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root -### Receiving threaded receipts +{ + "thread_id": "main" +} +``` -This would then come down `/sync` for the user with other receipts: +### Receiving threaded receipts via `/sync`. -```json +The client would receive this as part of `/sync` response similar to other receipts: + +```json5 { "content": { "$thread_reply": { "m.read": { "@rikj:jki.re": { "ts": 1436451550453, - "thread_id": "$thread_root" + "thread_id": "$thread_root" // or "main" } } } @@ -113,9 +128,15 @@ This would then come down `/sync` for the user with other receipts: } ``` -Since [event bodies must be treated as untrusted](https://spec.matrix.org/latest/client-server-api/#room-event-format) -the `thread_id` field may be of an invalid form. It should be treated as missing -if the field is not a non-empty string. +If there is no `thread_id` field than no thread information was supplied with +the receipt, clients may interpret this as only applying to the main timeline or +as applying across all threads, if possible. + +### Sending threaded receipts over federation + +Homeservers should provide the `thread_id` (as provided by the client) in the +[Receipt Metadata](https://spec.matrix.org/v1.3/server-server-api/#receipts) when +sending the `m.receipt` EDU over federation. ### Notifications @@ -125,13 +146,28 @@ not provide a way to clear threaded notifications. A threaded read receipt (i.e. a `m.read` or `m.read.private` receipt with a `thread_id` property) should clear notifications for the matching thread following the -[same rules](https://spec.matrix.org/latest/client-server-api/#receiving-notifications) -as notifications which are not part of a thread. +[current rules](https://spec.matrix.org/v1.3/client-server-api/#receiving-notifications), +but only clear notifications with a matching `thread_id`. + +An unthreaded read receipt (i.e. a `m.read` or `m.read.private` receipt *without* +a `thread_id`) should apply follow the [current rules](https://spec.matrix.org/v1.3/client-server-api/#receiving-notifications) +as today and disregard thread information when clearing notifications. + +Using the example room DAG from the preamble of this MSC, consider the following +scenarios (all of which start with 5 unread messages: 3 on the main timeline and +2 in thread `A`). -XXX Add an example here. +* A threaded read receipt sent for event `C` on thread `A`, results in: 3 unread + on the main timeline and 1 on thread `A`. +* A threaded read receipt sent for event `D` on the main timeline: 1 unread on + the main timeline and 2 on thread `A`. +* An unthreaded read receipt send for event `D`: 1 unread on the main timeline + and 1 on thread `A`. ## Potential issues +### Long-lived rooms + For long-lived rooms or rooms with many threads there could be a significant number of receipts. This has a few downsides: @@ -139,19 +175,23 @@ of receipts. This has a few downsides: * The effort to generate and process the receipts for each room would increase without bound. -[MSC2285](https://github.com/matrix-org/matrix-spec-proposals/pull/2285) suggests -that the `/read_markers` endpoint should become a generic bulk receipt endpoint. -This is not compatible with the additional `threadId` parameter in this MSC. - ### Compatibility with unthreaded clients When a user has both a client which is "unthreaded" and "threaded" then there is a possibility for read receipts to be misrepresented when switching between -clients. Solutions to this problem are deemed out of scope of this MSC. +clients. Using the example room DAG from the preamble of this MSC: + +* A user which has an unthreaded receipt on event `D` and a threaded receipt on + event `E` would likely see event `E` as unread on an "unthreaded" client. + +Solutions to this problem are deemed out of scope of this MSC. ### Second-order relations -XXX Is this valid? +For simplicity, clients may wish to send receipts for events which are not directly +related to a root thread event (e.g. a reaction to thread event). This should +generally be treated as acceptable by the server (i.e. the server should only care +about the ordering of events, not about whether the events are in the same thread). ## Alternatives From 1c60533b3b913d6fb70cfd9b9fb04605c297dd7c Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 6 Sep 2022 15:32:05 -0400 Subject: [PATCH 16/29] Add link to the current spec. Co-authored-by: Travis Ralston --- proposals/3771-read-receipts-for-threads.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 95be0ef05cd..a862d98c637 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -1,7 +1,7 @@ # MSC3771: Read receipts for threads Currently, each room can only have a single receipt of each type per user. The -read receipt (`m.read` or `m.read.private`) is used to sync the read status of a +read receipt ([`m.read`](https://spec.matrix.org/v1.3/client-server-api/#receipts) or [`m.read.private`](https://github.com/matrix-org/matrix-spec-proposals/pull/2285)) is used to sync the read status of a room across clients, to share with other users which events have been read and is used by the homeserver to calculate the number of unread messages. From 347d33283a1a709aa91c32c6d49217b59cb32663 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 6 Sep 2022 15:39:00 -0400 Subject: [PATCH 17/29] Clarify that false positives are deliberate in the design. --- proposals/3771-read-receipts-for-threads.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index a862d98c637..94891d13c54 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -184,7 +184,12 @@ clients. Using the example room DAG from the preamble of this MSC: * A user which has an unthreaded receipt on event `D` and a threaded receipt on event `E` would likely see event `E` as unread on an "unthreaded" client. -Solutions to this problem are deemed out of scope of this MSC. +The proposed solution may result in events being incorrectly marked as unread +(when they have been read). The false positive for unread notifications is +deliberate to avoid losing message / missing notifications. + +Solutions to this problem are deemed out of scope of this MSC. A solution that +was briefly explored was [ranged read receipts](https://hackmd.io/Gxm8zuuSROeencoJ6gjgSg). ### Second-order relations From 5d73531899f0d000ea5b4d66badc95ca217a0587 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 6 Sep 2022 15:41:28 -0400 Subject: [PATCH 18/29] Receipts must move forward. --- proposals/3771-read-receipts-for-threads.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 94891d13c54..5c94f07fc66 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -50,12 +50,12 @@ user then reads the thread, the client has no way to mark `E` as read. ### Threaded receipts This MSC proposes allowing the same receipt type to exist multiple times in a room, -once per thread (and once for the "main" timeline of the room). +once per thread (and once for the "main" timeline of the room). This still does not +allow a caller to move their receipts backwards in a room. The body of request to the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) gains the following fields: - * `thread_id` (`string`): The thread that the receipt belongs to (i.e. the `event_id` contained within the `m.relates_to` of the event represented by `eventId`). From ef736e4341bb025e959c14b113f35e07573788a2 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 6 Sep 2022 15:48:07 -0400 Subject: [PATCH 19/29] More info on unthreaded receipts. --- proposals/3771-read-receipts-for-threads.md | 31 ++++++++++++++++----- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 5c94f07fc66..284737af919 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -49,12 +49,18 @@ user then reads the thread, the client has no way to mark `E` as read. ### Threaded receipts -This MSC proposes allowing the same receipt type to exist multiple times in a room, -once per thread (and once for the "main" timeline of the room). This still does not -allow a caller to move their receipts backwards in a room. +This MSC proposes allowing the same receipt type to exist multiple times in a room: + +* Once for the unthreaded timeline. +* Once for the "main" timeline in the room. +* Once per threaded timeline. + +This still does not allow a caller to move their receipts backwards in a room. + +It is not expected that a client would send both "unthreaded" and "threaded" receipts. The body of request to the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) -gains the following fields: +gains the following optional fields: * `thread_id` (`string`): The thread that the receipt belongs to (i.e. the `event_id` contained within the `m.relates_to` of the event represented by @@ -63,6 +69,9 @@ gains the following fields: A special value of `"main"` corresponds to the receipt being for the "main" timeline (i.e. events which are not part of a thread). + If this field is not provided than the receipt applies to the unthreaded + version of the room. + The following conditions are errors and should be rejected with a `400` error with `errcode` of `M_INVALID_PARAM`: @@ -107,6 +116,14 @@ POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root } ``` +For backwards compatibility, not providing the `thread_id` field is supported: + +``` +POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root + +{} +``` + ### Receiving threaded receipts via `/sync`. The client would receive this as part of `/sync` response similar to other receipts: @@ -128,9 +145,9 @@ The client would receive this as part of `/sync` response similar to other recei } ``` -If there is no `thread_id` field than no thread information was supplied with -the receipt, clients may interpret this as only applying to the main timeline or -as applying across all threads, if possible. +If there is no `thread_id` field then the receipt applies to the unthreaded +timeline. Clients may interpret this as applying only to the main timeline or +as applying across the main timeline and all threaded timelines. ### Sending threaded receipts over federation From de7ba497abbb310315a84e42c8691e47b39b3f69 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 8 Sep 2022 12:46:50 -0400 Subject: [PATCH 20/29] Reflow. --- proposals/3771-read-receipts-for-threads.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 284737af919..79fa379fc47 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -1,9 +1,11 @@ # MSC3771: Read receipts for threads Currently, each room can only have a single receipt of each type per user. The -read receipt ([`m.read`](https://spec.matrix.org/v1.3/client-server-api/#receipts) or [`m.read.private`](https://github.com/matrix-org/matrix-spec-proposals/pull/2285)) is used to sync the read status of a -room across clients, to share with other users which events have been read and -is used by the homeserver to calculate the number of unread messages. +read receipt ([`m.read`](https://spec.matrix.org/v1.3/client-server-api/#receipts) +or [`m.read.private`](https://github.com/matrix-org/matrix-spec-proposals/pull/2285)) +is used to sync the read status of a room across clients, to share with other +users which events have been read and is used by the homeserver to calculate the +number of unread messages. Unfortunately a client displaying threads may show a subset of a room's messages at a time, causing a user's read receipt to be misleading. From e1ab8e5d8a9959032e8f4a06a5f42e2030fdbf36 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 8 Sep 2022 13:39:25 -0400 Subject: [PATCH 21/29] Clarify the proposal to explain why both threaded and unthreaded receipts need to exist and what the main timeline is. --- proposals/3771-read-receipts-for-threads.md | 179 +++++++++++++++----- 1 file changed, 132 insertions(+), 47 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 79fa379fc47..ce7af9088d8 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -1,65 +1,141 @@ # MSC3771: Read receipts for threads +## Background + Currently, each room can only have a single receipt of each type per user. The read receipt ([`m.read`](https://spec.matrix.org/v1.3/client-server-api/#receipts) or [`m.read.private`](https://github.com/matrix-org/matrix-spec-proposals/pull/2285)) is used to sync the read status of a room across clients, to share with other -users which events have been read and is used by the homeserver to calculate the +users which events have been read, and is used by the homeserver to calculate the number of unread messages. -Unfortunately a client displaying threads may show a subset of a room's messages -at a time, causing a user's read receipt to be misleading. +Now that [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440) has merged +to add support for threads, there are two ways to display messages: + +* *Unthreaded*: The traditional way of displaying messages before threads existed. + All messages are just shown in the order they’re provided by the server as a + single timeline[^1]. +* *Threaded*: Taking into account the `m.thread` and other relations to separate + a room DAG into multiple sub-timelines: + * One timeline for each root message (I.e. the target of a thread relation) + * One for messages which are not part of a thread: the main timeline. -This might better be described by an example. Given a room with the following -DAG of events (note that the dotted lines are a thread relation, as specified by -[MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440)): +For an example room DAG (solid lines are show topological ordering, dotted lines +show event relations): ```mermaid flowchart RL + I-->H + H-->G + G-->F F-->E E-->D D-->C C-->B B-->A - C-.->A - E-.->A + C-.->|m.thread|A + D-.->|m.thread|B + E-.->|m.thread|A + F-.->|m.thread|B + G-.->|m.reaction|C + H-.->|m.edit|E ``` -A client might interpret this as: +This can be separated into three threaded timelines: ```mermaid flowchart RL - subgraph Main timeline - D-->B + subgraph "Main" timeline B-->A + I-->B + end + subgraph Thread A timeline + C-->A + E-->C + G-.->|m.reaction|C + H-.->|m.edit|E + end + subgraph Thread B timeline + D-->B F-->D end - subgraph Thread timeline +``` + +Due to this separation of messages into separate timelines a single read receipt +per room causes missed (or flaky) notification counts and does not give an accurate +representation of what messages have been read by people. + +Note that it is expected that some clients will continue to show only an unthreaded +view of the room, either until they are able to support a threaded view or because +they do not wish to incorporate threads. + +## Proposal + +This MSC proposes allowing a receipt per thread, as well as an unthreaded receipt. +Thus, receipts are split into two categories, which this document calls "unthreaded" +and "threaded". Threaded receipts are identified by the root message of the thread; +additionally there is a special pseudo-thread for the main timeline. + +The most significant difference between threaded and unthreaded receipts is how +they clear notifications: + +* Unthreaded receipts clear notifications just as they do today (i.e. + "notifications prior to and including that event MUST be marked as read"). +* Threaded receipts clear notifications in a similar way, but taking into account + the thread the receipt is part of (i.e. "notifications generated from events + with a thread relation matching the receipt’s thread ID prior to and including + that event which are MUST be marked as read") + +Using the above diagrams with threaded read receipts on `E` and `I`; and an +unthreaded read receipt on `D` would give: + +``` +flowchart RL + subgraph "Main" timeline + B-->A + I-->B + end + subgraph Thread A timeline C-->A E-->C + G-.->|m.reaction|C + H-.->|m.edit|E + end + subgraph Thread B timeline + D-->B + F-->D end - style A fill:cyan,stroke:#333,stroke-width:2px - style F fill:orange,stroke:#333,stroke-width:2px + classDef unthreaded fill:yellow,stroke:#333,stroke-width:2px + classDef threaded fill:crimson,stroke:#333,stroke-width:2px + classDef both fill:orange,stroke:#333,stroke-width:2px + + %% An unthreaded read receipt on D marks A, B, C, D as read. + class A,B,C both; + class D unthreaded; + %% Threaded read receipts on E and I mark C, E and A, B, I as + %% read, respectively. + class E,I threaded; ``` -While viewing the "main" timeline of the room, a client might move the read -receipt from event `A` to event `F` without ever showing events `C` and `E`. The -user then reads the thread, the client has no way to mark `E` as read. +As denoted by the colors: -## Proposal +* The unthreaded read receipt on `D` would mark `A`, `B`, `C`, and `D` as read. +* The threaded read receipt on `E` would mark `C` and `E` as read. +* The threaded read receipt on `I` would mark `A`, `B`, and `I` as read. ### Threaded receipts -This MSC proposes allowing the same receipt type to exist multiple times in a room: +This MSC proposes allowing the same receipt type to exist multiple times in a room +per user: * Once for the unthreaded timeline. -* Once for the "main" timeline in the room. +* Once for the main timeline in the room. * Once per threaded timeline. -This still does not allow a caller to move their receipts backwards in a room. - -It is not expected that a client would send both "unthreaded" and "threaded" receipts. +No other changes to receipts are proposed, i.e. this still does not allow a caller +to move their receipts backwards in a room. The relationship between `m.read` and +`m.read.private` is not changed. The body of request to the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) gains the following optional fields: @@ -68,11 +144,11 @@ gains the following optional fields: `event_id` contained within the `m.relates_to` of the event represented by `eventId`). - A special value of `"main"` corresponds to the receipt being for the "main" + A special value of `"main"` corresponds to the receipt being for the main timeline (i.e. events which are not part of a thread). If this field is not provided than the receipt applies to the unthreaded - version of the room. + version of the room.[^2] The following conditions are errors and should be rejected with a `400` error with `errcode` of `M_INVALID_PARAM`: @@ -80,9 +156,6 @@ with `errcode` of `M_INVALID_PARAM`: * A non-string `thread_id` (or empty) `thread_id` field. * Providing the `thread_id` properties for a receipt of type `m.fully_read`. -This updates the unique tuple for receipts from -`(room ID, user ID, receipt type)` to `(room ID, user ID, receipt type, thread ID)`. - Given a threaded message: ```json @@ -108,7 +181,7 @@ POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply } ``` -And to send a receipt on the "main" timeline (e.g. on the root event): +And to send a receipt on the main timeline (e.g. on the root event): ``` POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root @@ -118,7 +191,7 @@ POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root } ``` -For backwards compatibility, not providing the `thread_id` field is supported: +As it is today, not providing the `thread_id` field sends an unthreaded receipt: ``` POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root @@ -137,7 +210,7 @@ The client would receive this as part of `/sync` response similar to other recei "m.read": { "@rikj:jki.re": { "ts": 1436451550453, - "thread_id": "$thread_root" // or "main" + "thread_id": "$thread_root" // or "main" or absent } } } @@ -153,9 +226,10 @@ as applying across the main timeline and all threaded timelines. ### Sending threaded receipts over federation -Homeservers should provide the `thread_id` (as provided by the client) in the +Homeservers should include a `thread_id` field for threaded receipts in the [Receipt Metadata](https://spec.matrix.org/v1.3/server-server-api/#receipts) when -sending the `m.receipt` EDU over federation. +sending the `m.receipt` EDU over federation. Unthreaded receipts lack this field, +as they do today. ### Notifications @@ -166,22 +240,14 @@ not provide a way to clear threaded notifications. A threaded read receipt (i.e. a `m.read` or `m.read.private` receipt with a `thread_id` property) should clear notifications for the matching thread following the [current rules](https://spec.matrix.org/v1.3/client-server-api/#receiving-notifications), -but only clear notifications with a matching `thread_id`. +but only clear notifications with a matching `thread_id` (as discussed in MSC3773). +See the examples of the read receipts on `E` and `I` [above](#proposal). An unthreaded read receipt (i.e. a `m.read` or `m.read.private` receipt *without* -a `thread_id`) should apply follow the [current rules](https://spec.matrix.org/v1.3/client-server-api/#receiving-notifications) -as today and disregard thread information when clearing notifications. - -Using the example room DAG from the preamble of this MSC, consider the following -scenarios (all of which start with 5 unread messages: 3 on the main timeline and -2 in thread `A`). - -* A threaded read receipt sent for event `C` on thread `A`, results in: 3 unread - on the main timeline and 1 on thread `A`. -* A threaded read receipt sent for event `D` on the main timeline: 1 unread on - the main timeline and 2 on thread `A`. -* An unthreaded read receipt send for event `D`: 1 unread on the main timeline - and 1 on thread `A`. +a `thread_id`) should apply the [current rules](https://spec.matrix.org/v1.3/client-server-api/#receiving-notifications) +and disregard thread information when clearing notifications. To re-iterate, this +means it would clear any earlier notifications across *all* threads. This is +illustrated by the read receipt on event `D` [above](#proposal). ## Potential issues @@ -217,6 +283,16 @@ related to a root thread event (e.g. a reaction to thread event). This should generally be treated as acceptable by the server (i.e. the server should only care about the ordering of events, not about whether the events are in the same thread). +### Federation compatibility + +A homeserver which does not understand threaded receipts will be unable to properly +understand that multiple receipts exist in a room. They will generally be processed +as unthreaded receipts with the latest receipt winning, regardless of thread. + +This could make read receipts of remote users jump between threads, but this should +not be any worse than it is today. Additionally, since it only affects remote +users, it will not impact notifications. + ## Alternatives ### Thread ID location @@ -276,3 +352,12 @@ This MSC depends on the following MSCs, which have not yet been accepted into the spec: * [MSC3773](https://github.com/matrix-org/matrix-spec-proposals/pull/3773): Notifications for threads + +[^1]: Throughout this document "timeline" is used to mean what the user sees in +the user interface of their Matrix client. + +[^2]: Generally it would be surprising if the same client sent both threaded and +unthreaded receipts, but it is allowed. The only known use-case for this is that +a threaded client can use this to clear *all* notifications in a room by sending +an unthreaded read receipt on the latest event in the room (regardless of which +thread it appears in). From 6e71034505701d7d28e8ec28b227c0fb9f67f51d Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 12 Sep 2022 15:16:15 -0400 Subject: [PATCH 22/29] Add information about validating that an event is part of a thread. --- proposals/3771-read-receipts-for-threads.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index ce7af9088d8..cfa92b371df 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -155,6 +155,13 @@ with `errcode` of `M_INVALID_PARAM`: * A non-string `thread_id` (or empty) `thread_id` field. * Providing the `thread_id` properties for a receipt of type `m.fully_read`. +* If the given `event_id` is not related to the `thread_id`. There may be multiple + relations between events ((e.g. a `m.annotation` to `m.thread`), homeservers + should apply a reasonable maximum number of relations to traverse when attempting + to identify if an event is part of a thread. + + It is recommended that at least 3 relations are traversed when attempting to find + a thread, implementations should be careful to not infinitely recurse.[^3] Given a threaded message: @@ -361,3 +368,10 @@ unthreaded receipts, but it is allowed. The only known use-case for this is that a threaded client can use this to clear *all* notifications in a room by sending an unthreaded read receipt on the latest event in the room (regardless of which thread it appears in). + +[^3]: Three relations is relatively arbitrary, but is meant to cover an edit or +reaction to a thread (to an event with no relations, i.e. the root of a thread): +`A<--[m.thread]--B<--[m.annotation]--C`. +With an additional leftover for future improvements. This is considered reasonable +since threads cannot fork, edits cannot modify relation information, and generally +annotations to annotations are ignored by user interfaces. From 2b857b7a2dca8743e8465d7ea52dfdde1929225a Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 12 Sep 2022 15:19:31 -0400 Subject: [PATCH 23/29] Remove section on second-order relations. --- proposals/3771-read-receipts-for-threads.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index cfa92b371df..cf554ecd9f4 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -283,13 +283,6 @@ deliberate to avoid losing message / missing notifications. Solutions to this problem are deemed out of scope of this MSC. A solution that was briefly explored was [ranged read receipts](https://hackmd.io/Gxm8zuuSROeencoJ6gjgSg). -### Second-order relations - -For simplicity, clients may wish to send receipts for events which are not directly -related to a root thread event (e.g. a reaction to thread event). This should -generally be treated as acceptable by the server (i.e. the server should only care -about the ordering of events, not about whether the events are in the same thread). - ### Federation compatibility A homeserver which does not understand threaded receipts will be unable to properly From d42354250733785cc1cf4b0dd8615085b2d0952b Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 13 Sep 2022 13:50:57 -0400 Subject: [PATCH 24/29] Use proper syntax highlighting. Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3771-read-receipts-for-threads.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index cf554ecd9f4..50cd62c87bf 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -89,7 +89,7 @@ they clear notifications: Using the above diagrams with threaded read receipts on `E` and `I`; and an unthreaded read receipt on `D` would give: -``` +```mermaid flowchart RL subgraph "Main" timeline B-->A From ecbd6c4228784560de245834658635a0f7839386 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 20 Sep 2022 08:22:53 -0400 Subject: [PATCH 25/29] Clarify unthreaded vs. main timeline receipts. --- proposals/3771-read-receipts-for-threads.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 50cd62c87bf..012de159d1a 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -74,7 +74,9 @@ they do not wish to incorporate threads. This MSC proposes allowing a receipt per thread, as well as an unthreaded receipt. Thus, receipts are split into two categories, which this document calls "unthreaded" and "threaded". Threaded receipts are identified by the root message of the thread; -additionally there is a special pseudo-thread for the main timeline. +additionally there is a special pseudo-thread for the main timeline. This allows marking +the main timeline (a pseudo-thread) as read, without marking any actual threads (split + off from the main timeline) as read. The most significant difference between threaded and unthreaded receipts is how they clear notifications: From 1a5b5aed88487d95513f528e159c9179a445ea85 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 20 Sep 2022 10:43:05 -0400 Subject: [PATCH 26/29] Fix typos. Co-authored-by: Hubert Chathi --- proposals/3771-read-receipts-for-threads.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 012de159d1a..444895d47ac 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -76,7 +76,7 @@ Thus, receipts are split into two categories, which this document calls "unthrea and "threaded". Threaded receipts are identified by the root message of the thread; additionally there is a special pseudo-thread for the main timeline. This allows marking the main timeline (a pseudo-thread) as read, without marking any actual threads (split - off from the main timeline) as read. +off from the main timeline) as read. The most significant difference between threaded and unthreaded receipts is how they clear notifications: @@ -149,7 +149,7 @@ gains the following optional fields: A special value of `"main"` corresponds to the receipt being for the main timeline (i.e. events which are not part of a thread). - If this field is not provided than the receipt applies to the unthreaded + If this field is not provided then the receipt applies to the unthreaded version of the room.[^2] The following conditions are errors and should be rejected with a `400` error @@ -320,7 +320,7 @@ read receipts, the opposite is not possible (to the author's knowledge). In shor it seems the [compatibility issues discussed above](#compatibility-with-unthreaded-clients) would not be solved by adding more receipt types. -This also gets more complcated with the addition of the `m.read.private` receipt -- +This also gets more complicated with the addition of the `m.read.private` receipt -- would there additionally be an `m.read.private.thread`? How do you map between all of these? From 7471923092fec374b92a947f10dd2cd8bfb81523 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 20 Sep 2022 10:44:55 -0400 Subject: [PATCH 27/29] Clarify wording. Co-authored-by: Hubert Chathi --- proposals/3771-read-receipts-for-threads.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 444895d47ac..889fa08ffda 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -139,7 +139,7 @@ No other changes to receipts are proposed, i.e. this still does not allow a call to move their receipts backwards in a room. The relationship between `m.read` and `m.read.private` is not changed. -The body of request to the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) +The request body to the [`/receipt` endpoint](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid) gains the following optional fields: * `thread_id` (`string`): The thread that the receipt belongs to (i.e. the From b171a3133dad638825f9438b5f5fb6014e501e27 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 20 Sep 2022 11:02:55 -0400 Subject: [PATCH 28/29] Clarify example. Co-authored-by: Hubert Chathi --- proposals/3771-read-receipts-for-threads.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index 889fa08ffda..f675fc5a42c 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -203,7 +203,7 @@ POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root As it is today, not providing the `thread_id` field sends an unthreaded receipt: ``` -POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_root +POST /_matrix/client/r0/rooms/!room:example.org/receipt/m.read/$thread_reply {} ``` From 668c9de1a9519abefbe6db6413216129807be20d Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 20 Sep 2022 11:07:01 -0400 Subject: [PATCH 29/29] Fix alternatives section. --- proposals/3771-read-receipts-for-threads.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/proposals/3771-read-receipts-for-threads.md b/proposals/3771-read-receipts-for-threads.md index f675fc5a42c..df5f6c86256 100644 --- a/proposals/3771-read-receipts-for-threads.md +++ b/proposals/3771-read-receipts-for-threads.md @@ -299,14 +299,15 @@ users, it will not impact notifications. ### Thread ID location -Instead of adding the thread ID as a new path part, it could be added to the body -of the receipt. There may be a small backwards compatibility benefit to this, but -it seems clearer to put it as part of the URL. - -Instead of encoding the thread ID as an integral part of the receipt, all of the -read threads could be added to the body of the single receipt. This could cause -data integrity issues if multiple clients attempt to update the receipt without -first reading it. +Instead of adding the thread ID in the body, it could be provided as part of the +URL path or as a query parameter. Adding it to the URL (as part of the path or a +query parameter) would make it difficult to differentiate the receipt's event ID +field from the thread ID. + +Another idea was to encode information for all threads in the single receipt, e.g. +by adding them to the body of the single read receipt. This could cause data +integrity issues if multiple clients attempt to update the receipt without first +reading it. ### Receipt type