Tracking issue for timeline API

[jplatte]

Now that the initial PR is merged, here is an incomplete list of things we don't have yet in the timeline API, that we need to implement before calling it complete, in no particular order:

• Messages
  • Add unable-to-decrypt timeline item support #1140
  • Retry decryption of UTD timeline items #1215
  • High-level replied-to event / timeline item model (currently we just tell ppl. the event ID)
  • Raw JSON of edits
  • Raw JSON before decryption? (first has to be added to EncryptionInfo, I think)
  • Allow adding a reaction to a local echo of a message #2127
  • Editing local echoes
  • Local echo for editing
  • Make sure editing a reply works right
  • Extensible message events
• State events
  • feat(sdk): Add support for state events in the timeline #1347
  • Local echo
• General event-item related bits
  • Add FailedToParse timeline items #1224
  • fix(sdk): Respect server ordering for remote echoes #1393
  • feat(sdk): LocalEventTimelineItem has a new send_state field #1419 — An error state for local echo items that failed to send
  • Update timeline when members are synced #1406
  • Expose event sending errors through timeline item #1423
  • Update sender display names on ambiguity change
  • "Raw" timeline item for unsupported events
  • "Raw" timeline item for events that got aggregated into an existing timeline item
• Virtual timeline items
  • feat(sdk): Add read marker logic to the timeline API #1189 (own read marker)
  • Add pagination loading indicator as a virtual timeline item #1324
  • feat(sdk): Add day dividers to the experimental timeline #1251
• Make timeline pagination more flexible and smarter #1327
• Read markers for other people
• Creating a timeline from a specific point in the history
• Timeline size control (unload items far out of view)
• Forwards pagination
• Storing timelines on disk

[poljar]

Please add the things I mentioned in my initial PR review as well, most notably support for m.room.encrypted events.

[jplatte]

Ah yes, that's similar to the raw timeline items but not quite the same thing.

[zecakeh]

• "Raw" timeline item for events that got aggregated into an existing timeline item
• Raw JSON of reactions / edits

Aren't those the same thing?

[jplatte]

No. The former is about emitting an extra timeline item for events that update an existing event and aren't usually shown (Element's "show hidden events"); the latter is about what to return when a user wants the JSON representation of a timeline item that consists of multiple events (currently we only return the first event's JSON).

[jmartinesp]

I'm not editing the issue since I'm not sure if this is the best place to ask for this (sorry if it's not!).

Could we add an item for toggling user reactions in an event? Sending a reaction is simple, but we also need to handle:

• Knowing the user has reacted to an event, and with which keys (which sound like Expose more data in EventTimeline's Reactions #1792).
• Allowing the user to redact that reaction. Currently, I believe we have no way from the FFI side to know the id of a reaction event so we can redact it. Also, it would be a lot easier for us to just have a toggleReaction exposed in the FFI side so the SDK is the one who checks if we need to send or redact the event.

[jplatte]

Soo I'm leaving Element and there's still a lot to do here. Here's my attempt to write down my thoughts on the remaining tasks w.r.t the timeline API:

• Raw JSON of edits: Is already stored for the latest edit. Unclear whether we should keep around full edit history to be able to render it on demand, or make edit history a feature that requires being online.

• Allow adding a reaction to a local echo: I'm not aware of any clients that support this, would have to set up reactions to be sent once event ID is known.

• Editing local echoes: Supported by Element Web, but it "just" sequences the original message and edit (unclear how realistic it is to cancel the original message and send the edit as original instead, would require knowing that cancellation succeeded) and creates broken events. Very similar to the previous point.

• Local echo for editing: We already do this, but in a broken way AFAIK. There is no indication that the edit is pending (but I'm unclear on whether any other clients have that) and more importantly, when sending the edit fails the item is not reset to its previous state.

• "Raw" timeline items: Those are these things in Element Web:

  

  They show up if you open /devtools and enable the setting to show "hidden" events. Element Web allow toggling this without reloading the timeline, but with the architecture we have going with timelines in EX that's very hard. To my own use of that feature, it would make a lot more sense UX-wise to have a toggle that loads a new timeline with debug mode enabled as a separate temporary view (not persisted on disk) anyways. Are there people who really want this setting enabled permanently?

• Creating a timeline from a specific point in the history: This is also called "permalink support" and usually starts with the client making a /context request.

  An MVP of this would probably create a separate "detached" timeline object that does not listen to live events and just paginates (backwards / forwards) as the user scrolls. I expect clients would also render a "return to end of the timeline" button that would switch to the "regular" timeline, so concerns around what happens when the user paginates all the way to the bottom wouldn't be super pressing.

  Then there is merging of timelines. I think Element Web implements this to some extent, the mobile apps probably don't. Just detecting that two disjoint timelines (one populated by /sync events, one populated by /messages-paginated events) have overlap and merging them based on that can result in gaps because those two ways of obtaining events use different ordering, at least in Synapse (there's a discussion on changing that here). But then, pagination can by itself already have gaps, as discussed in MSC3871 (I just learned about this when writing this summary!). Add to this that the spec defines pagination order as (reverse-)"chronological" and a bit further down states that the definition for that is "dependent on the server implementation", and you can probably see why I don't have a plan for non-live timeline beyond the MVP described above.

• Timeline size control (unload items far out of view): The hardest part of this is already done in that we store every pagination token we get and associate it with an event ID. I don't remember exactly whether we store the event ID of the most fitting "visible" event or that of the first / last in the chunk, but in any case that's the only thing to be mindful of.

  I guess there's also a decision to be made on what the public API will look like, in particular whether we automatically discard old events, how that combines with users back-paginating a lot – maybe we need to instantiate a second detached timeline from the upmost N items of the initial one and then discard items at the end as the user back-paginates or discard items at the start as the user forward-paginates.

• Storing timelines on disk: Has some overlap with the point right above: If we have an on-disk store, unloading items from a live (non-detached) timeline should mean discarding them from memory, but keeping them on disk. There's probably little point in ever storing non-live timelines on disk. This would mean that any gappy sync would have to clear not just the in-memory timeline but also the on-disk timeline though (unless timeline merging is attempted).

  To clarify if this is not already clear, the idea here is to serialize the timeline model we currently have in memory to disk pretty much as-is, i.e. we'd store timeline items plus metadata, not (all) events in their raw JSON format. This reduces the amount of work for restoring things from disk and has some other advantages that I can unfortunately not remember right now. Note that even if we were to store events, we would need additional metadata like pagination tokens in the store.

[bnjbvr]

An extra note: fetching the entire timeline, on disk, at least once, would be necessary for indexing in the case of full-text search. This would also help with perfectly computing read receipts, as the last known read receipt may live on a very old event, and we'd need all the events between the old event and the latest ones to properly compute the number of unread messages as well as highlights/notifications.

It might also imply that we maintain all timelines alive, and keep only a handful of events in memory for those not active (not in the foreground), so we can update them accordingly.