Skip to content

Fetch events from multiple homeservers #5

@MadLittleMods

Description

@MadLittleMods

Problem

When using the MSC2716 /batch_send endpoint, it creates events all at the same depth and are only correctly sorted between because of the stream_ordering. This works great for the local homeserver as they are all in the correct order but can get sticky when federating to other homeservers because stream_ordering is just a local number and not federated like depth. If other homeservers always backfilled in order from newest -> oldest like scrollbar in a room, everything would be fine. But...

[...] it still doesn't guarantee the same stream_ordering (and more importantly the /messages order) on the other server. For example, if a room has a bunch of history imported and someone visits a permalink to a historical message back in time, their homeserver will skip over the historical messages in between and insert the permalink as the next message in the stream_order and totally throw off the sort.

  • This will be even more the case when we add the MSC3030 jump to date API endpoint so the static archives can navigate and jump to a certain date.
  • We're solving this in the future by switching to online topological ordering and chunking which by its nature will apply retroactively to fix any inconsistencies introduced by people permalinking

-- matrix-org/synapse#11114

This is especially important for the gitter.im homeserver because we will use MSC2716 to backfill the entire message history from Gitter to Matrix as part of the sunset plan. We want the messages to be in the correct order and available correctly in the archive. Pulling directly from the gitter.im homeserver will allow us to bide our time until we get to solving graph linearization the correct way via online topological ordering.

Potential solutions

Instead of pulling from a single config.matrixServerUrl with config.matrixAccessToken, we can have a map from homeserver to access token and if a given room alias matches a homeserver we have the access token for, we can pull from it directly. This saves time from not having to join over federation in those cases but also very important for the MSC2716 problem described above.

{
  "matrixServerSecretMap": {
    "matrix.org": "xxx",
    "gitter.im": "xxx"
  }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-archive-room-viewThe view to look at a room day by day in the archiveA-backendRelated to the backend Node.js server piecesT-EnhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions