fix: provide remote servers a way to find out about an event created during the remote join handshake (#71)

### TLDR
Use a "dummy" event to tie together forward extremities, and proactively
send it to all servers in the room. This allows recently joined servers
to become aware of recent events that would otherwise have "slipped
through the cracks" and thus not be retrievable.

NOTE: While this does send the "dummy" event to all servers in the room,
regardless of if they should care or not, at some point a new event will
reference this dummy event and require it's retrieval. Since it was
proactively sent, this will now not be necessary. This assists in
preventing forks in the DAG

### Alternatives
Unlike #51 which 'pushes' the missing event directly, this causes the
event to be 'pulled' by referencing it as a `prev_event` of a dummy
event. Since the 'dummy event' does not get passed into the client, it
is effectively invisible.

Draw-backs of #51 meant it was not always certain if the 'pushed event'
would show up in `/sync` or in `/messages`, but usually was in `/sync`.
This method always has the 'missing event' show up in `/messages`, which
I feel is more technically correct as that event was(albeit just barely)
created before the 'join event' is persisted.

### The Process
The order of events:
1. `make_join` from remote server, response sent
2. Message A sent from local server
3. `send_join` from remote server, response from local server. Message A
is not in this(as it is not state and is not referenced in any events
that are included). Join event is persisted on local server.
4. Local server realizes there are two forward extremities just after
persisting the join event.
A. Creates a `org.matrix.dummy_event` that has `prev_events` containing
both the join and message A.
   B. Sends this dummy event to all servers in the room. 
5. Remote server receives the dummy event via it's `/send` endpoint,
saves it in a queue until the partial state join begins syncing
additional room state

Author: FrenchGithubUser

synapse/federation/federation_server.py

@@ -766,6 +766,10 @@ async def on_send_join_request(
         event, context = await self._on_send_membership_event(
             origin, content, Membership.JOIN, room_id
         )
+        # Collect this now, the internal metadata of event(which should have it) doesn't
+        stream_ordering_of_join = (
+            await self.store.get_current_room_stream_token_for_room_id(room_id)
+        )
 
         prev_state_ids = await context.get_prev_state_ids()
 
@@ -806,6 +810,27 @@ async def on_send_join_request(
             "members_omitted": send_partial_state_response,
         }
 
+        # Check the forward extremities for the room here. If there is more than one, it
+        # is likely that another event was created in the room during the
+        # make_join/send_join handshake. Without being able to determine how long until
+        # the next event will be created that references this 'missing event',
+        # proactively send a dummy extensible event that ties these forward extremities
+        # together. The remote server should search out this missing event on its own.
+        #
+        # By not sending the 'missing event' directly, the stream ordering for it will
+        # be consistent between servers(in that it technically was created before the
+        # join itself).
+
+        forward_extremities = await self.store._get_forward_extremeties_for_room(
+            room_id, stream_ordering_of_join.get_max_stream_pos()
+        )
+
+        if len(forward_extremities) > 1:
+            # I do not feel it is necessary to set this onto the FederationServer class
+            # itself. Its likelihood of being used is extremely low. Make it on-demand
+            _creation_handler = self.hs.get_event_creation_handler()
+            await _creation_handler._send_dummy_events_to_patch_room(room_id)
+
         if servers_in_room is not None:
             resp["servers_in_room"] = list(servers_in_room)
 

synapse/handlers/message.py

@@ -2210,7 +2210,36 @@ async def _send_dummy_events_to_fill_extremities(self) -> None:
                 now = self.clock.time_msec()
                 self._rooms_to_exclude_from_dummy_event_insertion[room_id] = now
 
-    async def _send_dummy_event_for_room(self, room_id: str) -> bool:
+    async def _send_dummy_events_to_patch_room(self, room_id: str) -> None:
+        """
+        Send a dummy event into this room to patch in a missed forward extremity.
+        This should only be triggered during a remote join if there was a forward
+        extremity that occurred during the make_join/send_join handshake.
+        """
+        async with self._worker_lock_handler.acquire_read_write_lock(
+            NEW_EVENT_DURING_PURGE_LOCK_NAME, room_id, write=False
+        ):
+            dummy_event_sent = await self._send_dummy_event_for_room(
+                room_id, proactively_send=True
+            )
+
+        if not dummy_event_sent:
+            # Did not find a valid user in the room, so remove from future attempts
+            # Exclusion is time limited, so the room will be rechecked in the future
+            # dependent on _DUMMY_EVENT_ROOM_EXCLUSION_EXPIRY
+            logger.info(
+                "Failed to send dummy event into room %s. Will exclude it from "
+                "future attempts until cache expires",
+                room_id,
+            )
+            # This mapping is room_id -> time of last attempt(in ms)
+            self._rooms_to_exclude_from_dummy_event_insertion[room_id] = (
+                self.clock.time_msec()
+            )
+
+    async def _send_dummy_event_for_room(
+        self, room_id: str, proactively_send: bool = False
+    ) -> bool:
         """Attempt to send a dummy event for the given room.
 
         Args:
@@ -2242,8 +2271,7 @@ async def _send_dummy_event_for_room(self, room_id: str) -> bool:
                             },
                         )
                         context = await unpersisted_context.persist(event)
-
-                        event.internal_metadata.proactively_send = False
+                        event.internal_metadata.proactively_send = proactively_send
 
                         # Since this is a dummy-event it is OK if it is sent by a
                         # shadow-banned user.