oembed: ignore oembed results from mastodon

This adds a check in the oembed preview for cases where the oembed
is going to contain less information than the OpenGraph; in these
cases, the oembed is ignored and the OpenGraph data is retained.
At the moment, the only case in here is Mastodon posts, but it
could be expanded in the future.

Mastodon's oembed is JavaScript based and only contains stub HTML
that's intended to be filled in by client-side JavaScript that
synapse doesn't execute. It doesn't contain the actual post text.
The og:description, on the other hand, does contain the post text.
When synapse unilaterally prefers the oembed over OpenGraph, it
ends up accidentally filling in a generic and non-useful preview
in place of the actual post content.

Author: mistydemeo

changelog.d/19286.feature

@@ -0,0 +1 @@
+Improved previews for Mastodon posts.

synapse/media/oembed.py

@@ -232,6 +232,18 @@ def _fetch_urls(tree: "etree._Element", tag_name: str) -> list[str]:
     return results
 
 
+def _should_reject_description(description: str) -> bool:
+    """
+    Determines whether or not this description should be preferred over the
+    og:description. Certain web apps with client-side JavaScript will serve
+    stub documents via oembed that are intended to be filled in by client-
+    side javascript that synapse won't execute; in these cases, the
+    og:description will actually be more complete.
+    """
+
+    return description.startswith("Post by @") and "View on Mastodon" in description
+
+
 def calc_description_and_urls(open_graph_response: JsonDict, html_body: str) -> None:
     """
     Calculate description for an HTML document.
@@ -273,5 +285,5 @@ def calc_description_and_urls(open_graph_response: JsonDict, html_body: str) ->
         open_graph_response["og:video"] = video_urls[0]
 
     description = parse_html_description(tree)
-    if description:
+    if description and not _should_reject_description(description):
         open_graph_response["og:description"] = description