how to suppress previews for links in HTML `content`/`summary`?

[snarfed]

Originally filed as mastodon/mastodon#35717.

When user-facing ActivityStreams services render objects with HTML content, they often generate a link preview for links (<a> tags) in it. That's great! ...except for when they're not wanted. Some links are sensitive, or not informationally useful, or won't have good preview metadata. It'd be nice for services that create and federate objects to be able to specify which content links shouldn't get a preview. Maybe with rel=nopreview, which doesn't exist yet, or a similar class?

Evidently rel=tag, class=u-url, class=h-card currently have this effect in some fediverse implementations. They're not ideal, since they carry semantics, and they're not officially supported for this use, but it's a start.

From https://discord.com/channels/231908446830723072/734716731150303242 :

A: One idea was that it should be based on if there's a Link object attached or not, but by the sounds of it, Mastodon will generate a link preview from just the content?
A: Another idea was maybe rel="nopreview" but that's not currently a thing within HTML
B: lol rel=nopreview is one thing I was thinking. great minds, etc
B: could also (re)use rel=nofollow 🤷‍♂️
B: yeah Mastodon generates previews from tags in content alone
A: Oh, interesting, looks like it doesn't even pay attention to the Link attachments
A: as a hack you could mark the links with class="u-url"
C: Twitter used to generate previews based on the last URL in the body of the text (and save those as cards in the backend). A Tweet could only have one type of attachment though, so if an image was also included in the post, it would override the link preview. I mention this because Mastodon had similar behaviours to Twitter at the beginning - I have not look at what the current implementation does.
C: that's not a way to suppress a link preview in a text-only post, regardless.
D: Some accounts do exactly that to avoid the thundering herd problem of thousands of servers all connecting at once to generate the preview.
B: should I file an issue? there are a few existing issues, but afaict all of them are for Mastodon users or admins, not for external services federating into Mastodon
E: rel=tag or class=u-url or class=h-card will currently skip link preview generation but none of these are specifically designed to suppress them, if rel=nopreview is a real thing I'd be happy to add support for it

swicg/activitypub-trust-and-safety#97 is related, as are mastodon/mastodon#20188, mastodon/mastodon#23612, mastodon/mastodon#24015, mastodon/mastodon#30802.

[ThisIsMissEm]

So my take on this after some consideration is that we probably shouldn't be extracting random links from the content property and then using those to generate previews, instead we should probably encourage software to be intentional with the links attached to content.

That is to say, if an author using publishing software intends a link to have a link preview, then that software should add a Link into the attachments array.

This would also encourage software to use as:sensitive on Link to giving authors a way to prevent harmful media from appearing automatically via link previews (for example media depicting violence or gore that may be fetched from opengraph metadata from news websites)

[trwnh]

mastodon has a PR right now mastodon/mastodon#36104 that suggests putting a Link in attachment, with more discussion there. i think what they're going for is this:

{
  "attachment": {
    "type": "Link",
    "href": "https://link.example/destination"
  }
}

because they still don't trust other publishers and want to fetch and generate their own link previews.

if they did start trusting other publishers, then an alternative would be something like this:

{
  "type": "http://joinmastodon.org/ns#LinkPreview",
  "name": "Some Title",
  "summary": "Here is a description of the thing",
  "image": {
    "url": {
      "href": "https://link.example/destination/cover.jpg",
      "width": 1200,
      "height": 630,
      "mediaType": "image/jpg"
    }
  },
  "url": {
    "href": "https://link.example/destination"
  }
}

or a direct AS2 representation could be used:

{
  "id": "https://link.example/destination",
  "type": "Article",
  "name": "Some Title",
  "summary": "Here is a description of the thing",
  "image": {
    "url": {
      "href": "https://link.example/destination/cover.jpg",
      "width": 1200,
      "height": 630,
      "mediaType": "image/jpg"
    }
  }
}

or opengraph could be reused:

{
  "@context": {
    "og": "https://ogp.me/ns#",
    "article": "https://ogp.me/ns/article#",
    "profile": "http://ogp.me/ns/profile#"
  },
  "og:title": "Some Title",
  "og:type": "article",
  "og:image": "https://link.example/destination/cover.jpg",
  "og:url": "https://link.example/destination",
  "article:author": [
    {
      "profile:username": "alansmithee",
      "profile:first_name": "Alan",
      "profile:last_name": "Smithee"
    }
  ]
}

[ThisIsMissEm]

Mirroring from the Mastodon PR:

I'd probably say an attachment type of Link is correct, however, for link previews, I suspect the answer would be adding an additional preview property to Link which could then carry the appropriate information (if one did want to federate link preview information)

[trwnh]

mirroring from the mastodon PR as well:

{
  "attachment": {
    "type": "Link",
    "href": "https://link.example/destination",
    "preview": {
      "type": "Article", // maybe?
      "name": "Some Title",
      "summary": "Some description of the thing",
      "image": {
        "url": {
          "href": "https://link.example/destination/cover.jpg",
          "mediaType": "image/jpg",
          "width": 1200,
          "height": 630
        }
      },
      "attributedTo": {
        "name": "Alan Smithee",
        "id": "https://actor.example/"
      }
    }
  }
}

if the Link.preview is not trusted then you can generate your own.

[trwnh]

per mastodon/mastodon#36104 (comment) mastodon has clarified intention to consume the following:

{
  "attachment": {
    "href": "https://link.example/destination"
  }
}

and has also clarified intention that they will currently fall back to plucking the first link from the content HTML in the absence of such a Link attachment, but intends or hopes to remove the fallback in the future (so that link previews are only generated for explicitly attached links).

[trwnh]

https://socialhub.activitypub.rocks/t/fep-8967-generating-link-previews-for-attached-links/5598

https://w3id.org/fep/8967 now describes Mastodon's intent to start including links in attachment and eventually to stop generating link previews for the first link in content

[evanp]

I definitely prefer using attachment, since it is more explicit and much easier to parse than working with HTML content and checking rel= values.

I might suggest using an Object-derived type, like Page, as the representation, since it's a little richer in what we can include in the metadata.

I think the backwards compatibility is the tricky part. Adding this feature will not stop existing processors from showing link previews. However, moving links to be previewed into attachment may give better control.

There's also the thundering herd problem -- having many federated servers fetch the same link almost simultaneously. Including rich metadata (excerpts, image, authorship, ...) in the attachment would help give an initial preview, which could be confirmed at a later point (90 seconds, 20 minutes, ...) or just accepted from a trusted source.

I think the best next step is a FEP, like https://w3id.org/fep/8967 .

[github-actions]

This issue has been labelled as potentially needing a FEP, and contributors are welcome to submit a FEP on the topic.
Note that issues may be closed without the FEP being created; that does not mean that the FEP is no longer needed.

[ThisIsMissEm]

Whilst you could use a Object-derived type, this would probably imply that the object exists (even though it wouldn't be dereferenceable), so I think Link is probably safer along with using the preview property that could then use a dereferenceable Object-derived type to represent that specific preview object for that link.

[trwnh]

I think the backwards compatibility is the tricky part. Adding this feature will not stop existing processors from showing link previews

This is in effect "their problem", since they are generating link previews on their own without any signal from publishers. In the interim, Mastodon plans to retain this logic as a fallback if no attachment is present, but remove it later. So backwards compatibility should not be an issue. Vaguely less link previews might be!

using an Object-derived type

I do agree that this could be better, and it is presented as an "Alternative approach" in the FEP, but at least two people have expressed explicitly wanting to work with Link and not Object. I think we already have ecosystem support for directly attaching at least Document, Image, Video, Audio. Unknown attachments can be skipped. In the case of Link, the Link.href can be taken while ignoring the Link.preview if untrusted.

The point that @ThisIsMissEm brings up about expecting a JSON-LD representation of directly attached objects is also worth considering, because while not a strictly valid assumption to make, it is a common assumption. This could be handled by an anonymous object with a url instead of an id, but at that point you're really splitting hairs on whether you take attachment[*].href or attachment[*].id or attachment[*].url[*].href (because an Object can have multiple url values).