Formally spec an API for interacting with integration managers

[turt2live]

Background

Integration managers are applications that help clients bring integrations (bridges, bots, etc) to users. This may be for things like setting up bridging to an IRC channel, adding Giphy to your room, or sharing a document with a widget. Integration managers were popularized by Riot, however other clients are starting to bring in support for the concept as well.

There are 2 integration managers at the time of writing this, each with different goals and APIs. Scalar (aka Modular) is the proprietary manager that ships with Riot and is developed specifically for Riot. Dimension (a project of mine) is open source and caters towards self-hosted integrations. As mentioned, both of these managers have different APIs once getting past the (very) basic client interaction. A formal spec should be documented so that future managers, and the existing ones, can adhere to a common standard for clients to use.

Components to consider for the spec

Although clients are encouraged to just nest an iframe (or similar) in their application, it can be difficult or undesirable to do so. A client with an objective for rich integration support may not want to use an iframe and instead opt to interact with the manager directly, providing their own UI/UX. Fractal, for example, is currently doing this with stickers: they have their own sticker picker which reaches out to the API directly instead of using an iframe.

The components available today are:

• The widget postMessage API - Document widget API / specs matrix-spec-proposals#1089 https://github.com/matrix-org/matrix-doc/issues/1236
• The integration manager postMessage API (unspecced, informal documentation here: https://github.com/turt2live/matrix-dimension/blob/master/docs/reference/scalar_client_api.md)
• The API used by clients (Riot) to get information (namely /register, /account, and /widgets/title_lookup)

Some additional things that should be documented in my opinion are:

• The API the manager uses to power its UI. Scalar's is informally documented here: https://github.com/turt2live/matrix-dimension/blob/master/docs/reference/scalar_server_api.md while Dimension's isn't formally documented (however, it does differ). The motivation for having a common spec for this is for those clients which do not want to (or can't) embed an iframe and instead want their own UI.
• Inter-manager APIs. These currently don't exist, and it'd be nice if there was a way for managers to 'share' integrations. For example, using Giphy as supplied by Scalar and using a self-hosted IRC bridge. Full disclosure: this is on the roadmap for Dimension.

[turt2live]

ftr this is on my personal todo list, although I wouldn't object if someone beat me to it (hence documenting it publicly).

[turt2live]

At some point while discussing matrix-org/matrix-react-sdk#2062 the idea of a generic "make me a widget" API came up. This API should be well-standardized and be used by clients like Riot to create widgets.

The API would take in a widget type, and possibly a room ID with user information, and produce a JSON object that the client can send as a state event. The use case for this is so Riot, and other clients, can create Jitsi widgets, however the API can easily be expanded upon later.

This kind of API is important because matrix-org/matrix-spec-proposals#1089 only specifies what the data attributes for a widget are, not the query string. Although we could spec the query string, that would break existing widgets in the wild. Instead, the create widget API can produce a reliable URL for the client to use.

Edit: conversation context: https://matrix.to/#/!DdJkzRliezrwpNebLk:matrix.org/$153210051726DcSwY:t2l.io

[ara4n]

ftr, we absolutely need this - the only reason this ticket hasn't been progressed by the core team (including @turt2live :) is lack of tuits.

[turt2live]

Things to definitely include are:

• Document widget API / specs matrix-spec-proposals#1089
• https://github.com/matrix-org/matrix-doc/issues/1236
• https://github.com/matrix-org/matrix-doc/issues/1354
• Redirect starter links to the desired integration manager matrix-react-sdk#1342 (or a variation thereof)
• Scalar client API to request an OpenID token (read: improve auth dance for widgets) element-hq/element-web#7153

Things to definitely reference are:

• Custom emoji and sticker packs in matrix matrix-spec-proposals#1256