MSC4075: MatrixRTC invites and notifications

[toger5]

Rendered

To-do:

• How do you decline a call? MSC4310
• Implementations using revised format from 04635eb (#4075)
  • Ruma MatrixRTC: Add updated m.rtc.notification event (depreceate m.call.notify) ruma/ruma#2199
  • js-sdk Allow sending notification events when starting a call matrix-js-sdk#4826
• Implementation using revised formats with relations 7c996bb (#4075)
  • js-sdk Allow sending notification events when starting a call matrix-js-sdk#4826
  • Ruma MatrixRTC: Add updated m.rtc.notification event (depreceate m.call.notify) ruma/ruma#2199
• Implementation using revised formats with lifetime 6362c23 (#4075)
  • js-sdk Allow sending notification events when starting a call matrix-js-sdk#4826
  • Ruma MatrixRTC: Add updated m.rtc.notification event (depreceate m.call.notify) ruma/ruma#2199
• This looks very similar to m.call.invite but with much less info. Have we taken into account learnings from this existing capability?
  • edit by @toger5: looking into this it seems that the lifetime concept makes sense. It allows the sending client to know/control when the receiver stops ringing. Additional precautions need to take place to not allow for infinite rings. This is part of 6362c23 (#MSC4075)

Dependencies:

• MSC4143: MatrixRTC – Real-time communication over Matrix #4143

Implementations:

• Element Web
  • sending rings: Add ringing for matrixRTC matrix-react-sdk#11870
  • ringing: Add ringing for matrixRTC matrix-react-sdk#11870
• Ruma
  • notify event support: Add CallNotify event as described by MSC4075 ruma/ruma#1704
• Element X
  • android sending rings
  • android ringing
  • ios sending rings
  • ios ringing
• Extensible events fallback
  • Element web
  • Element X ios
  • Element X android

[uhoreg]

@toger5 You've added some implementations in the PR description. Do these implement the MSC in full, or are there parts of the MSC that are still lacking implementation?

[toger5]

Yes it's implemented in full. At least on the platforms shown in the description (EW).

There are optional configurations that are still missing. (Allowing to send a notification to a specific subset of users) but the API for this is already part of the SDK. There is no ui for this however and we are not sure we want that in EW.

[robintown]

But then the remote server could choose to put origin_server_ts in the future as well, so I think this ultimately needs to be limited by the receipt time on the client. (Ideally it would be limited by the receipt time on the local server, so there's no risk of a notification that "comes back from the dead" every time you log in a new device… but an MSC fixing age / introducing a new received_ts thing is of course a whole nother project)

[Johennes]

Hm, good point. The 2 minute cap would still apply there, too, though. So I think both malicious users and malicious servers could only cause 2-minute rings – which I think is fine?

[robintown]

The 2 minute cap would still apply there, too, though.

Ah, I guess I can read "The lifetime, as measured from sender_ts and capped to 2 minutes, has not elapsed" that way, yes. My initial reading led me to think this was instructing to use stop_ringing_ts = origin_server_ts + min(lifetime, 2 minutes) rather than ring_duration = min(origin_server_ts + lifetime - now, 2 minutes).

[robintown]

Is this redundant to the "push rules produce an action of notify" condition?

[Johennes]

Yeah, something is wrong here. 🤔

I think we should separate validating invites from invite notifications. The latter only determine whether you get a ping / ring. But you should be able to also act on invites in a muted room when you observe them. So the validity rules here should only determine whether an invite is actionable for you.

For that, I think, we'd need the m.mentions condition because an invited directed at another user should not be actionable to you. But we should drop the notify condition from above because you that only determines whether you were notified.

Does that make sense?

[robintown]

That makes sense!

[robintown]

With legacy VoIP, Element clients had a variety of issues with stale invites coming back from the dead on client startup or in cases of poor push/sync performance. I see a potential for client devs to fall into very similar traps with m.rtc.invite if they implement the above bullet list naively, but I think we could significantly improve the odds of a decent implementation if we essentially fold the "three options" specification at the end (join/decline/timeout) into the bullet list, and then make the "three options" explanation non-normative.

Specifically the problem I see is that clients can have gappy syncs, syncs that contain not only an invite but also a join event followed by a leave event, etc., and it's not clear what it means to "observe" a join or decline event in all these cases. Therefore it might be better to phrase the conditions in terms of "what's currently sticky" and "what's in the ephemeral map", which are more well-defined.

I had written out more of my thinking here, but it's probably easiest if I send a PR to show what I mean?

[Johennes]

I had written out more of my thinking here, but it's probably easiest if I send a PR to show what I mean?

Yes, that would be awesome. I think I haven't yet fully understood the change you propose but it'll probably get clearer then.

[robintown]

A downside of reusing m.mentions for invites is that the distinction between a room: true invite and an invite without m.mentions is pretty subtle, I don't know how you'd explain that in a user interface 😕

[Johennes]

Hm, yeah. I think we'd have the same problem if we introduced a separate invitees property (similar to legacy VoIP's invitee)?

I would naively say that without mentions, a call shouldn't ring whereas with mentions it should. Maybe we could achieve that by removing the ring tweak from the underride rule? 🤔

[robintown]

If we switched to an invitees property, then we could rule out the existence of something like an m.mentions: { room: true } invite though, no? (Or are you imagining invitees would exist in addition to m.mentions?)

In my mind it would be ideal to have just two variants: invites with specific invitees, and invites directed at the general room. We shouldn't need a third variant that's more 'passive' than a room invite, because you can always just start a session without sending any invites. I'm not experienced enough with push rules to know whether that's doable…

[robintown]

How would a client implement a "mentions and calls only" setting for a room?

[Johennes]

I think you'd just need a single room-kind rule that mutes anything that isn't caught by override rules. Then the two default override rules for m.rtc.invite and the two default override rules for m.mentions catch both calls and normal mentions.

[Johennes]

You could also achieve calls but no mentions otherwise by using the same room-kind rule and disabling the two default override rules for m.mentions.

Or, thirdly, no calls but using the room-kind rule and setting the actions on the two override rules for m.rtc.invite to [] (which means stop processing).

Do you think it would help to spell these combinations out in the proposal?

[robintown]

Thanks, sounds plausible, I just wanted to check that it was possible with this push rule design, as I haven't personally studied up on push rules. Unless someone else has the same question, don't think there's a particular need to include this.

[Johennes]

@jmartinesp raised the question of how clients are supposed to decide if and which mentions to include.

Naively, I think in DMs, you would probably always include the other user in the mention because you want to ping/ring them. In rooms, you'd use either a room-mention (if you want to ping/ring the room) or no mention at all (if you want to achieve a non-ringing / banner notification at most).

Allowing no mentions at all also helps still allow calls and invites when users lack the power level for room mentions.

All of this should probably be added to the proposal text.