E2E Encryption: Publishing to N subscribers

[om26er]

For E2E encryption we need to publish events to multiple subscribers, there may be multiple strategies but the one that I am investigating is to publish N times by using PublishOptions(eligible=recipient_session_id). How the session ID is discovered is a separate thing.

One thing that is currently missing in autobahn is to have a way to specify which responder_pub to use when an event is being published. Once it supports that, any helper function like publish_bulk(topic, [recipients...]) could be used to achieve the 1:N publishing.

[oberstet]

One thing that is currently missing in autobahn is to have a way to specify which responder_pub to use when an event is being published.

yes, because what you refer to with "E2E Encryption" is "cryptobox" I guess, and that was started from a different design: the originator/responder keys are tied to the URI

autobahn-python/autobahn/wamp/cryptobox.py

Line 57 in 9f425ff

Holds originator and responder keys for an URI.

and the resulting NaCl type https://pynacl.readthedocs.io/en/latest/public/#nacl.public.Box is used to create a Curve 25519 Public-Private Key cipher also tied to the URI

autobahn-python/autobahn/wamp/cryptobox.py

Line 93 in 9f425ff

self.originator_box = Box(self.originator_priv, self.responder_pub)

---

This design "E2Ev1" ties the cipher instances to URI, and also uses a PubPriv key cipher to directly encrypt the app payload.

Both of this has turned out to have issues.

Which is the reason for why we developed "E2Ev2" - aka XBR.

With this scheme, the app payloads are encrypted with XSalsa20Poly1305, and the symmetric encryption keys are ephemeral (either per WAMP action or similar), and the key is called data encryption key, and the whole management (distribution and trust) is considered an independent problem (for which XBR has one possible way ..)

All of this is one reason we first want to finish wamp-proto/wamp-proto#417

And then go to the key management, trust and distribution problem area.

---

I probably should note: I personally consider the E2Ev1 approach to be more or less a dead end. It works, but there are issues resulting (as you noted). Hence, I wouldn't want to touch that stuff .. also, btw, I can't see how to achieve "Publishing to N subscribers" with that approach (when the responder keys vary that is of course)

[om26er]

also, btw, I can't see how to achieve "Publishing to N subscribers" with that approach (when the responder keys vary that is of course)

Just to clarify that: the idea was to publish in a loop with each PUBLISH using the public key of the intended receiver AND using PublishOptions(eligible=recipient_session_id) So that the EVENT is only received by the intended session.

[oberstet]

So that the EVENT is only received by the intended session.

ok, that works (technically), but it destroys the actual PubSub routing .. or at least any router mediated message direction needs to be simulated from the client side purely. In any case, I think it would be opposite to the design approach taken by E2Ev1 .. it doesn't match up. And hence E2Ev2, with its explicit use of enc_key (ID of data encryption key).

this way the problem of key management is decoupled. fetching the data encryption key (which can be rotated or otherwise set by the originator as it likes) then needs to be done (this is one option) via an RPC (either to the originator directly, or indirectly via a state channel intermediary). the authenticity and identity of the responding client can then be checked in the originator .. but this is now getting into key management already.

[oberstet]

I should also note: E2Ev1 can be implemented over E2Ev2, not otherwise round (no, there currently isn't an implementation for that .. but it could be done).

well, "logically simulated", since E2Ev2 has a stream cipher for payload encryption, and a second cipher for the data encryption key transfer ... and a signature scheme to establish trust (which is also missing from E2Ev1 completely, but needs to be "burned into" the app ... via the KeyRings which must match up to the degree needed)

[meejah]

I suspect a promising direction is to examine the state-of-the-art for group messaging (e.g. Signal protocol for groups).

However, this has certain features that can be seen as disadvantages: late-joiners can't "catch up" with the group messages (they will never have the right keys).

Some messaging clients like Keybase take a different approach, using a symmetric key for the group messages -- this allows "catch up" for late joiners, but lacks some of the forward-secrecy features of the the Signal-like protocol.

It stil might be interesting to pursue both? That is, some users might not want forward secrecy, instead preferring history etc features to work coherently for late-subscribing clients.

(I haven't tried to consider what a ratchet-style protocol might actually mean for key-management etc inside crossbar, mind ;) )

[oberstet]

@meejah that's definitely interesting .. will examine this a bit.

for background, the approaches for which we have experimental code including app level examples are:

1. "cryptobox": (in above also called "E2Ev1"): the original implementation based on originator/responder public-private key pairs tied to URIs
2. "xbr": (in above also called "E2Ev2"): using symmetric keys ("app payload / data encryption keys") tied to messages (via enc_key) plus - decoupled from payload enc - the management of those symmetric keys eg by secure exchange via RPC to responder (directly or via a "market maker")

---

The Signal Protocol also supports end-to-end encrypted group chats. The group chat protocol is a combination of a pairwise double ratchet and multicast encryption.

https://www.ieee-security.org/TC/SP2015/papers-archived/6949a232.pdf

we divide secure messaging into three nearly orthogonal problem areas addressed in dedicated sections: the trust establishment problem (Section III), ensuring the distribution of cryptographic long-term keys and proof of association with the owning entity; the conversation security problem (Section IV), ensuring the protection of exchanged messages during conversations; and the transport privacy problem (Section V), hiding the communication metadata.

[meejah]

Okay, so I guess my suggestion would amount to a third option: "ratchet" or something.

The main functional difference would be in a scenario where some clients "join later"; those clients wouldn't be able to decrypt any history or retained messages, only messages sent after they joined. (So, this might e.g. be an anti-feature for some of the market-type use-cases, where if you pay for the "last hour" of data or whatever you do indeed want symmetric keys so that you can "catch up").

As I implied earlier, even in chat-clients there doesn't seem to be agreement. For something like "the company Sl*ck channel" you probably want new employees to read old messages (symmetric keys). For something more subscription-oriented, maybe you don't (e.g. you get livestream data only from the moment you paid/joined onwards, not everything that got streamed already) so then you want ratchet-based keys.

Probably best to file this as "possible future enhancement" until there's a use-case that isn't handled by the "2." above (since symmetric encryption is easier, and somewhat simpler to think about for developers and users of crossbar). :)

[oberstet]

Couldn't you still do a Ratchet based scheme on top of approach 2) like so?

1. publishers subscribe to the WAMP meta API event for "new subscriber" added to subscription
2. when receiving such an event, rotate the data encryption key

This will result in a new data encryption key, and this new key needs to be requested (directly or indirectly) by all (then existing) subscribers once they receive a first event with the new keyid

All previous events are encrypted with different keys, so the new subscriber isn't able to decrypt those (even if it had access to the ciphertexts by retrieving eg encrypted event history)

[oberstet]

Thinking about above ... that should work for a single node mesh out of the box, for a rlinked multi-node mesh, it wouldn't work. This is because a new remote subscriber wouldn't lead to a new (reverse) rlink subscription (if at least one remote subscriber was present before)

We could forward meta API events over rlinks .. mmmh ... maybe they are already? ;) Honestly, I dunno .. could be .. never tested rlink forwarding with meta API stuff ...

[meejah]

That sounds .. plausible.

I'm not a cryptographer ;) but sounds like a similar effect at least (new events under new random key). Although I've read Noise and 1:1 Signal protocols, it's been a while and I've never delved into the full details of the group-chat ratchet protocols so probably missing something here, potentially.

[oberstet]

but sounds like a similar effect at least

yes, same effect as in "provides forward secrecy" (btw I forgot, one needs to subscribe to meta API subscriber add and remove and rotate keys on each)

---

I've also not analyzed "ratchet protocols" in detail, quick search turns up a couple of interesting links (but I need to find time to actually read them;)

https://security.stackexchange.com/questions/126768/which-protocols-exist-for-end-to-end-encrypted-group-chat
https://www.ieee-security.org/TC/SP2015/papers-archived/6949a232.pdf
https://whispersystems.org/blog/advanced-ratcheting/
https://github.com/trevp/double_ratchet/wiki
https://eprint.iacr.org/2017/666.pdf
https://messaginglayersecurity.rocks/mls-architecture/draft-ietf-mls-architecture.html