How to support both sframe and JS transform at the same time

[jan-ivar]

Turning on sframe encryption alone is super easy, barely an inconvenience (using per-packet as an example):

transceiver.sender.transform = new SFrameTransform({mode: "per-packet"});

Goodbye JS-transform (RTCRtpScriptTransform) for E2EE! Except what about its other use cases: adding metadata and fan-out?

Model A: sframe and JS transform as orthogonal features

With sframe handled entirely in the browser—near packetization and depacketization—a first-principles approach might arrive at letting websites turn it on independently, leaving everything else the same, including RTCRtpScriptTransform. Pick a switch:

1.  transceiver.sender.transform = new RTCRtpScriptTransform(worker);
    transceiver.sender.stransform = new SFrameTransform;

2.  transceiver.sender.transform = new RTCRtpScriptTransform(worker, {sframe: "per-packet"});

3.  transceiver.sender.transform = new SFrameTransform(worker, {mode: "per-packet"});

4.  transceiver.sframe = "per-packet";

Ditto receiver. This switch would turn on sframe encrypt/decrypt at X (per-frame) or Y (per-packet):

• MST → encoder → JS transform (optional) → (X) packetizer (Y) → network
• play ← decoder ← JS transform (optional) ← (X) depacketizer (Y) ← network

Works the same with both per-frame and per-packet, existing unchanged JS transforms, and without JS transform.

Model B: sframe as a feature inside JS transform

A single line in the spec SFrameTransform includes GenericTransformStream; suggests SFrameTransform doubles as a Transform Stream #262.

This suggests another way of combining the two features might have been intended:

// main.html
transceiver.sender.transform = new RTCRtpScriptTransform(worker, {type: "sframe"});

// worker.js
onrtctransform = async ({transformer: {readable, writable}}) => {
  await readable
      .pipeThrough(new TransformStream(transform))
      .pipeThrough(new SFrameTransform({role: "encrypt"}))
      .pipeTo(writable);
};

Receiver-side SFrameTransform({role: "decrypt"})) would come before transform.

But this only works per-frame, not per-packet. It's incompatible with existing JS transforms, and incompatible with Model A (since the receiver-side JS transform is fed sframe-encrypted frames.

Why does Model B need a switch? I dunno, but this was suggested in our most recent meeting, so I've left it in here.

Model B lets JS read and manipulate the sframe-encrypted frame, at the cost of per-packet. Is there a use case for this? Otherwise I'd go with A.