Skip to content

Commit 6309e55

Browse files
youennfyouennf
committed
WIP: transform model
1 parent 06540f9 commit 6309e55

File tree

2 files changed

+922
-1467
lines changed

2 files changed

+922
-1467
lines changed

‎index.bs

Lines changed: 132 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -115,13 +115,38 @@ partial dictionary RTCConfiguration {
115115
boolean encodedInsertableStreams = false;
116116
};
117117

118+
enum SFrameTransformRole {
119+
"encrypt",
120+
"decrypt"
121+
};
122+
123+
dictionary SFrameTransformOptions {
124+
SFrameTransformRole role = "encrypt";
125+
};
126+
127+
[Exposed=(Window,DedicatedWorker)]
128+
interface SFrameTransform {
129+
constructor(optional SFrameTransformOptions options);
130+
// FIXME: add key handling methods.
131+
};
132+
SFrameTransform includes GenericTransformStream;
133+
134+
[Exposed=(Window)]
135+
interface RTCRtpScriptTransform {
136+
constructor(any worker, optional object options); // worker as DedicatedWorker.
137+
};
138+
139+
typedef (SFrameTransform or RTCRtpScriptTransform) RTCRtpTransform;
140+
118141
// New methods for RTCRtpSender and RTCRtpReceiver
119142
partial interface RTCRtpSender {
120143
RTCInsertableStreams createEncodedStreams();
144+
attribute RTCRtpTransform? transform;
121145
};
122146

123147
partial interface RTCRtpReceiver {
124148
RTCInsertableStreams createEncodedStreams();
149+
attribute RTCRtpTransform? transform;
125150
};
126151
</pre>
127152

@@ -134,31 +159,120 @@ argument, ensure that the codec is disabled and produces no output.
134159

135160
### Stream creation ### {#stream-creation}
136161

137-
Let the {{RTCRtpSender}} or {{RTCRtpReceiver}} have an internal slot,
138-
`[[Streams]]`, initialized to null.
162+
At construction of each {{RTCRtpSender}} or {{RTCRtpReceiver}}, run the following steps:
163+
1. Initialize `[[Streams]]` to null.
164+
2. Initialize `[[transform]]` to null.
165+
3. Initialize `[[readable]]` to the result of <a href="https://streams.spec.whatwg.org/#readablestream-create">creating a</a> {{ReadableStream}}. `[[readable]]` is provided frames using the [=readEncodedData=] algorithm given |this| as parameter.
166+
4. Set `[[readable]]`.`[[owner]]` to |this|.
167+
5. Initialize `[[writable]]` to the result of <a href="https://streams.spec.whatwg.org/#writablestream-create">creating a</a> {{WritableStream}}, its <a href="https://streams.spec.whatwg.org/#writablestream-create-writealgorithm">writeAlgorithm</a> set to [=writeEncodedData=] given |this| as parameter.
168+
6. Set `[[writable]]`.`[[owner]]` to |this|.
169+
7. Initialize `[[pipeToController]]` to null.
170+
8. Initialize `[[lastReceivedFrameTimestamp]]` to zero.
171+
9. If the {{RTCPeerConnection}}'s configuration does not have {{RTCConfiguration/encodedInsertableStreams}} set to "true", queue a task to run the following steps:
172+
1. If `[[pipeToController]]` is not null, abort these steps.
173+
2. Call <a href="https://streams.spec.whatwg.org/#readable-stream-pipe-to">pipeTo</a> with `[[readable]]`, `[[writable]]`, preventClose equal to true, preventAbort equal to true, preventCancel equal to true and `[[pipeToController]]`.signal.
139174

140175
When {{RTCRtpSender/createEncodedStreams}}() is
141176
called, run the following steps:
142177

143-
* If the {{RTCPeerConnection}}'s configuration does not have {{RTCConfiguration/encodedInsertableStreams}} set to "true", throw an {{InvalidStateError}} and abort these steps.
144-
* If the data source does not permit access, throw an {{InvalidAccessError}} and abort these steps.
145-
* If `[[Streams]]` is not null, throw an {{InvalidStateError}}.
146-
* Create an {{RTCInsertableStreams}} object |s|.
147-
* Set |s|.{{RTCInsertableStreams/readable}} to a {{ReadableStream}} representing the encoded data source.
148-
* Set |s|.{{RTCInsertableStreams/writable}} to a {{WritableStream}} representing the encoded data sink.
149-
* Enable the encoded data source.
150-
* Store |s| in the internal slot `[[Streams]]`.
151-
* Return |s|
178+
1. If the {{RTCPeerConnection}}'s configuration does not have {{RTCConfiguration/encodedInsertableStreams}} set to "true", throw an {{InvalidStateError}} and abort these steps.
179+
2. If the data source does not permit access, throw an {{InvalidAccessError}} and abort these steps.
180+
3. If `[[Streams]]` is not null, throw an {{InvalidStateError}}.
181+
4. If `[[pipeToController]]` is not null, throw an {{InvalidStateError}}.
182+
5. Create an {{RTCInsertableStreams}} object |s|.
183+
6. Set |s|.{{RTCInsertableStreams/readable}} to `[[readable]]`.
184+
7. Set |s|.{{RTCInsertableStreams/writable}} to `[[writable]]`.
185+
8. Enable the encoded data source.
186+
9. Store |s| in the internal slot `[[Streams]]`.
187+
10. Return |s|
152188

153189
### Stream processing ### {#stream-processing}
154190

155-
When a frame is produced from the encoded data source, place it on the
156-
`[[Streams]]`.{{RTCInsertableStreams/readable}}.
157-
158-
When a frame appears on the `[[Streams]]`.{{RTCInsertableStreams/writable}}, do the following:
159-
* Check that the frame is a a valid frame that has been created by the encoded data source; if it is not, discard it. A processor cannot create frames, or move frames between streams.
160-
* Check that the frame's {{RTCEncodedVideoFrame/timestamp}} is equal to or larger than any previously received frame. A processor cannot reorder frames, although it may delay them or drop them.
161-
* Process the frame as if it came directly from the encoded data source.
191+
The <dfn>readEncodedData</dfn> algorithm is given a |rtcObject| as parameter. It is defined by running the following steps:
192+
1. Wait for a frame to be produced by |rtcObject|'s encoder if it is a {{RTCRtpSender}} or |rtcObject|'s packetizer if it is a {{RTCRtpReceiver}}.
193+
2. Let |frame| be the newly produced frame.
194+
3. Set |frame|.`[[owner]]` to |rtcObject|.
195+
4. <a href="https://streams.spec.whatwg.org/#readablestream-enqueue">Enqueue</a> |frame| in |rtcObject|.`[[readable]]`.
196+
197+
The <dfn>writeEncodedData</dfn> algorithm is given a |rtcObject| as parameter and a |frame| as input. It is defined by running the following steps:
198+
1. If |frame|.`[[owner]]` is not equal to |rtcObject|, abort these steps. A processor cannot create frames, or move frames between streams.
199+
2. If the |frame|'s {{RTCEncodedVideoFrame/timestamp}} is equal to or larger than |rtcObject|.`[[lastReceivedFrameTimestamp]]`, abort these steps. A processor cannot reorder frames, although it may delay them or drop them.
200+
3. Set |rtcObject|.`[[lastReceivedFrameTimestamp]]` to the |frame|'s {{RTCEncodedVideoFrame/timestamp}}.
201+
4. Process the frame as if it came directly from the encoded data source:
202+
* If |rtcObject| is a {{RTCRtpSender}}, enqueue it |rtcObject|'s packetizer.
203+
* If |rtcObject| is a {{RTCRtpReceiver}}, enqueue it to |rtcObject|'s decoder.
204+
205+
## Extension attribute ## {#attribute}
206+
207+
A RTCRtpTransform has two private slots called `[[readable]]` and `[[writable]]`.
208+
209+
On getting the transform attribute, the user agent MUST run the following steps:
210+
1. Let |rtcObject| be the {{RTCRtpReceiver}} or {{RTCRtpSender}} on which the getter is invoked.
211+
2. Return |rtcObject|.`[[transform]]`.
212+
213+
On setting the transform attribute, the user agent MUST run the following steps:
214+
1. Let |rtcObject| be the {{RTCRtpReceiver}} or {{RTCRtpSender}} on which the setter is invoked.
215+
2. Let |transform| be the argument to the setter.
216+
3. Let |checkedTransform| set to |transform| if it is not null or to an <a href="https://streams.spec.whatwg.org/#identity-transform-stream">identity transform</a> otherwise.
217+
3. Let |reader| be the result of <a href="https://streams.spec.whatwg.org/#readablestream-get-a-reader">getting a reader</a> for |checkedTransform|.`[[readable]]`.
218+
4. Let |writer| be the result of <a href="https://streams.spec.whatwg.org/#writablestream-get-a-writer">getting a writer</a> for |checkedTransform|.`[[writable]]`.
219+
5. Initialize |newPipeToController| to a new AbortController.
220+
6. If |rtcObject|.`[[pipeToController]]` is not null, run the following steps:
221+
1. <a href="https://dom.spec.whatwg.org/#abortsignal-add">Add</a> the [=chain transform algorithm=] to |rtcObject|.`[[pipeToController]]`:
222+
2. Call <a href="https://dom.spec.whatwg.org/#abortsignal-signal-abort">abort</a> on |rtcObject|.`[[pipeToController]]`.
223+
7. Else, run the [=chain transform algorithm=] steps.
224+
8. Set |rtcObject|.`[[pipeToController]]` to |newPipeToController|.
225+
9. Set |rtcObject|.`[[transform]]` to |transform|.
226+
227+
The <dfn>chain transform algorithm</dfn> steps are defined as:
228+
1. If |newPipeToController| <a href="https://dom.spec.whatwg.org/#abortsignal-aborted-flag">aborted flag</a> is true, abort these steps.
229+
2. Call <a href="https://streams.spec.whatwg.org/#readablestreamdefaultreader-release">release</a> with |reader|.
230+
3. Call <a href="https://streams.spec.whatwg.org/#writablestreamdefaultwriter-release">release</a> with |writer|.
231+
4. Assert that |newPipeToController| is the same object as |rtcObject|.`[[pipeToController]]`.
232+
5. Call <a href="https://streams.spec.whatwg.org/#readable-stream-pipe-to">pipeTo</a> with |rtcObject|.`[[readable]]`, |checkedTransform|.`[[writable]]`, preventClose equal to false, preventAbort equal to false, preventCancel equal to true and |newPipeToController|.signal.
233+
6. Call <a href="https://streams.spec.whatwg.org/#readable-stream-pipe-to">pipeTo</a> with |checkedTransform|.`[[readable]]`, |rtcObject|.`[[writable]]`, preventClose equal to true, preventAbort equal to true, preventCancel equal to false and |newPipeToController|.signal.
234+
235+
This algorithm is defined so that transforms can be updated dynamically.
236+
There is no guarantee on which frame will happen the switch from the previous transform to the new transform.
237+
238+
If a web application sets the transform synchronously at creation of the {{RTCRtpSender}} (for instance when calling addTrack), the transform will receive the first frame generated by the {{RTCRtpSender}}'s encoder.
239+
Similarly, if a web application sets the transform synchronously at creation of the {{RTCRtpReceiver}} (for instance when calling addTrack, or at track event handler), the transform will receive the first full frame generated by the {{RTCRtpReceiver}}'s packetizer.
240+
241+
# SFrameTransform # {#sframe}
242+
243+
When the SFrameTransform.constructor() is invoked, run the following steps:
244+
1. Let |transformAlgorithm| be an algorithm which takes a |frame| as input and runs the <a href="#sframe-transform-algorithm">SFrame transform algorithm</a> with |this| and |frame|.
245+
2. Set |this|.<a href="https://streams.spec.whatwg.org/#generictransformstream-transform">`[[transform]]`</a> to the result of <a href="https://streams.spec.whatwg.org/#transformstream-create">creating a TransformStream</a>, its <a href="https://streams.spec.whatwg.org/#transformstream-create-transformalgorithm">transformAlgorithm</a> set to |transformAlgorithm|.
246+
3. Let |options| be the method's first argument.
247+
4. Set |this|.`[[role]]` to |options|.role.
248+
5. Set |this|.`[[readable]]` to |this|.<a href="https://streams.spec.whatwg.org/#generictransformstream-transform">`[[transform]]`</a>.`[[readable]]`.
249+
6. Set |this|.`[[writable]]` to |this|.<a href="https://streams.spec.whatwg.org/#generictransformstream-transform">`[[transform]]`</a>.`[[writable]]`.
250+
251+
## SFrame transform algorithm ## {#sframe-transform-algorithm}
252+
253+
The SFrame transform algorithm, given |sframe| as a SFrameTransform object and |frame|, runs these steps:
254+
1. Let |role| be |sframe|.`[[role]]`.
255+
2. If |frame|.`[[rtcObject]]` is a {{RTCRtpSender}}, set |role| to 'encrypt'.
256+
3. If |frame|.`[[rtcObject]]` is a {{RTCRtpReceiver}}, set |role| to 'decrypt'.
257+
4. Let |data| be undefined.
258+
5. If |frame| is an ArrayBuffer, set |data| to |frame|. // FIXME Support BufferSource
259+
6. If |frame| is an {{RTCEncodedAudioFrame}}, set |data| to |frame|.{{RTCEncodedAudioFrame/data}}
260+
7. If |frame| is an {{RTCEncodedVideoFrame}}, set |data| to |frame|.{{RTCEncodedVideoFrame/data}}
261+
8. If |data| is undefined, abort these steps.
262+
9. Let |buffer| be the result of running the SFrame algorithm with |data| and |role| as parameters. This algorithm is defined by the <a href="https://datatracker.ietf.org/doc/draft-omara-sframe/">SFrame specification</a>.
263+
10. If |frame| is an ArrayBuffer, set |frame| to |buffer|.
264+
11. If |frame| is an {{RTCEncodedAudioFrame}}, set |frame|.{{RTCEncodedAudioFrame/data}} to |buffer|.
265+
12. If |frame| is an {{RTCEncodedVideoFrame}}, set |frame|.{{RTCEncodedVideoFrame/data}} to |buffer|.
266+
13. <a href="https://streams.spec.whatwg.org/#transformstream-enqueue">Enqueue</a> |frame| in |sframe|.`[[transform]]`.
267+
268+
# RTCRtpScriptTransform # {#scriptTransform}
269+
270+
When the RTCRtpScriptTransform.constructor() is invoked, run the following steps:
271+
1. Create two <a href="https://streams.spec.whatwg.org/#identity-transform-stream">identity transforms</a> named |t1| and |t2|.
272+
2. Set |this|.`[[writable]]` to |t1|.`[[writable]]`.
273+
3. Set |this|.`[[readable]]` to |t2|.`[[readable]]`.
274+
4. FIXME: transfer |t1|.`[[readable]]` and |t2|.`[[writable]]` to the dedicated worker.
275+
5. FIXME: Create counterpart of |this| in dedicated worker, for instance expose transfered |t1|.`[[readable]]` and |t2|.`[[writable]]`.
162276

163277
# Privacy and security considerations # {#privacy}
164278

0 commit comments

Comments
 (0)