Skip to content

Commit 12fa1cd

Browse files
alvestrandalvestrand
authored
Merge pull request #212 from fippo/what-is-data-anyway
Describe "data" attribute
2 parents 1839134 + 0331aa9 commit 12fa1cd

File tree

1 file changed

+154
-2
lines changed

1 file changed

+154
-2
lines changed

‎index.bs

Lines changed: 154 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,27 @@ spec:webidl; type:dfn; text:resolve
2323
"href":
2424
"https://www.ietf.org/archive/id/draft-ietf-sframe-enc-04.html",
2525
"title": "Secure Frame (SFrame)"
26+
},
27+
"VP9": {
28+
"href":
29+
"https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp9-bitstream-specification-v0.6-20160331-draft.pdf",
30+
"title": "VP9 Bitstream & Decoding Process Specification",
31+
"publisher": "The WebM Project"
32+
},
33+
"ITU-T-REC-H.264": {
34+
"href": "https://www.itu.int/rec/T-REC-H.264",
35+
"title": "H.264 : Advanced video coding for generic audiovisual services",
36+
"publisher": "ITU"
37+
},
38+
"ITU-G.711": {
39+
"href": "https://www.itu.int/rec/T-REC-G.711/",
40+
"title": "G.711 : Pulse code modulation (PCM) of voice frequencies",
41+
"publisher": "ITU"
42+
},
43+
"ITU-G.722": {
44+
"href": "https://www.itu.int/rec/T-REC-G.722/",
45+
"title": "G.722 : 7 kHz audio-coding within 64 kbit/s",
46+
"publisher": "ITU"
2647
}
2748
}
2849
</pre>
@@ -125,6 +146,7 @@ The <dfn abstract-op>writeEncodedData</dfn> algorithm is given a |rtcObject| as
125146

126147
On sender side, as part of [$readEncodedData$], frames produced by |rtcObject|'s encoder MUST be enqueued in |rtcObject|.`[[readable]]` in the encoder's output order.
127148
As [$writeEncodedData$] ensures that the transform cannot reorder frames, the encoder's output order is also the order followed by packetizers to generate RTP packets and assign RTP packet sequence numbers.
149+
The packetizer may expect the transformed data to still conform to the original format, e.g. a series of NAL units separated by Annex B start codes.
128150

129151
On receiver side, as part of [$readEncodedData$], frames produced by |rtcObject|'s packetizer MUST be enqueued in |rtcObject|.`[[readable]]` in the same encoder's output order.
130152
To ensure the order is respected, the depacketizer will typically use RTP packet sequence numbers to reorder RTP packets as needed before enqueuing frames in |rtcObject|.`[[readable]]`.
@@ -426,8 +448,70 @@ interface RTCEncodedVideoFrame {
426448
</dt>
427449
<dd>
428450
<p>
429-
The encoded frame data.
451+
The encoded frame data. The format of the data depends on the video codec that is
452+
used to encode/decode the frame which can be determined by looking at the
453+
{{RTCEncodedVideoFrameMetadata/mimeType}}.
454+
For <a href="https://w3c.github.io/webrtc-svc/">SVC</a>, each spatial layer
455+
is transformed separately.
456+
<p class="note">
457+
Since packetizers may drop certain elements, e.g. AV1 temporal delimiter OBUs,
458+
the input to an receive-side transform may be different from the output of
459+
a send-side transform.
460+
</p>
461+
The following table gives a number of examples:
430462
</p>
463+
<table class="simple">
464+
<thead>
465+
<tr>
466+
<th>mimeType</th><th>Data format</th>
467+
</tr>
468+
</thead>
469+
<tbody>
470+
<tr>
471+
<td>
472+
video/VP8
473+
</td>
474+
<td>
475+
The data starts with the "uncompressed data chunk" defined in
476+
<a href="https://datatracker.ietf.org/doc/html/rfc6386#section-9.1">
477+
section 9.1</a> of [[RFC6386]] and is followed by the rest of the
478+
frame data. The <a href="https://www.rfc-editor.org/rfc/rfc7741#section-4.1">
479+
VP8 payload descriptor</a> is not accessible.
480+
</td>
481+
</tr>
482+
<tr>
483+
<td>
484+
video/VP9
485+
</td>
486+
<td>
487+
The data is a frame as described in Section 6 of [[VP9]].
488+
The <a href="https://datatracker.ietf.org/doc/html/draft-ietf-payload-vp9#section-4.2">
489+
VP9 payload descriptor</a> is not accessible.
490+
</td>
491+
</tr>
492+
<tr>
493+
<td>
494+
video/H264
495+
</td>
496+
<td>
497+
The data is a series of NAL units in Annex B format,
498+
as defined in [[ITU-T-REC-H.264]] Annex B.
499+
</td>
500+
</tr>
501+
<tr>
502+
<td>
503+
video/AV1
504+
</td>
505+
<td>
506+
The data is a series of OBUs compliant to the
507+
<a href="https://aomediacodec.github.io/av1-spec/#low-overhead-bitstream-format">
508+
low-overhead bitstream format</a> as described in Section 5 of [[AV1]].
509+
The <a href="https://aomediacodec.github.io/av1-rtp-spec/#41-rtp-header-usage">
510+
AV1 aggregation header</a> is not accessible.
511+
</td>
512+
</tr>
513+
</tbody>
514+
</table>
431515
</dd>
432516
</dl>
433517

@@ -554,8 +638,76 @@ interface RTCEncodedAudioFrame {
554638
</dt>
555639
<dd>
556640
<p>
557-
The encoded frame data.
641+
The encoded frame data. The format of the data depends on the audio codec that is
642+
used to encode/decode the frame which can be determined by looking at the
643+
{{RTCEncodedAudioFrameMetadata/mimeType}}.
644+
The following table gives a number of examples:
558645
</p>
646+
<table class="simple">
647+
<thead>
648+
<tr>
649+
<th>mimeType</th><th>Data format</th>
650+
</tr>
651+
</thead>
652+
<tbody>
653+
<tr>
654+
<td>
655+
audio/opus
656+
</td>
657+
<td>
658+
The data is Opus packets, as described in
659+
<a href="https://datatracker.ietf.org/doc/html/rfc6716#section-3">
660+
section 3</a> of [[RFC6716]].
661+
</td>
662+
</tr>
663+
<tr>
664+
<td>
665+
audio/PCMU
666+
</td>
667+
<td>
668+
The data is a sequence of bytes of arbitrary length, where each byte is a u-law
669+
encoded PCM sample as defined by Table 2a and 2b in [[ITU-G.711]].
670+
</td>
671+
</tr>
672+
<tr>
673+
<td>
674+
audio/PCMA
675+
</td>
676+
<td>
677+
The data is a sequence of bytes of arbitrary length, where each byte is
678+
an A-law encoded PCM sample as defined by Tables 1a and 1b in [[ITU-G.711]].
679+
</td>
680+
</tr>
681+
<tr>
682+
<td>
683+
audio/G722
684+
</td>
685+
<td>
686+
The data is G.722 audio as described in [[ITU-G.722]].
687+
</td>
688+
</tr>
689+
<tr>
690+
<td>
691+
audio/RED
692+
</td>
693+
<td>
694+
The data is Redundant Audio Data as described in
695+
<a href="https://www.rfc-editor.org/rfc/rfc2198#section-3">
696+
section 3</a> of [[RFC2198]].
697+
</td>
698+
</tr>
699+
<tr>
700+
<td>
701+
audio/CN
702+
</td>
703+
<td>
704+
The data is Comfort Noise as described in
705+
<a href="https://www.rfc-editor.org/rfc/rfc3389#section-3">
706+
section 3</a> of [[RFC3389]].
707+
</td>
708+
</tr>
709+
</tbody>
710+
</table>
559711
</dd>
560712
</dl>
561713

0 commit comments

Comments
 (0)