Skip to content

Commit 44db13c

Browse files
ossrs-aiwinlinvip
ossrs-ai
authored and
winlinvip
committed
AI: Design a WebRTC private protocol for rust-turbo.
1 parent e59b303 commit 44db13c

File tree

1 file changed

+253
-0
lines changed

1 file changed

+253
-0
lines changed

‎trunk/doc/WebRtcPrivateTcp.md‎

Lines changed: 253 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
# WebRTC Private TCP Protocol
2+
3+
## Problem Statement
4+
5+
SRS ecosystem includes various components that need efficient, low-latency communication with SRS servers:
6+
7+
- **rust-turbo**: High-performance Rust-based media processing component that handles WebRTC with DTLS/SRTP for external clients
8+
- **Internal Services**: Custom media processing or analysis services within your infrastructure
9+
10+
While SRS supports standard protocols (RTMP, HTTP-FLV, WebRTC over HTTP API), these have limitations for **internal system communication**:
11+
12+
1. **RTMP**: Legacy protocol, limited codec support, higher latency
13+
2. **HTTP-FLV**: Pull-only, not suitable for bidirectional communication
14+
3. **WebRTC over HTTP API**: Requires separate HTTP signaling + media channels, designed for browsers
15+
16+
## Solution: WebRTC Private TCP Protocol
17+
18+
A **unified TCP channel** (default port 9999) that combines:
19+
20+
- **JSON signaling**: Simple WHIP/WHEP-like protocol for session establishment
21+
- **WebRTC media**: Standard STUN/DTLS/RTP/RTCP transport (plaintext, not encrypted)
22+
- **Single connection**: Both signaling and media over one TCP socket
23+
24+
This protocol is designed for:
25+
26+
1. **Internal Components**: rust-turbo connecting to SRS for high-performance media processing
27+
2. **Custom Services**: Third-party services that need efficient WebRTC access to SRS
28+
29+
**Note**: This is an **internal protocol** for SRS ecosystem components.
30+
31+
## Transport Layer
32+
33+
The protocol uses **RFC 4571** framing over TCP:
34+
35+
```
36+
┌─────────────────┬──────────────────────────────┐
37+
│ Length (2B) │ Packet Content (variable) │
38+
│ Big-endian │ JSON/STUN/DTLS/RTP/RTCP │
39+
└─────────────────┴──────────────────────────────┘
40+
```
41+
42+
- **Length**: 16-bit unsigned integer, network byte order (big-endian)
43+
- **Content**: Variable-length packet data
44+
45+
## Packet Type Multiplexing
46+
47+
Different packet types are distinguished by **first byte inspection**:
48+
49+
| First Byte Range | Packet Type | Description |
50+
|-----------------|-------------|-------------|
51+
| `0x00 - 0x01` | STUN | ICE connectivity checks |
52+
| `0x14 - 0x3F` | DTLS | Key exchange (20-63) |
53+
| `0x80 - 0xBF` | RTP/RTCP | Media packets (V=2) |
54+
| `0x7B` | JSON | Signaling (`{`) |
55+
56+
**Key Insight**: JSON packets starting with `{` (0x7B) do **not conflict** with any WebRTC packet types, enabling safe multiplexing. All JSON messages must be objects (not arrays).
57+
58+
## Connection Flow
59+
60+
**Example: rust-turbo Publishing to SRS**
61+
62+
```
63+
rust-turbo SRS Server
64+
│ │
65+
├──────── TCP Connect to :9999 ──────────>│
66+
│ │
67+
├──── JSON Signaling (Publish) ──────────>│
68+
│<───── JSON Response (SDP Answer) ───────┤
69+
│ │
70+
├────────── STUN Binding Request ────────>│
71+
│<───────── STUN Binding Response ────────┤
72+
│ │
73+
├────────── DTLS ClientHello ────────────>│
74+
│<───────── DTLS ServerHello ─────────────┤
75+
│ (DTLS Handshake - Key Exchange) │
76+
│ │
77+
├────────── RTP packets (plaintext) ─────>│
78+
├────────── RTCP packets (plaintext) ────>│
79+
│<───────── RTCP feedback (plaintext) ────┤
80+
│ │
81+
```
82+
83+
**Note**: After DTLS handshake, RTP/RTCP packets are sent in **plaintext** (not encrypted with SRTP/SRTCP).
84+
85+
**Why DTLS handshake but plaintext RTP/RTCP?**
86+
87+
This protocol is designed to work with **rust-turbo**, which handles WebRTC connections with external clients using full DTLS/SRTP encryption. The DTLS handshake between SRS and rust-turbo allows rust-turbo to obtain the necessary keys for SRTP encryption/decryption with external clients. However, the communication between SRS and rust-turbo itself uses **plaintext RTP/RTCP** because:
88+
89+
1. **Internal Communication**: SRS and rust-turbo are internal system components, typically on the same network or trusted infrastructure
90+
2. **Performance**: Plaintext transport reduces CPU overhead for encryption/decryption between internal components
91+
3. **Simplicity**: Easier to debug and monitor internal traffic without encryption overhead
92+
93+
For other use scenarios, you can implement your own service using this protocol. Since it's designed for **internal system communication**, plaintext is acceptable and provides better performance.
94+
95+
## JSON Message Format
96+
97+
All JSON messages are framed with RFC 4571 length prefix:
98+
99+
```
100+
┌──────┬────────────────────────┐
101+
│ 0x00 │ 0x4A (74 bytes) │ ← Length
102+
├──────┴────────────────────────┤
103+
│ { "type": "publish", ... } │ ← JSON content
104+
└───────────────────────────────┘
105+
```
106+
107+
## JSON Publish Request
108+
109+
**Direction**: Publisher → Receiver
110+
111+
```json
112+
{
113+
"type": "publish",
114+
"stream": "/live/livestream",
115+
"sdp": "v=0\r\no=- 1234567890 2 IN IP4 192.168.1.100\r\ns=SRSPublishSession\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=ice-lite\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:4ZcD\r\na=ice-pwd:bILRCdg4YchH1OqRIvvscZAP\r\na=fingerprint:sha-256 ...\r\na=setup:actpass\r\na=mid:0\r\na=sendonly\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\nm=video 9 UDP/TLS/RTP/SAVPF 106\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:4ZcD\r\na=ice-pwd:bILRCdg4YchH1OqRIvvscZAP\r\na=fingerprint:sha-256 ...\r\na=setup:actpass\r\na=mid:1\r\na=sendonly\r\na=rtcp-mux\r\na=rtpmap:106 H264/90000\r\na=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\n",
116+
"server_id": "srs-server-a"
117+
}
118+
```
119+
120+
**Fields**:
121+
122+
- `type`: Must be `"publish"`
123+
- `stream`: Stream path (e.g., `/live/livestream`, `/app/stream`)
124+
- `sdp`: SDP offer in standard WebRTC format
125+
- `server_id`: Identifier of the publishing server (optional)
126+
127+
## JSON Publish Response
128+
129+
**Direction**: Receiver → Publisher
130+
131+
**Success Response**:
132+
133+
```json
134+
{
135+
"code": 0,
136+
"session": "4ZcD:bILRCdg4YchH1OqRIvvscZAP",
137+
"sdp": "v=0\r\no=- 9876543210 2 IN IP4 192.168.1.200\r\ns=SRSReceiveSession\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=ice-lite\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 192.168.1.200\r\na=ice-ufrag:bILR\r\na=ice-pwd:4ZcDYchH1OqRIvvscZAP\r\na=fingerprint:sha-256 ...\r\na=setup:passive\r\na=mid:0\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\nm=video 9 UDP/TLS/RTP/SAVPF 106\r\nc=IN IP4 192.168.1.200\r\na=ice-ufrag:bILR\r\na=ice-pwd:4ZcDYchH1OqRIvvscZAP\r\na=fingerprint:sha-256 ...\r\na=setup:passive\r\na=mid:1\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:106 H264/90000\r\na=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\n",
138+
"server_id": "srs-server-b"
139+
}
140+
```
141+
142+
**Error Response**:
143+
144+
```json
145+
{
146+
"code": 1001,
147+
"message": "Stream already publishing",
148+
"details": "Stream /live/livestream is busy"
149+
}
150+
```
151+
152+
**Fields**:
153+
154+
- `code`: 0 for success, non-zero for error
155+
- `session`: ICE username (ufrag pair) for session identification
156+
- `sdp`: SDP answer in standard WebRTC format
157+
- `server_id`: Identifier of the receiving server (optional)
158+
- `message`: Error message (only for errors)
159+
- `details`: Detailed error information (optional)
160+
161+
## JSON Play Request
162+
163+
**Direction**: Player → Publisher
164+
165+
```json
166+
{
167+
"type": "play",
168+
"stream": "/live/livestream",
169+
"sdp": "v=0\r\no=- 1111111111 2 IN IP4 192.168.1.150\r\ns=SRSPlaySession\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=ice-lite\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:xYz9\r\na=ice-pwd:pQrS1234567890AbCdEf\r\na=fingerprint:sha-256 ...\r\na=setup:actpass\r\na=mid:0\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\nm=video 9 UDP/TLS/RTP/SAVPF 106\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:xYz9\r\na=ice-pwd:pQrS1234567890AbCdEf\r\na=fingerprint:sha-256 ...\r\na=setup:actpass\r\na=mid:1\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:106 H264/90000\r\n",
170+
"server_id": "srs-server-c"
171+
}
172+
```
173+
174+
**Fields**: Same as publish request, but `type` is `"play"` and SDP direction is `recvonly`.
175+
176+
## JSON Play Response
177+
178+
**Direction**: Publisher → Player
179+
180+
Same format as publish response.
181+
182+
## WebRTC Media Protocol
183+
184+
After JSON signaling completes, the connection follows standard WebRTC flow:
185+
186+
1. ICE Connectivity (STUN)
187+
188+
**STUN Binding Request** (first byte: 0x00):
189+
190+
```
191+
┌──────┬──────┬────────────────────────────────┐
192+
│ 0x00 │ 0x5C │ STUN Binding Request (92 bytes)│
193+
└──────┴──────┴────────────────────────────────┘
194+
```
195+
196+
**STUN Binding Response** (first byte: 0x01):
197+
198+
```
199+
┌──────┬──────┬────────────────────────────────┐
200+
│ 0x00 │ 0x68 │ STUN Binding Response (104 B) │
201+
└──────┴──────┴────────────────────────────────┘
202+
```
203+
204+
Session lookup: STUN username attribute contains ICE ufrag pair from SDP.
205+
206+
2. DTLS Handshake
207+
208+
**DTLS ClientHello** (first byte: 0x16 = 22):
209+
210+
```
211+
┌──────┬──────┬────────────────────────────────┐
212+
│ 0x01 │ 0x2A │ DTLS ClientHello (298 bytes) │
213+
└──────┴──────┴────────────────────────────────┘
214+
```
215+
216+
DTLS handshake is performed to exchange keys. rust-turbo uses these keys to set up SRTP encryption/decryption for its external WebRTC clients. However, the actual media transport **between SRS and rust-turbo** uses **plaintext RTP/RTCP** (not encrypted).
217+
218+
3. RTP/RTCP Media Transport (Plaintext)
219+
220+
**RTP Packet** (first byte: 0x80-0x9F, V=2, no marker):
221+
222+
```
223+
┌──────┬──────┬────────────────────────────────┐
224+
│ 0x05 │ 0xDC │ RTP packet (1500 bytes) │
225+
└──────┴──────┴────────────────────────────────┘
226+
```
227+
228+
**RTCP Packet** (first byte: 0x80-0x9F, PT=192-223):
229+
230+
```
231+
┌──────┬──────┬────────────────────────────────┐
232+
│ 0x00 │ 0x40 │ RTCP packet (64 bytes) │
233+
└──────┴──────┴────────────────────────────────┘
234+
```
235+
236+
**Important**: RTP/RTCP packets are sent in **plaintext** without SRTP/SRTCP encryption. The DTLS handshake provides keys for rust-turbo to encrypt/decrypt SRTP with external clients, but the transport between rust-turbo and SRS uses plaintext for performance and simplicity in internal communication.
237+
238+
## SRS Server Configuration
239+
240+
Enable WebRTC Private TCP listener on SRS server:
241+
242+
```nginx
243+
rtc_server {
244+
enabled on;
245+
246+
# WebRTC Private TCP listener for internal components
247+
private_tcp {
248+
enabled on;
249+
listen 9999;
250+
}
251+
}
252+
```
253+

0 commit comments

Comments
 (0)