|
| 1 | +# ESP Peer – WebRTC PeerConnection Component |
| 2 | + |
| 3 | +`esp_peer` is a full-featured WebRTC PeerConnection implementation optimized for Espressif platforms. It enables peer-to-peer communication with audio, video, and data channels, adhering to the standard WebRTC protocol stack. |
| 4 | + |
| 5 | +--- |
| 6 | + |
| 7 | +## 🔍 Overview |
| 8 | + |
| 9 | +Derived from [libpeer](https://github.com/sepfy/libpeer.git), `esp_peer` adds platform-specific enhancements and optimizations for the ESP32 series. It provides a production-grade WebRTC stack supporting: |
| 10 | + |
| 11 | +- **ICE** – Interactive Connectivity Establishment |
| 12 | +- **DTLS** – Datagram Transport Layer Security |
| 13 | +- **SCTP** – Stream Control Transmission Protocol |
| 14 | +- **RTP/SRTP** – Real-time Transport Protocol with secure variants |
| 15 | + |
| 16 | +--- |
| 17 | + |
| 18 | +## ✨ Key Features |
| 19 | + |
| 20 | +### ✅ WebRTC Protocol Stack |
| 21 | + |
| 22 | +- **TURN Support**: Implements [RFC5766](https://datatracker.ietf.org/doc/html/rfc5766) and [RFC8656](https://datatracker.ietf.org/doc/html/rfc8656) |
| 23 | +- **Dual ICE Roles**: Operates as *Controlling* or *Controlled* |
| 24 | +- **Optimized ICE Pairing**: Fast and efficient candidate selection |
| 25 | +- **RTP Reliability**: Implements NACK, jitter buffering, and retransmission |
| 26 | +- **Robust SCTP**: Supports multi-streaming, fragmentation, and SACK |
| 27 | +- **Threaded Operation**: Non-blocking send/receive with dedicated tasks |
| 28 | + |
| 29 | +### 🎹 Media Support |
| 30 | + |
| 31 | +- **Audio Codecs**: G711A (PCMA), G711U (PCMU), OPUS |
| 32 | +- **Video Codecs**: H.264, MJPEG |
| 33 | +- **Data Channels**: Reliable/unreliable; ordered/unordered modes |
| 34 | +- **Flexible Media Modes**: Send-only, receive-only, or full-duplex |
| 35 | + |
| 36 | +### 🧹 Modular & Configurable |
| 37 | + |
| 38 | +- **Pluggable Interfaces**: Clean abstraction via `esp_peer_ops_t` |
| 39 | +- **Default Implementation**: Ready-to-use via `esp_peer_get_default_impl()` |
| 40 | +- **Resource Control**: Tune memory usage, timeouts, buffer sizes |
| 41 | + |
| 42 | +--- |
| 43 | + |
| 44 | +## 🚀 Getting Started |
| 45 | + |
| 46 | +### 1️⃣ Peer Configuration |
| 47 | + |
| 48 | +```c |
| 49 | +esp_peer_cfg_t cfg = { |
| 50 | + .role = ESP_PEER_ROLE_CONTROLLING, |
| 51 | + .ice_trans_policy = ESP_PEER_ICE_TRANS_POLICY_ALL, |
| 52 | + .audio_info = { |
| 53 | + .codec = ESP_PEER_AUDIO_CODEC_OPUS, |
| 54 | + .sample_rate = 48000, |
| 55 | + .channel = 1 |
| 56 | + }, |
| 57 | + .video_info = { |
| 58 | + .codec = ESP_PEER_VIDEO_CODEC_H264, |
| 59 | + .width = 640, |
| 60 | + .height = 480, |
| 61 | + .fps = 30 |
| 62 | + }, |
| 63 | + .audio_dir = ESP_PEER_MEDIA_DIR_SEND_RECV, |
| 64 | + .video_dir = ESP_PEER_MEDIA_DIR_SEND_RECV, |
| 65 | + .enable_data_channel = true, |
| 66 | + .ctx = user_context, |
| 67 | + .on_state = state_callback, |
| 68 | + .on_msg = message_callback, |
| 69 | + .on_audio_data = audio_callback, |
| 70 | + .on_video_data = video_callback, |
| 71 | + .on_data = data_callback |
| 72 | +}; |
| 73 | +``` |
| 74 | + |
| 75 | +--- |
| 76 | + |
| 77 | +### 2️⃣ Migration: Browser WebRTC → ESP Peer |
| 78 | + |
| 79 | +Easily migrate from browser-style WebRTC code to the embedded `esp_peer` API. Refer to [`peer_demo.c`](examples/peer_demo/main/peer_demo.c) for a complete walkthrough. |
| 80 | + |
| 81 | +#### 🔄 API Mapping |
| 82 | + |
| 83 | +| Browser WebRTC | ESP Peer | Notes | |
| 84 | +| -------------------------------------------------- | ----------------------------------- | --------------------------------- | |
| 85 | +| `new RTCPeerConnection()` | `esp_peer_open()` | Configuration-based instantiation | |
| 86 | +| `pc.onicecandidate` | `.on_msg` callback | Candidate/SDP exchange | |
| 87 | +| `pc.onconnectionstatechange` | `.on_state` callback | Connection lifecycle | |
| 88 | +| `pc.ontrack` | `.on_audio_data` / `.on_video_data` | Media reception | |
| 89 | +| `pc.ondatachannel` | `.on_data` callback | Data channel messages | |
| 90 | +| `createOffer()` / `createAnswer()` | `esp_peer_new_connection()` | Auto-generates SDP | |
| 91 | +| `setLocalDescription()` / `setRemoteDescription()` | `esp_peer_send_msg()` | Exchange SDP manually | |
| 92 | +| `dataChannel.send()` | `esp_peer_send_data()` | Send data via SCTP | |
| 93 | +| Manual event loop | `esp_peer_main_loop()` | Handled in background task | |
| 94 | + |
| 95 | +--- |
| 96 | + |
| 97 | +### 3️⃣ Data Channel Usage |
| 98 | + |
| 99 | +#### ✔️ Auto Creation |
| 100 | + |
| 101 | +If `.enable_data_channel = true`: |
| 102 | + |
| 103 | +- **Client Role**: Creates the default channel automatically |
| 104 | +- **Server Role**: Waits for peer to create and pair the channel |
| 105 | + |
| 106 | +#### 🛠 Manual Creation |
| 107 | + |
| 108 | +For advanced use cases (e.g., multiple or custom-configured channels): |
| 109 | + |
| 110 | +```c |
| 111 | +esp_peer_cfg_t cfg = { |
| 112 | + .enable_data_channel = true, |
| 113 | + .manual_ch_create = true, |
| 114 | + // ... |
| 115 | +}; |
| 116 | +esp_peer_open(&cfg, esp_peer_get_default_impl(), &peer); |
| 117 | + |
| 118 | +// After ESP_PEER_STATE_DATA_CHANNEL_CONNECTED |
| 119 | +esp_peer_data_channel_cfg_t ch_cfg = { |
| 120 | + .type = ESP_PEER_DATA_CHANNEL_RELIABLE, |
| 121 | + .ordered = true, |
| 122 | + .label = "my_channel" |
| 123 | +}; |
| 124 | +esp_peer_create_data_channel(peer, &ch_cfg); |
| 125 | + |
| 126 | +// Once open: |
| 127 | +esp_peer_data_frame_t data_frame = { |
| 128 | + .type = ESP_PEER_DATA_CHANNEL_STRING, |
| 129 | + .stream_id = 0, |
| 130 | + .data = (uint8_t*)"Hello WebRTC!", |
| 131 | + .size = 13 |
| 132 | +}; |
| 133 | +esp_peer_send_data(peer, &data_frame); |
| 134 | +``` |
| 135 | +
|
| 136 | +## 4️⃣ Signaling Explanation |
| 137 | +
|
| 138 | +WebRTC requires a signaling mechanism to exchange SDP and ICE candidates between peers. This module **does not provide a built-in signaling transport**. |
| 139 | +
|
| 140 | +You can refer to the complete [esp-webrtc-solution](https://github.com/espressif/esp-webrtc-solution) which includes practical signaling implementations such as handy OpenAI, WHIP, and AppRTC, or you may develop a custom signaling system tailored to your application. |
| 141 | +
|
| 142 | +--- |
| 143 | +
|
| 144 | +## ⚙️ Fine-Tuning (Optional) |
| 145 | +
|
| 146 | +Tune for memory or performance trade-offs using `esp_peer_default_cfg_t`: |
| 147 | +
|
| 148 | +```c |
| 149 | +esp_peer_default_cfg_t default_cfg = { |
| 150 | + .agent_recv_timeout = 100, |
| 151 | + .data_ch_cfg = { |
| 152 | + .cache_timeout = 5000, |
| 153 | + .send_cache_size = 102400, |
| 154 | + .recv_cache_size = 102400, |
| 155 | + }, |
| 156 | + .rtp_cfg = { |
| 157 | + .audio_recv_jitter = { |
| 158 | + .cache_timeout = 100, |
| 159 | + .resend_delay = 20, |
| 160 | + .cache_size = 102400, |
| 161 | + }, |
| 162 | + .video_recv_jitter = { |
| 163 | + .cache_timeout = 100, |
| 164 | + .resend_delay = 20, |
| 165 | + .cache_size = 409600, |
| 166 | + }, |
| 167 | + .send_pool_size = 409600, |
| 168 | + .send_queue_num = 256, |
| 169 | + .max_resend_count = 3 |
| 170 | + } |
| 171 | +}; |
| 172 | +
|
| 173 | +cfg.extra_cfg = &default_cfg; |
| 174 | +cfg.extra_size = sizeof(default_cfg); |
| 175 | +esp_peer_open(&cfg, esp_peer_get_default_impl(), &peer); |
| 176 | +``` |
| 177 | + |
| 178 | +--- |
| 179 | + |
| 180 | +## 📉 Minimum Resource Requirements |
| 181 | + |
| 182 | +`esp_peer` is highly configurable to support low-memory environments. Even on platforms **without PSRAM**, a minimal setup uses **< 60 KB** RAM. |
| 183 | + |
| 184 | +See the [`peer_demo`](examples/peer_demo) example for how two peers can run concurrently on an **ESP32-S3** without external memory. |
| 185 | + |
| 186 | +--- |
| 187 | + |
| 188 | +## 📦 Dependencies |
| 189 | + |
| 190 | +### Libraries |
| 191 | + |
| 192 | +- **libSRTP** – Secure RTP (bundled) |
| 193 | +- **mbedTLS** – Required for DTLS (bundled with ESP-IDF) |
| 194 | + |
| 195 | +### ESP-IDF Settings |
| 196 | + |
| 197 | +Ensure these config options are enabled: |
| 198 | + |
| 199 | +```ini |
| 200 | +CONFIG_MBEDTLS_SSL_PROTO_DTLS=y |
| 201 | +CONFIG_MBEDTLS_SSL_DTLS_SRTP=y |
| 202 | +``` |
| 203 | + |
| 204 | +--- |
| 205 | + |
| 206 | +## 🔀 PeerConnection State Machine |
| 207 | + |
| 208 | +```mermaid |
| 209 | +stateDiagram-v2 |
| 210 | +direction LR |
| 211 | +[*] --> NEW_CONNECTION |
| 212 | +NEW_CONNECTION --> PAIRING |
| 213 | +PAIRING --> PAIRED |
| 214 | +PAIRED --> CONNECTING |
| 215 | +CONNECTING --> CONNECTED |
| 216 | +CONNECTED --> DATA_CHANNEL_CONNECTED |
| 217 | +DATA_CHANNEL_CONNECTED --> DATA_CHANNEL_OPENED |
| 218 | +DATA_CHANNEL_OPENED --> DISCONNECTED |
| 219 | +DISCONNECTED --> CLOSED |
| 220 | +CLOSED --> [*] |
| 221 | +``` |
| 222 | + |
| 223 | +--- |
| 224 | + |
| 225 | +## 🧠 Performance Tips |
| 226 | + |
| 227 | +- **Tune Buffer Sizes**: Trade latency for memory based on use case |
| 228 | +- **Adjust Timeouts**: Adapt to high-latency or lossy networks |
| 229 | +- **Use Dedicated Task**: Run `esp_peer_main_loop()` in its own thread |
| 230 | +- **Profile Resource Usage**: Monitor heap and stack for optimization |
| 231 | + |
| 232 | +--- |
| 233 | + |
| 234 | +## 📬 Contact & Support |
| 235 | + |
| 236 | +This component is part of the [esp-webrtc-solution](https://github.com/espressif/esp-webrtc-solution), offering complete WebRTC capabilities including media stream (esp_capture, av_render), varieties of signaling (OpenAI, WHIP, AppRTC, etc.) and many practical examples. |
| 237 | + |
| 238 | +🔧 Found a bug? Have a suggestion?\ |
| 239 | +Open an issue on GitHub: [esp-webrtc-solution/issues](https://github.com/espressif/esp-webrtc-solution/issues) |
| 240 | + |
| 241 | +We’re here to help! |
| 242 | + |
| 243 | +--- |
0 commit comments