Skip to content

Commit f89f08f

Browse files
TempoTianTempoTian
committed
Add WebRTC video call application
1 parent ebb620a commit f89f08f

File tree

18 files changed

+1182
-7
lines changed

18 files changed

+1182
-7
lines changed

‎README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,7 @@ This demo implements a doorbell application that can:
2121
- Send real-time video data to a controller while supporting two-way audio communication
2222

2323
### 3. Peer Demo
24-
This demo mainly show how to use `esp_peer` API to buildup a WebRTC application from scratch.
24+
This demo mainly show how to use `esp_peer` API to buildup a WebRTC application from scratch.
25+
26+
### 4. Video Call Solution
27+
This demo show how to use `esp_webrtc` data channel to build up video call application.

‎components/esp_capture/src/impl/capture_audio_src/capture_aud_aec_src.c

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -335,15 +335,15 @@ static int audio_aec_src_start(esp_capture_audio_src_if_t *h)
335335
ESP_LOGE(TAG, "Failed to allocate cache frame");
336336
return ESP_CAPTURE_ERR_NOT_SUPPORTED;
337337
}
338+
src->samples = 0;
339+
src->cached_read_pos = src->cache_fill = 0;
340+
src->stopping = false;
338341

339342
media_lib_thread_handle_t thread = NULL;
340343
media_lib_thread_create_from_scheduler(&thread, "buffer_in", audio_aec_src_buffer_in_thread, src);
341344
#endif
342345
src->start = true;
343-
src->samples = 0;
344-
src->cached_read_pos = src->cache_fill = 0;
345346
src->in_quit = false;
346-
src->stopping = false;
347347
return ESP_CAPTURE_ERR_OK;
348348
}
349349

@@ -458,4 +458,5 @@ esp_capture_audio_src_if_t *esp_capture_new_audio_aec_src(esp_capture_audio_aec_
458458
src->channel_mask = cfg->channel_mask;
459459
return &src->base;
460460
}
461+
461462
#endif

‎solutions/doorbell_demo/sdkconfig.defaults.esp32p4

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,6 @@ CONFIG_CAMERA_SC2336_MIPI_RAW8_1920x1080_30FPS=y
1313
# Enable ISP pipelines
1414
CONFIG_ESP_VIDEO_ENABLE_ISP_PIPELINE_CONTROLLER=y
1515

16+
# Set Slave target type
17+
CONFIG_IDF_SLAVE_TARGET="esp32c6"
18+
CONFIG_SLAVE_IDF_TARGET_ESP32C6=y

‎solutions/videocall_demo/CMakeLists.txt

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# The following lines of boilerplate have to be in your project's CMakeLists
2+
# in this exact order for cmake to work correctly
3+
cmake_minimum_required(VERSION 3.5)
4+
5+
# include($ENV{ADF_PATH}/CMakeLists.txt)
6+
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
7+
8+
if("${IDF_TARGET}" STREQUAL "esp32p4")
9+
set(EXTRA_COMPONENT_DIRS "$ENV{IDF_PATH}/examples/ethernet/basic/components/ethernet_init")
10+
endif()
11+
12+
list(APPEND EXTRA_COMPONENT_DIRS "../../components")
13+
14+
list(APPEND EXTRA_COMPONENT_DIRS "../common")
15+
16+
project(videocall_demo)

‎solutions/videocall_demo/README.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# VideoCall Demo
2+
3+
## OverView
4+
This demo showcases how to use `esp_webrtc` to build a device to device video call application. The code uses a modified version of the [apprtc](https://github.com/webrtc/apprtc) as signaling server and a customized command channel.
5+
6+
7+
## Hardware requirement
8+
The default setup uses the [ESP32P4-Function-Ev-Board](https://docs.espressif.com/projects/esp-dev-kits/en/latest/esp32p4/esp32-p4-function-ev-board/user_guide.html), which includes a SC2336 camera. For a video call, two `ESP32P4-Function-Ev-Board` devices are required, one as the caller and the other as the callee.
9+
10+
## How to build
11+
12+
### IDF version
13+
Can select IDF master or IDF release v5.4.
14+
15+
### Change Default Settings
16+
1. Modify the Wi-Fi SSID and password in the file in [settings.h](main/settings.h)
17+
2. If you are using a different camera type or resolution, update the settings for the camera type and resolution in [settings.h](main/settings.h)
18+
3. If you are using USB-JTAG to download, uncomment the following configuration in [sdkconfig.defaults.esp32p4](sdkconfig.defaults.esp32p4)
19+
```
20+
CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y
21+
```
22+
23+
### Build
24+
```
25+
idf.py -p YOUR_SERIAL_DEVICE flash monitor
26+
```
27+
28+
## Testing
29+
30+
After the board boots up, it will attempt to connect to the configured Wi-Fi SSID. If you want to connect to a different Wi-Fi STA, you can use the following CLI command:
31+
```
32+
wifi ssid psw
33+
```
34+
35+
After a successful Wi-Fi connection, use the join command to join a random room. Both boards must enter the same room:
36+
```
37+
join mytestroom
38+
```
39+
40+
### Interactions
41+
42+
1. **Ring**
43+
- One board clicks the boot button or enters the `b`` command in the console to ring the peer board.
44+
- The peer board receives the `Ring` command and plays ring music.
45+
46+
2. **Accept:**
47+
- The peer board presses the boot button or enters the `b` command to accept the call.
48+
- The call will automatically reset after a timeout if not accepted.
49+
50+
3. **Hang Off:**
51+
- The peer board presses the boot button or enters the `b` command to end the ongoing call.
52+
53+
4. **Clear-up Test:**
54+
- On either board, enter the `leave` command to exit the room.
55+
56+
## Technical Details
57+
To support the video call functionality, this demo uses [apprtc](https://github.com/webrtc/apprtc) and requires separate signaling from the peer connection build logic. The peer connection is only established when a special signaling message is received from the peer.
58+
59+
### Key Changes in `esp_webrtc`:
60+
- **`no_auto_reconnect` Configuration**: This configuration disables the automatic building of the peer connection when the signaling connection is established.
61+
- **`esp_webrtc_enable_peer_connection` API**: A new API is introduced to manually control the connection and disconnection of the peer connection.
62+
- **Video over Data Channel**: To enable video over the data channel, use the following configuration:
63+
```
64+
.enable_data_channel = true,
65+
.video_over_data_channel = true,
66+
```
67+
All other steps follow the typical call flow of `esp_webrtc`. For more details on the standard connection build flow, refer to the [Connection Build Flow](../../components/esp_webrtc/README.md#typical-call-sequence-of-esp_webrtc).

‎solutions/videocall_demo/main/CMakeLists.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
idf_component_register(SRCS "webrtc.c" "main.c" "board.c" "media_sys.c"
2+
EMBED_TXTFILES "ring.aac"
3+
INCLUDE_DIRS ".")

‎solutions/videocall_demo/main/board.c

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
/* Do simple board initialize
2+
3+
This example code is in the Public Domain (or CC0 licensed, at your option.)
4+
5+
Unless required by applicable law or agreed to in writing, this
6+
software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
7+
CONDITIONS OF ANY KIND, either express or implied.
8+
*/
9+
10+
#include <stdio.h>
11+
#include "esp_log.h"
12+
#include "codec_init.h"
13+
#include "codec_board.h"
14+
#include "esp_codec_dev.h"
15+
#include "sdkconfig.h"
16+
#include "settings.h"
17+
18+
static const char *TAG = "Board";
19+
20+
void init_board()
21+
{
22+
ESP_LOGI(TAG, "Init board.");
23+
set_codec_board_type(TEST_BOARD_NAME);
24+
// Notes when use playback and record at same time, must set reuse_dev = false
25+
codec_init_cfg_t cfg = {.reuse_dev = false};
26+
init_codec(&cfg);
27+
board_lcd_init();
28+
}

‎solutions/videocall_demo/main/common.h

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
/* Video Call Demo
2+
3+
This example code is in the Public Domain (or CC0 licensed, at your option.)
4+
5+
Unless required by applicable law or agreed to in writing, this
6+
software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
7+
CONDITIONS OF ANY KIND, either express or implied.
8+
*/
9+
10+
#pragma once
11+
12+
#include "settings.h"
13+
#include "media_sys.h"
14+
#include "network.h"
15+
#include "sys_state.h"
16+
17+
#ifdef __cplusplus
18+
extern "C" {
19+
#endif
20+
21+
/**
22+
* @brief Initialize for board
23+
*/
24+
void init_board(void);
25+
26+
/**
27+
* @brief Start WebRTC
28+
*
29+
* @param[in] url Signaling URL
30+
*
31+
* @return
32+
* - 0 On success
33+
* - Others Fail to start
34+
*/
35+
int start_webrtc(char *url);
36+
37+
/**
38+
* @brief Query WebRTC status
39+
*/
40+
void query_webrtc(void);
41+
42+
/**
43+
* @brief Stop WebRTC
44+
*
45+
* @return
46+
* - 0 On success
47+
* - Others Fail to stop
48+
*/
49+
int stop_webrtc(void);
50+
51+
#ifdef __cplusplus
52+
}
53+
#endif

‎solutions/videocall_demo/main/idf_component.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
## IDF Component Manager Manifest File
2+
dependencies:
3+
## Required IDF version
4+
idf:
5+
version: ">=4.1.0"
6+
capture_audio_src:
7+
path: ../../../components/esp_capture/src/impl/capture_audio_src
8+
capture_video_src:
9+
path: ../../../components/esp_capture/src/impl/capture_video_src
10+
capture_audio_enc:
11+
path: ../../../components/esp_capture/src/impl/capture_audio_enc
12+
capture_video_enc:
13+
path: ../../../components/esp_capture/src/impl/capture_video_enc
14+
peer_default:
15+
path: ../../../components/esp_webrtc/impl/peer_default
16+
render_impl:
17+
path: ../../../components/av_render/render_impl

0 commit comments

Comments
 (0)