video: add video compression support to tcpserversink sample #95862
Conversation
zephyrbot
requested review from
avolmat-st,
JarmouniA,
josuah,
kartben,
loicpoulain,
nashif and
ngphibang
|
Good idea to split dependencies out of #92884 to help review. Is the current PR expected to be reviewed now? I am asking just because I did not see You may be interested in this PR which introduces H.264 into the Thank you for bringing this forward! |
hfruchet-st
force-pushed
the
pr-tcpserversink-h264-n6
branch
from
926cd0c to
0fa7a7e
Compare
hfruchet-st
force-pushed
the
pr-tcpserversink-h264-n6
branch
from
0fa7a7e to
a6f426c
Compare
hfruchet-st
force-pushed
the
pr-tcpserversink-h264-n6
branch
from
a6f426c to
e0c6dad
Compare
|
All comments addressed so far... |
There was a problem hiding this comment.
Thanks @hfruchet-st. Few rather small points, appart from that LGTM.
samples/drivers/video/tcpserversink/boards/stm32n6570_dk_stm32n657xx_fsbl.conf
Outdated
Show resolved
Hide resolved
hfruchet-st
force-pushed
the
pr-tcpserversink-h264-n6
branch
2 times, most recently
from
ae60e9b to
bbf7119
Compare
hfruchet-st
force-pushed
the
pr-tcpserversink-h264-n6
branch
3 times, most recently
from
b744fe4 to
a50ba68
Compare
hfruchet-st
force-pushed
the
pr-tcpserversink-h264-n6
branch
from
a50ba68 to
a2db919
Compare
Add the zephyr,videoenc chosen node documentation for hardware video encoder support such as H264 or MJPEG video encoder. Signed-off-by: Hugues Fruchet <[email protected]>
Change printk debug traces to LOG_*. Signed-off-by: Hugues Fruchet <[email protected]>
Sync with video capture sample. Signed-off-by: Hugues Fruchet <[email protected]>
Allow to configure the number of allocated capture frames. This allows to make tradeof between framerate versus memory usage. 2 buffers allows to capture while sending data (optimal framerate). 1 buffer allows to reduce memory usage but capture framerate is lower. Signed-off-by: Hugues Fruchet <[email protected]>
Add video compression support to lowerize network bandwidth.
To visualise camera content on host PC, use GStreamer command line:
$> gst-launch-1.0 tcpclientsrc host=<board ip address> port=5000 ! \
decodebin ! autovideosink sync=false
Signed-off-by: Hugues Fruchet <[email protected]>
Add configuration files for the stm32n6570_dk board. This enables streaming over ethernet of the images captured by MB1854 camera module compressed in 1920x1080 H264 video bitstream. Signed-off-by: Hugues Fruchet <[email protected]>
hfruchet-st
force-pushed
the
pr-tcpserversink-h264-n6
branch
from
a2db919 to
4adeb03
Compare
|
There was a problem hiding this comment.
In order to avoid having this app always use the VENC in case of usage on the N6, I tried to have snippets in order to select the video encoder.
As an example I pushed a commit here avolmat-st@993aea3 as well a commit which removes unnecessary bits from the board specific conf in tcpserversink sample:
avolmat-st@6a76fac
I also made a similar snippet for the jpegenc which I've pushed in the stm32-jpeg PR (96678) in this commit 078a3a0
| CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=10 | ||
|
|
||
| # Camera interface | ||
| CONFIG_VIDEO_STM32_DCMIPP_SENSOR_PIXEL_FORMAT="pRAA" |
There was a problem hiding this comment.
Those 3 DCMIPP / SENSOR related config are not needed since they depend on the sensor and moreover are already set within the boards/shields/st_b_cams_imx_mb1854/Kconfig.defconfig which is applied when selecting the st_b_cams_imx_mb1854 shield
| CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=10 | ||
|
|
||
| # Camera interface | ||
| CONFIG_VIDEO_STM32_DCMIPP_SENSOR_PIXEL_FORMAT="pRAA" |
There was a problem hiding this comment.
Those 3 DCMIPP / SENSOR related config are not needed since they depend on the sensor and moreover are already set within the boards/shields/st_b_cams_imx_mb1854/Kconfig.defconfig which is applied when selecting the st_b_cams_imx_mb1854 shield
| CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=10 | ||
|
|
||
| # Camera interface | ||
| CONFIG_VIDEO_STM32_DCMIPP_SENSOR_PIXEL_FORMAT="pRAA" |
There was a problem hiding this comment.
Those 3 DCMIPP / SENSOR related config are not needed since they depend on the sensor and moreover are already set within the boards/shields/st_b_cams_imx_mb1854/Kconfig.defconfig which is applied when selecting the st_b_cams_imx_mb1854 shield
This PR adds support of camera streaming with compression of camera frames in video format such as H264 instead of sending big raw uncompressed frames, reducing drastically the network bandwidth needed for such use-case.
This video compression mode is enabled when a video encoder is found in device-tree, such as the VENC video encoder driver of the STM32N6 platform [1].
On :zephyr:board:
stm32n6570_dk, the MB1854 IMX335 camera module must be plugged in the CSI-2 camera connector. A RJ45 ethernet cable must be plugged in the ethernet CN6 connector.For an optimal image experience, it is adviced to embed the STM32 image signal processing middleware [2].
To build sample: (flash using ST-Link and boot with FSBL [3])
The default configuration allows to capture and stream 1080p camera content from STM32N6 which can be received, decoded and displayed by an host PC using a GStreamer command line:
[1] #92884
[2] https://github.com/stm32-hotspot/zephyr-stm32-mw-isp
[3] https://docs.zephyrproject.org/latest/boards/st/stm32n6570_dk/doc/index.html