video: introduction of STM32 VENC driver for H264 hardware video compression #92884
Conversation
|
The following west manifest projects have changed revision in this Pull Request:
✅ All manifest checks OK Note: This message is automatically posted and updated by the Manifest GitHub Action. |
|
There was a problem hiding this comment.
Thank you for preparing VENC!
This will really make video over network on Zephyr interesting.
It seems like Zephyr lacks a few APIs from Linux (no H.264 format definition, no planar API for NV12...), so if anything feels like missing, let us know so that the missing API can be provided in parallel.
No specific processing is necessary currently as part of the stm32_dcmipp_isp_start function hence remove the assert(0) currently preventing stop of the isp. Signed-off-by: Alain Volmat <[email protected]>
Allow to call HAL_DCMIPP_PIPE_SetConfig when the pipe state is READY. Currently it is only possible to use this function if the pipe state is RESET and ERROR states. Indeed, it is possible to reconfigure the PIPE as soon as the pipe is not currently being used, aka in the READY state. With that done, it becomes possible to change the PIPE configuration between two use-cases without having to DeInit / Init the HAL_DCMIPP or use the unitary pixel packer functions. Signed-off-by: Alain Volmat <[email protected]>
There was a problem hiding this comment.
First round of comment, driver aside.
Please split the PR and do a specific PR for the first commits that are not directly linked to the VENC driver. These can be reviewed and integrated faster.
For the changes on the sample, I let video team decide if this should be kept within the tcpserver sample or split into a different sample
hfruchet-st
force-pushed
the
iso_stm32n6_1080p_h264_streaming
branch
from
a32b1da to
deac1fb
Compare
zephyrbot
requested review from
djiatsaf-st,
etienne-lms,
FRASTM,
gautierg-st,
GeorgeCGV,
loicpoulain and
marwaiehm-st
Pull hal_stm32 vc8000nanoe library. Signed-off-by: Hugues Fruchet <[email protected]>
Addition of description for the STM32 Video encoder (VENC). Signed-off-by: Hugues Fruchet <[email protected]>
The STM32 video encoder (VENC) peripheral is a hardware accelerator allowing to compress RGB/YUV frames into H264 video bitstream chunks. Signed-off-by: Hugues Fruchet <[email protected]>
Add node describing the venc in stm32n6.dtsi Signed-off-by: Hugues Fruchet <[email protected]>
hfruchet-st
force-pushed
the
iso_stm32n6_1080p_h264_streaming
branch
from
857cd02 to
7b4b66b
Compare
JarmouniA
removed
area: Build System
area: Debugging
area: Devicetree
platform: nRF
JarmouniA
removed request for
carlescufi,
ioannis-karachalios,
anangl,
KhiemNguyenT,
nika-nordic,
masz-nordic,
krish2718,
sachinthegreen and
djiatsaf-st
|
Bot added unrelated labels and reviewers. |
|
Please retry analysis of this Pull-Request directly on SonarQube Cloud |
| ret = encoder_start(data, output); | ||
|
|
||
| /* Output buffer is now filled with SPS/PPS header, return it to application */ | ||
| goto out; |
There was a problem hiding this comment.
My understanding is that when starting the encoder, this will only produce data into the output buffer, and input buffer is not used in such case. Is that correct ?
I feel that in the way it is currently done, the first input frame is going to be lost / unused and returned to the application (and thus back to the camera side) without actually having been used.
If that's the case it might be better to add a comment here, with maybe a TODO / FIXME or something like that in order to address in a further PR.
| LOG_DBG("output=%p, encOut.streamSize=%d", output, encOut.streamSize); | ||
|
|
||
| k_fifo_put(&data->out_fifo_in, output); | ||
| k_fifo_put(&data->out_fifo_out, output); |
There was a problem hiding this comment.
The same output cannot be put back into both side of the output queue. Doing so would mean that for example the encoder could override the output frame while it is currently used by the application (currently tcpserversink), is my understanding correct ? It should only be put into the out_fifo_out I think.
|
|
||
| out: | ||
| k_fifo_put(&data->in_fifo_out, input); | ||
| k_fifo_put(&data->out_fifo_out, output); |
There was a problem hiding this comment.
doing k_fifo_put of output into the out_fifo_out leads to pushing the output buffer twice into out_fifo_out if we consider the k_fifo_put already done above ??
|
|
||
| static int stm32_venc_get_caps(const struct device *dev, struct video_caps *caps) | ||
| { | ||
| caps->format_caps = fmts; |
There was a problem hiding this comment.
I think the format_caps should depends on the caps->type and return different format depending on INPUT or OUTPUT. Currently that might be RGB565/NV12 for the INPUT and H264 for the OUTPUT ?
This PR introduces H264 hardware video compression thanks to VENC peripheral of STM32N6.
A new stm32 video encoder driver has been introduced which relies on vc8000nanoe software stack [1].
This is a first basic porting to enable H264 encoding use-case.
RAM memory resources required by VENC encoding process are allocated in external PSRAM, camera frames
captured by DCMIPP and encoded by VENC are also allocated in external PSRAM.
The video encoder driver currently lack of controls to configure the encoding process, default configuration
targets H264 1080p 2Mb/s constant bitrate.
In order to test, the tcpserversink video sample has been enhanced to support video compression in order to stream small H264 compressed video chunks instead of big raw uncompressed images [2].
[1] zephyrproject-rtos/hal_stm32#295
[2] #95862