stm32: addition of the STM32 JPEG Codec (NV12 -> JPEG encoding) #96678
Conversation
Add description of the ST JPEG codec IP node. Signed-off-by: Alain Volmat <[email protected]>
zephyrbot
added
area: Video
zephyrbot
requested review from
djiatsaf-st,
erwango,
etienne-lms,
FRASTM,
gautierg-st,
GeorgeCGV,
josuah,
loicpoulain,
marwaiehm-st,
mathieuchopstm and
ngphibang
avolmat-st
force-pushed
the
pr-stm32-jpeg
branch
from
63ec263 to
76d0e06
Compare
avolmat-st
force-pushed
the
pr-stm32-jpeg
branch
from
76d0e06 to
5b55717
Compare
| if (data->m2m.in.fmt.pixelformat != VIDEO_PIX_FMT_JPEG) { | ||
| struct stm32_jpeg_fmt_conf *conf = | ||
| stm32_jpeg_get_conf(data->m2m.in.fmt.pixelformat); | ||
| HAL_StatusTypeDef hret; |
There was a problem hiding this comment.
There are some inconsistencies in where variables are declared. Either move all of them to top-of-function, or move JPEG_ConfTypeDef Conf declaration to here.
drivers/video/video_stm32_jpeg.c
Outdated
| /* Convert one MCU at a time */ | ||
| stm32_jpeg_nv12_to_ycbcr_mcu(data->current_x_mcu++, data->current_y_mcu, | ||
| data->current_in->buffer, data->current_in->buffer + | ||
| data->m2m.in.fmt.width * data->m2m.in.fmt.height, | ||
| data->mcu_ycbcr, data->m2m.in.fmt.width); | ||
| if (data->current_x_mcu >= data->m2m.in.fmt.width / MCU_WIDTH) { | ||
| data->current_x_mcu = 0; | ||
| data->current_y_mcu++; | ||
| } |
There was a problem hiding this comment.
Seems to be an exact duplicate from L204 - maybe it could be factored in a common function.
drivers/video/video_stm32_jpeg.c
Outdated
| if ((data->m2m.in.fmt.pixelformat == VIDEO_PIX_FMT_JPEG && | ||
| data->m2m.out.fmt.pixelformat == VIDEO_PIX_FMT_JPEG) || | ||
| (data->m2m.in.fmt.pixelformat != VIDEO_PIX_FMT_JPEG && | ||
| data->m2m.out.fmt.pixelformat != VIDEO_PIX_FMT_JPEG)) { | ||
| LOG_ERR("One of input or output format must be JPEG"); |
There was a problem hiding this comment.
This is a bit difficult to follow IMO. How about:
| if ((data->m2m.in.fmt.pixelformat == VIDEO_PIX_FMT_JPEG && | |
| data->m2m.out.fmt.pixelformat == VIDEO_PIX_FMT_JPEG) || | |
| (data->m2m.in.fmt.pixelformat != VIDEO_PIX_FMT_JPEG && | |
| data->m2m.out.fmt.pixelformat != VIDEO_PIX_FMT_JPEG)) { | |
| LOG_ERR("One of input or output format must be JPEG"); | |
| /* shorthands, not *necessary* but helps readability */ | |
| uint32_t in_fmt = data->m2m.in.fmt.pixelformat; | |
| uint32_t out_fmt = data->m2m.out.fmt.pixelformat; | |
| if (!((in_fmt == VIDEO_PIX_FMT_JPEG && out_fmt != VIDEO_PIX_FMT_JPEG) | |
| || (in_fmt != VIDEO_PIX_FMT_JPEG && out_fmt == VIDEO_PIX_FMT_JPEG))) { | |
| LOG_ERR("Exactly one of input/output format must be JPEG"); |
| /* Check that input / output formats are correct */ | ||
| if ((data->m2m.in.fmt.pixelformat == VIDEO_PIX_FMT_JPEG && | ||
| data->m2m.out.fmt.pixelformat == VIDEO_PIX_FMT_JPEG) || | ||
| (data->m2m.in.fmt.pixelformat != VIDEO_PIX_FMT_JPEG && | ||
| data->m2m.out.fmt.pixelformat != VIDEO_PIX_FMT_JPEG)) { | ||
| LOG_ERR("One of input or output format must be JPEG"); | ||
| ret = -EINVAL; | ||
| goto out; | ||
| } | ||
|
|
||
| /* FIXME - temporary until the decoder support get added */ | ||
| if (data->m2m.in.fmt.pixelformat == VIDEO_PIX_FMT_JPEG) { | ||
| LOG_ERR("Decoder not yet implemented"); | ||
| ret = -EIO; | ||
| goto out; | ||
| } |
There was a problem hiding this comment.
Nit: These two checks operate on global data rather than common so maybe they can be done somewhere else.
There was a problem hiding this comment.
Usually we do those checks at the set_stream time right before starting the device. I could have also block (the decoder) at set_fmt time, but in such case I have need to distinguish input / output in the supported format table so I found it less intrusive to put this FIXME (and the check) here, considering that this JPEG check is going to disappear when the decoder will be implemented
drivers/video/video_stm32_jpeg.c
Outdated
|
|
||
| data->dev = dev; | ||
|
|
||
| /* Enable DCMIPP / CSI clocks */ |
There was a problem hiding this comment.
Forgot to update comment?
There was a problem hiding this comment.
Indeed. Actually useless comment (same as the next one about reset). The function name called are clear enough.
Thus removed.
drivers/video/video_stm32_jpeg.c
Outdated
| ret = video_init_ctrl(&data->jpeg_quality, dev, VIDEO_CID_JPEG_COMPRESSION_QUALITY, | ||
| (struct video_ctrl_range) {.min = 5, .max = 100, | ||
| .step = 1, .def = 50}); | ||
| if (ret) { |
There was a problem hiding this comment.
Nit: for consistency
| if (ret) { | |
| if (ret < 0) { |
drivers/video/video_stm32_jpeg.c
Outdated
| .jpeg_hclken = \ | ||
| {.bus = DT_CLOCKS_CELL(DT_DRV_INST(n), bus), \ | ||
| .enr = DT_CLOCKS_CELL(DT_DRV_INST(n), bits)}, \ |
There was a problem hiding this comment.
| .jpeg_hclken = \ | |
| {.bus = DT_CLOCKS_CELL(DT_DRV_INST(n), bus), \ | |
| .enr = DT_CLOCKS_CELL(DT_DRV_INST(n), bits)}, \ | |
| .jpeg_hclken = STM32_DT_INST_CLOCKS(n), \ |
avolmat-st
force-pushed
the
pr-stm32-jpeg
branch
from
5b55717 to
0946ea7
Compare
Initial version of the support for the STM32 JPEG codec, currently supporting only NV12 to JPEG without DMA support and using SW based conversion from NV12 to MCU required for the JPEG codec. Signed-off-by: Alain Volmat <[email protected]>
Add the node describing the JPEG codec within the stm32n6 Signed-off-by: Alain Volmat <[email protected]>
Addition of a snippet in order to enable and configure the STM32 JPEG encoder. Signed-off-by: Alain Volmat <[email protected]>
avolmat-st
force-pushed
the
pr-stm32-jpeg
branch
from
0946ea7 to
078a3a0
Compare
|
The PR adds an initial version of the STM32 JPEG Codec, available on several STM32. This PR adds it within the N6.
While the HW can perform both encoding & decoding, for the time being only encoding is provided.
The HW needs data in MCU format in order to encore into JPEG. Formating of the data into MCU is provided only for the NV12 for the time being (for that purpose probably only NV12 should be kept in this driver in order to avoid confusion).
DMA can also be used in order to inject data into the codec but this is not implemented in this version.
Main purpose of this PR is actually to provide material in order to discuss about a possible abstraction for the m2m buffers.
The driver already provide several structures which embed k_fifo but it also make it clear that m2m helper functions could be possible in order to handle in a generic way checking about in & out buffer, and have the helper call a new m2m codec api which would be in charge to perform the conversion only.
As shown in this code, the queue / dequeue function do not have anything HW specific and also take care of pushing / pulling into / from the right fifo and triggering a codec run if all condition are matched.
I hope this could generate discussion about this and based on that I can provide some helpers to hide the non HW specific code (buffer handling).
This has been tested using the tcpclientsink application modified within the PR #95862.
I however had to perform some more changes into the application, especially in order to properly set the in / out formats of the encoded video device. I provide those modifications for reference within this commit (avolmat-st@86249bf) part of my git repo.
This has been tested successfully on the STM32N6 connected over network with laptop into which following command is ran in order to grab JPEG data: