Computer Vision application demonstrating the deployment of object detection models on the STM32N6570-DK board and stream results through USB using UVC/H264 format.
Following UVC resolution and framerate will be output according to connected camera :
- 1280x720@30fps for IMX335
- 1120x720@30fps for VD66GY
- 640x480@30fps for VD55G1
This top readme gives an overview of the app. Additional documentation is available in the Doc folder.
- Multi-threaded application flow (FreeRTOS)
- NPU accelerated quantized AI model inference
- Dual DCMIPP pipes
- DCMIPP crop, decimation, downscale
- DCMIPP ISP usage
- H264 encoder
- USB UVC (Azure RTOS USBX)
- Dev mode
- Boot from external flash
Supported development platforms:
- STM32N6570-DK Discovery Board
- Connect to the onboard ST-LINK debug adapter (CN6) using a USB-C to USB-C cable for sufficient power.
- An additional USB cable to connect USB1 (CN18) to the host computer for UVC streaming.
- OTP fuses are configured for xSPI IOs to achieve maximum speed (200MHz) on xSPI interfaces.
STM32N6570-DK board with MB1854B IMX335.
Supported camera modules:
- Provided IMX335 camera module
- STEVAL-55G1MBI
- STEVAL-66GYMAI1
- IAR Embedded Workbench for Arm (EWARM 9.40.1) + N6 patch (EWARMv9_STM32N6xx_V1.0.0)
- STM32CubeIDE (v1.17.0)
- STM32CubeProgrammer (v2.18.0)
- STEdgeAI (v2.2.0)
The STM32N6 series does not have internal flash memory. To retain firmware after a reboot, program it into the external flash. Alternatively, you can load firmware directly into SRAM (development mode), but note that the program will be lost if the board is powered off in this mode.
Development Mode: used for loading firmware into RAM during a debug session or for programming firmware into external flash.
Boot from Flash: used to boot firmware from external flash.
| STM32N6570-DK | |
|---|---|
| Boot from flash |  |
| Development mode |  |
You can see application messages by attaching a console application to the ST-Link console output. Use the following console parameters:
- Baud rate of 115200 bps.
- No parity.
- One stop bit.
Three binaries must be programmed in the board's external flash using the following procedure:
- Set the board to development mode.
- Program
Binary/ai_fsbl.hex(to be done once) (First stage boot loader). - Program
Binary/network_data.hex(parameters of the networks; to be changed only when the network is changed). - Program
Binary/x-cube-n6-ai-h264-usb-uvc.hex(firmware application). - Set the board to boot from flash mode.
- Power cycle the board.
- Launch host camera application
See How to Program Hex Files STM32CubeProgrammer.
Make sure to have the STM32CubeProgrammer bin folder added to your path.
export DKEL="<STM32CubeProgrammer_N6 Install Folder>/bin/ExternalLoader/MX66UW1G45G_STM32N6570-DK.stldr"
# First Stage Boot Loader
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w Binary/ai_fsbl.hex
# Network Parameters and Biases
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w Binary/network_data.hex
# Application Firmware
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w Binary/x-cube-n6-ai-h264-usb-uvc.hexObserve the streaming of the camera and the output of the computer vision network on a camera application:
-
Linux users: Use a webcam application that is able to decode H264 (e.g., guvcview, VLC).
-
Windows users: Install ffmpeg and then run the following command:
ffplay.exe -f dshow -i video="STM32 uvc"
Before building and running the application, you have to program network_data.hex (model weights and biases).
This step only has to be done once unless you change the AI model. See Quickstart Using Prebuilt Binaries for details.
More information about boot modes is available in the Boot Overview.
Set your board to development mode.
Double-click on STM32CubeIDE/.project to open the project in STM32CubeIDE. Build and run using the build and run buttons.
Double-click on EWARM/Project.eww to open the project in the IAR IDE. Build and run using the build and run buttons.
Before running the commands below, make sure to have the necessary commands in your PATH.
- Build the project using the provided
Makefile:
make -j8- Open a GDB server connected to the STM32 target:
ST-LINK_gdbserver -p 61234 -l 1 -d -s -cp <path-to-stm32cubeprogramer-bin-dir> -m 1 -g- In a separate terminal session, launch a GDB session to load the firmware image into the device memory:
$ arm-none-eabi-gdb build/Project.elf
(gdb) target remote :61234
(gdb) monitor reset
(gdb) load
(gdb) continueSet your board to development mode.
Double-click on STM32CubeIDE/.project to open the project in STM32CubeIDE. Build with build button.
Double-click on EWARM/Project.eww to open the project in the IAR IDE. Build with build button.
Before running the commands below, make sure to have the necessary commands in your PATH.
- Build the project using the provided
Makefile:
make -j8Once your app is built with Makefile, STM32CubeIDE, or EWARM, you must add a signature to the bin file:
STM32_SigningTool_CLI -bin build/Project.bin -nk -t ssbl -hv 2.3 -o build/Project_sign.binYou can program the signed bin file at the address 0x70100000.
export DKEL="<STM32CubeProgrammer_N6 Install Folder>/bin/ExternalLoader/MX66UW1G45G_STM32N6570-DK.stldr"
# Adapt build path to your IDE
STM32_Programmer_CLI -c port=SWD mode=HOTPLUG -el $DKEL -hardRst -w build/Project_sign.bin 0x70100000Note: Only the application binary needs to be programmed if fsbl and network_data.hex have already been programmed.
Set your board to boot from flash mode and power cycle to boot from external flash.
- (VENC_IMX335_WIDTH * 4) must be a multiple of 16
- VENC_IMX335_WIDTH must be greater or equal to VENC_IMX335_HEIGHT
- VENC_VD66GY_WIDTH must be greater or equal to VENC_VD66GY_HEIGHT
- VENC_VD55G1_WIDTH must be greater or equal to VENC_VD55G1_HEIGHT
