Skip to content

Commit ee5c9c4

Browse files
committed
fix conflicts in Yolo Learning Path
2 parents 53923da + b783561 commit ee5c9c4

File tree

10 files changed

+391
-275
lines changed

10 files changed

+391
-275
lines changed

content/learning-paths/microcontrollers/yolo-on-himax/_index.md

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -3,33 +3,37 @@ title: Run a Computer Vision Model on a Himax Microcontroller
33

44
minutes_to_complete: 90
55

6-
who_is_this_for: This is an introduction topic for beginners on how to run a computervision application on an embedded device from Himax. This example uses an off-the-shelf Himax WiseEye2 module which is based on the Arm Cortex-M55 and Ethos-U55.
6+
who_is_this_for: This is an introduction topic for beginners on how to run a computer vision application on an embedded device from Himax. This example uses an off-the-shelf Himax WiseEye2 module which is based on the Arm Cortex-M55 and Ethos-U55.
7+
8+
learning_objectives:
9+
- Run a you-only-look-once (YOLO) object detection model on the edge device
10+
- Build the Himax Software Development Kit (SDK) and generate the firmware image file
11+
- Update the firmware on the edge device (Himax WiseEye2)
712

8-
learning_objectives:
9-
- Run a you-only-look-once (YOLO) computer vision model using off-the-shelf hardware based on the Arm Cortex-M55 and Ethos-U55.
10-
- Learn how to build the Himax SDK and generate firmware image file.
11-
- Learn how to update firmware on edge device (Himax WiseEye2).
12-
1313
prerequisites:
14-
- Seeed Studio [Grove Vision AI V2 Module](https://wiki.seeedstudio.com/grove_vision_ai_v2/)
15-
- OV5647-62 Camera module and included FPC cable
14+
- A [Seeed Grove Vision AI Module V2](https://www.seeedstudio.com/Grove-Vision-AI-Module-V2-p-5851.html) development board
15+
- A [OV5647-62 Camera Module](https://www.seeedstudio.com/OV5647-69-1-FOV-Camera-module-for-Raspberry-Pi-3B-4B-p-5484.html) and included FPC cable
1616
- A USB-C cable
17-
- A Linux/Windows-based PC on an x86 archiecture.
17+
- An x86 based Linux machine or a machine running Apple Silicon
1818

1919
author_primary: Chaodong Gong, Alex Su, Kieran Hejmadi
2020

2121
### Tags
22-
skilllevels: Beginner
22+
skilllevels: Introductory
2323
subjects: ML
2424
armips:
25-
- Cortex M55
26-
- Ethos U55
25+
- Cortex-M55
26+
- Ethos-U55
2727
tools_software_languages:
2828
- Himax SDK
29-
- Bash
29+
- Python
3030
operatingsystems:
3131
- Linux
32-
- Windows
32+
- macOS
33+
34+
draft: true
35+
cascade:
36+
draft: true
3337

3438

3539
### FIXED, DO NOT MODIFY
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: Build the firmware
3+
weight: 3
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
This section will walk you though the process of generating the firmware image file.
10+
11+
## Clone the Himax project
12+
13+
Himax has set up a repository containing a few examples for the Seeed Grove Vision AI V2 board. It contains third-party software and scripts to build and flash the image with the object detection application. By recursively cloning the Himax examples repo, git will include the necessary sub-repositories that have been configured for the project.
14+
15+
```bash
16+
git clone --recursive https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2.git
17+
cd Seeed_Grove_Vision_AI_Module_V2
18+
```
19+
20+
## Compile the firmware
21+
22+
For the object detection to activate, you need to edit the project's `makefile`, located in the `EPII_CM55M_APP_S` directory.
23+
24+
```bash
25+
cd EPII_CM55M_APP_S
26+
```
27+
28+
Use the `make` build tool to compile the source code. This should take up to 10 minutes depending on the number of CPU cores available on your host machine. The result is an `.elf` file written to the directory below.
29+
30+
```bash
31+
make clean
32+
make
33+
```
34+
35+
## Generate the firmware image
36+
37+
The examples repository contains scripts to generate the image file. Copy the `.elf` file to the `input_case1_secboot` directory.
38+
39+
```bash
40+
cd ../we2_image_gen_local/
41+
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
42+
```
43+
44+
Run the script corresponding to the OS of your host machine. This will create a file named `output.img` in the `output_case1_sec_wlcsp` directory.
45+
46+
47+
{{< tabpane code=true >}}
48+
{{< tab header="Linux" language="shell">}}
49+
./we2_local_image_gen project_case1_blp_wlcsp.json
50+
{{< /tab >}}
51+
{{< tab header="MacOS" language="shell">}}
52+
./we2_local_image_gen_macOS_arm64 project_case1_blp_wlcsp.json
53+
{{< /tab >}}
54+
{{< /tabpane >}}
55+
56+
Your terminal output should end with the following.
57+
58+
```output
59+
Output image: output_case1_sec_wlcsp/output.img
60+
Output image: output_case1_sec_wlcsp/output.img
61+
62+
IMAGE GEN DONE
63+
```
64+
65+
With this step, you are ready to flash the image onto the Himax development board.
Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
---
2+
title: Set up environment
3+
weight: 2
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
# Set up the development environment
10+
11+
This learning path has been validated on Ubuntu 22.04 LTS and macOS.
12+
13+
{{% notice %}}
14+
If you are running Windows on your host machine, you can use Ubuntu through Windows subsystem for Linux 2 (WSL2). Check out [this learning path](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) to get started.
15+
{{% /notice %}}
16+
17+
## Install Python, pip and git
18+
19+
You will use Python to build the firmware image and pip to install some dependencies. Verify Python is installed by running
20+
```bash
21+
python3 --version
22+
```
23+
24+
You should see an output like the following.
25+
```output
26+
Python 3.12.7
27+
```
28+
29+
Install `pip` and `venv` with the following commands.
30+
31+
```bash
32+
sudo apt update
33+
sudo apt install python3-pip python3-venv -y
34+
```
35+
36+
check the output to verify `pip` is installed correctly.
37+
```
38+
pip3 --version
39+
```
40+
41+
```output
42+
pip 24.2 from /<path-to>/pip (python 3.12)
43+
```
44+
45+
It is considered good practice to manage `pip` packages through a virtual environment. Create one with the steps below.
46+
47+
```bash
48+
python3 -m venv $HOME/yolo-venv
49+
source $HOME/yolo-venv/bin/activate
50+
```
51+
52+
Your terminal displays `(yolo-venv)` in the prompt indicating the virtual environment is active.
53+
54+
You will need to have the git version control system installed. Run the command below to verify that git is installed on your system.
55+
56+
```bash
57+
git --version
58+
```
59+
60+
You should see output similar to that below.
61+
62+
```output
63+
git version 2.39.3
64+
```
65+
66+
## Install make
67+
68+
Install the make build tool, which is used to build the firmware in the next section.
69+
70+
{{< tabpane code=true >}}
71+
{{< tab header="Linux" language="shell">}}
72+
sudo apt update
73+
sudo apt install make -y
74+
{{< /tab >}}
75+
{{< tab header="MacOS" language="shell">}}
76+
brew install make
77+
{{< /tab >}}
78+
{{< /tabpane >}}
79+
80+
Successful installation of make will show the following when the `make --version` command is run.
81+
82+
```output
83+
$ make --version
84+
GNU Make 4.3
85+
Built for x86_64-pc-linux-gnu
86+
Copyright (C) 1988-2020 Free Software Foundation, Inc.
87+
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
88+
This is free software: you are free to change and redistribute it.
89+
There is NO WARRANTY, to the extent permitted by law.
90+
```
91+
{{% notice Note %}}
92+
To run this learning path on macOS, you need to verify that your installation is for the GNU Make - not the BSD version.
93+
{{% /notice %}}
94+
## Install Arm GNU toolchain
95+
96+
The toolchain is used to compile code from the host to the embedded device architecture.
97+
98+
{{< tabpane code=true >}}
99+
{{< tab header="Linux" language="shell">}}
100+
cd $HOME
101+
wget https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
102+
tar -xvf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
103+
export PATH="$HOME/arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/:$PATH"
104+
{{< /tab >}}
105+
{{< tab header="MacOS" language="shell">}}
106+
cd $HOME
107+
wget https://developer.arm.com/-/media/Files/downloads/gnu/13.3.rel1/binrel/arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.tar.xz
108+
tar -xvf arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.tar.xz
109+
export PATH="$HOME/arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi/bin/:$PATH"
110+
{{< /tab >}}
111+
{{< /tabpane >}}
112+
113+
{{% notice %}}
114+
You can add the `export` command to the `.bashrc` file. This was, the Arm GNU toolchain is configured from new terminal sessions as well.
115+
{{% /notice %}}
116+
117+
118+
Now that your development environment is set up, move on to the next section where you will generate the firmware image.
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
---
2+
title: Flash firmware onto the microcontroller
3+
weight: 4
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
Now that you have generated an image file on the local host machine, you are ready to flash the microcontroller with this firmware.
10+
11+
## Install xmodem
12+
13+
`Xmodem` is a basic file transfer protocol which is easily installed using the Himax examples repository. Run the following command to install the dependency. If you cloned the repository to a different location, replace $HOME with the path.
14+
15+
```bash
16+
cd $HOME/Seeed_Grove_Vision_AI_Module_V2
17+
pip install -r xmodem/requirements.txt
18+
```
19+
20+
## Connect the module
21+
22+
To prepare for the next steps, it's time to get the board set up. Insert the Flexible printed circuit (FPC) into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below.
23+
24+
![unlatched](./unlatched.jpg)
25+
26+
Then, slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector.
27+
28+
![latched](./latched.jpg)
29+
30+
Now you can connect the Groove Vision AI V2 Module to your computer via the USB-C cable.
31+
32+
{{% notice Note %}}
33+
The development board may have two USB-C connectors. If you are running into issues connecting the board in the next step, make sure you are using the right one.
34+
{{% /notice %}}
35+
36+
## Find the COM port
37+
38+
You'll need to provide the communication port (COM) which the board is connected to in order to flash the image. There are commands to list all COMs available on your machine. Once your board is connected through USB, it'll show up in this list. The COM identifier will start with **tty**, which may help you determine which one it is. You can run the command before and after plugging in the board if you are unsure.
39+
40+
41+
{{< tabpane code=true >}}
42+
{{< tab header="Linux" language="shell">}}
43+
sudo grep -i 'tty' /var/log/dmesg
44+
{{< /tab >}}
45+
{{< tab header="MacOS" language="shell">}}
46+
ls /dev/tty.*
47+
{{< /tab >}}
48+
{{< /tabpane >}}
49+
50+
51+
{{% notice Note %}}
52+
If the port seems unavailable, try changing the permissions temporarily using the `chmod` command. Be sure to reset them afterwards, as this may pose a computer security vulnerability.
53+
54+
```bash
55+
chmod 0777 <COM port>
56+
```
57+
{{% /notice %}}
58+
59+
The full path to the port is needed in the next step, so be sure to note it down.
60+
61+
## Flash the firmware onto the module
62+
63+
Run the python script below to flash the firmware.
64+
65+
```bash
66+
python xmodem\xmodem_send.py --port=<COM port> \
67+
--baudrate=921600 --protocol=xmodem \
68+
--file=we2_image_gen_local\output_case1_sec_wlcsp\output.img
69+
```
70+
71+
{{% notice Note %}}
72+
When you run other example models demonstrated in the later section [Run additional models in the web toolkit](/learning-paths/microcontrollers/yolo-on-himax/web-toolkit/), you need to adapt this command with `--model` argument.
73+
{{% /notice %}}
74+
75+
After the firmware image burning is completed, the message `Do you want to end file transmission and reboot system? (y)` is displayed. Press the reset button indicated in the image below.
76+
77+
![reset button](./reset_button.jpg)
78+
79+
## Run the model
80+
81+
After the reset button is pressed, the board will start inference with the object detection automatically. Observe the output in the terminal to verify that the image is built correctly. If a person is in front of the camera, you should see the `person_score` value go over `100`.
82+
83+
```output
84+
b'SENSORDPLIB_STATUS_XDMA_FRAME_READY 240'
85+
b'write frame result 0, data size=15284,addr=0x340e04e0'
86+
b'invoke pass'
87+
b'person_score:113'
88+
b'EVT event = 10'
89+
b'SENSORDPLIB_STATUS_XDMA_FRAME_READY 241'
90+
b'write frame result 0, data size=15296,addr=0x340e04e0'
91+
b'invoke pass'
92+
b'person_score:112'
93+
b'EVT event = 10'
94+
```
95+
96+
This means the image works correctly on the device, and the end-to-end flow is complete.

0 commit comments

Comments
 (0)