Skip to content

Commit ab47648

Browse files
pareenavermapareenaverma
authored
Merge pull request #1369 from kieranhejmadi01/main
Run a YOLOv8 model on a Himax Microcontroller
2 parents 9920c9c + 36bfaf3 commit ab47648

File tree

15 files changed

+343
-0
lines changed

15 files changed

+343
-0
lines changed

‎content/learning-paths/microcontrollers/yolo-on-himax/_index.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
title: Run a Computer Vision Model on a Himax Microcontroller
3+
4+
minutes_to_complete: 90
5+
6+
who_is_this_for: This is an introduction topic for beginners on how to run a computervision application on an embedded device from Himax. This example uses an off-the-shelf Himax WiseEye2 module which is based on the Arm Cortex-M55 and Ethos-U55.
7+
8+
learning_objectives:
9+
- Run a you-only-look-once (YOLO) computer vision model using off-the-shelf hardware based on the Arm Cortex-M55 and Ethos-U55.
10+
- Learn how to build the Himax SDK and generate firmware image file.
11+
- Learn how to update firmware on edge device (Himax WiseEye2).
12+
13+
prerequisites:
14+
- Seeed Grove Vision AI V2 Module
15+
- OV5647-62 Camera module and included FPC cable
16+
- A USB-C cable
17+
- A Linux/Windows-based PC on an x86 archiecture.
18+
19+
author_primary: Chaodong Gong, Alex Su, Kieran Hejmadi
20+
21+
### Tags
22+
skilllevels: Beginner
23+
subjects: ML
24+
armips:
25+
- Cortex M55
26+
- Ethos U55
27+
tools_software_languages:
28+
- Himax SDK
29+
- Bash
30+
operatingsystems:
31+
- Linux
32+
- Windows
33+
34+
35+
### FIXED, DO NOT MODIFY
36+
# ================================================================================
37+
weight: 1 # _index.md always has weight of 1 to order correctly
38+
layout: "learningpathall" # All files under learning paths have this same wrapper
39+
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
40+
---

‎content/learning-paths/microcontrollers/yolo-on-himax/_next-steps.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
next_step_guidance: Navigate Machine Learning Development with Ethos-U processors
3+
4+
recommended_path: /learning-paths/microcontrollers/nav-mlek/
5+
6+
further_reading:
7+
- resource:
8+
title: Grove Vision AI Module V2 User Documentation
9+
link: https://wiki.seeedstudio.com/grove_vision_ai_v2/
10+
type: documentation
11+
- resource:
12+
title: WiseEye2 HX6538 processor blog (SoC powering Grove Vision AI Module V2)
13+
link: https://www.himax.com.tw/products/wiseeye-ai-sensing/wiseeye2-ai-processor/
14+
type: blog
15+
16+
# ================================================================================
17+
# FIXED, DO NOT MODIFY
18+
# ================================================================================
19+
weight: 21 # set to always be larger than the content in this path, and one more than 'review'
20+
title: "Next Steps" # Always the same
21+
layout: "learningpathall" # All files under learning paths have this same wrapper
22+
---

‎content/learning-paths/microcontrollers/yolo-on-himax/_review.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
review:
3+
- questions:
4+
question: >
5+
The Grove Vision AI V2 Module can run Yolov8 model in real time?
6+
answers:
7+
- True
8+
- False
9+
correct_answer: 1
10+
explanation: >
11+
The Grove Vision AI V2 Module can run object detection in real time using the Cortex-M55 and Ethos-U55.
12+
13+
14+
# ================================================================================
15+
# FIXED, DO NOT MODIFY
16+
# ================================================================================
17+
title: "Review" # Always the same title
18+
weight: 20 # Set to always be larger than the content in this path
19+
layout: "learningpathall" # All files under learning paths have this same wrapper
20+
---

‎content/learning-paths/microcontrollers/yolo-on-himax/face_detection.jpg‎

27.4 KB
Loading

‎content/learning-paths/microcontrollers/yolo-on-himax/himax_web_ui.jpg‎

26.8 KB
Loading

‎content/learning-paths/microcontrollers/yolo-on-himax/how-to-1.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
---
2+
title: Set Up Environment
3+
weight: 2
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Set up the Development Environment
10+
11+
### Step 1.1. Install Ubuntu
12+
13+
If you are running Windows on your host machine, we recommend using Ubuntu through Windows subsystem for Linux 2 (WSL2). Please see [this learning path](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) for assistance
14+
15+
This learning path has been validated on Ubuntu 22.04 LTS. However, we expect other linux distributions to work. To verify the Linux distribution you are using you can run the `cat /etc/*release*` command.
16+
17+
```bash
18+
cat /etc/*release*
19+
```
20+
The top lines from the terminal output will show the distribution version.
21+
22+
```output
23+
DISTRIB_ID=Ubuntu
24+
DISTRIB_RELEASE=22.04
25+
DISTRIB_CODENAME=jammy
26+
DISTRIB_DESCRIPTION="Ubuntu 22.04.5 LTS"
27+
...
28+
```
29+
30+
### Step 1.2. (Optional) Install Microsoft Visual Studio Code
31+
32+
This is only optional. You can use any text editor you are comfortable with to view or edit code. By typing “wsl” in VS Code terminal, you can switch to Linux environment.
33+
34+
### Step 1.3. Install python 3
35+
36+
Go to website python.org to download and install.
37+
Verify python is installed by
38+
python3 --version
39+
You should see an output like the following.
40+
```output
41+
Python 3.12.7
42+
```
43+
### Step 1.4. Install python-pip
44+
45+
```bash
46+
sudo apt update
47+
sudo apt install python3-pip -y
48+
pip3 --version
49+
```
50+
51+
If `pip3` is correctly installed you should see an output similar to tht following.
52+
53+
```output
54+
pip 24.2 from <path to pip3>/pip (python 3.12)
55+
```
56+
57+
### Step 1.5. Install make
58+
59+
You will need to install the make build tool in order to build the firmware in the following section.
60+
61+
```bash
62+
sudo apt update
63+
sudo apt install make -y
64+
```
65+
66+
Successful installation of make will show the following when the `make --version` command is run.
67+
68+
```output
69+
$ make --version
70+
GNU Make 4.3
71+
Built for x86_64-pc-linux-gnu
72+
Copyright (C) 1988-2020 Free Software Foundation, Inc.
73+
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
74+
This is free software: you are free to change and redistribute it.
75+
There is NO WARRANTY, to the extent permitted by law.
76+
```
77+
78+
### Step 1.6. Install ARM GNU toolchain
79+
80+
```bash
81+
cd ~
82+
wget https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
83+
tar -xvf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
84+
export PATH="$HOME/arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/:$PATH"
85+
```
86+
87+
Please note: you may want to add the command to your `bashrc` file. This enables the Arm GNU toolchain to be easily accessed from any new terminal session.
88+

‎content/learning-paths/microcontrollers/yolo-on-himax/how-to-2.md‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
---
2+
title: Build The Firmware
3+
weight: 3
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Build The Firmware
10+
11+
Next, we need to build an image that contains the embedded software (firmware). You will need to have the git version control system installed. Run the command below to verify that git is installed on your system.
12+
13+
```bash
14+
git --version
15+
```
16+
17+
You should see output similar to that below.
18+
19+
```output
20+
git version 2.39.3
21+
```
22+
23+
If not, please follow the steps to install git on your system.
24+
25+
### Step 2.1. Clone the Himax project
26+
27+
You will first need to recusively clone the Himax repository. This will also clone the necessary sub repos such as Arm CMSIS.
28+
29+
```bash
30+
git clone --recursive https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2.git
31+
cd Seeed_Grove_Vision_AI_Module_V2
32+
```
33+
34+
### Step 2.2. Compile the Firmware
35+
36+
The make build tool is used to compile the source code. This should take up around 2-3 minutes depending on the number of CPU cores available.
37+
38+
```bash
39+
cd EPII_CM55M_APP_S
40+
make clean
41+
make
42+
```
43+
44+
45+
### Step 2.3. Generate a Firmware Image
46+
47+
```bash
48+
cd ../we2_image_gen_local/
49+
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
50+
./we2_local_image_gen project_case1_blp_wlcsp.json
51+
```
52+
53+
Your terminal output should end with the following.
54+
55+
```output
56+
Output image: output_case1_sec_wlcsp/output.img
57+
Output image: output_case1_sec_wlcsp/output.img
58+
59+
IMAGE GEN DONE
60+
```

‎content/learning-paths/microcontrollers/yolo-on-himax/how-to-3.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
title: Flash Firmware onto the Microcontroller
3+
weight: 3
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Flash the Firmware
10+
11+
Now that we have generated a firmware file on our local machine, we need to flash the microcontroller with this firmware.
12+
13+
### Step 3.1. Install xmodem.
14+
15+
`Xmodem` is a basic file transfer protocol. Run the following command to install the dependencies for xmodem.
16+
17+
```bash
18+
cd $HOME/Seeed_Grove_Vision_AI_Module_V2 # If you cloned the repo to a different location replace $HOME with the path.
19+
pip install -r xmodem/requirements.txt
20+
```
21+
22+
### Step 3.2. Connect the module to PC by USB cable.
23+
24+
You will need to insert the FPC cable cable into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below.
25+
26+
![unlatched](./unlatched.jpg)
27+
28+
Then, slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector.
29+
30+
![latched](./latched.jpg)
31+
32+
Then connect the Groove Vision AI V2 Module to your computer via the USB-C cable.
33+
34+
### Step 3.4. Flash the firmware onto the moule.
35+
36+
Run the python script below to flash the firmware.
37+
38+
```python
39+
python xmodem\xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img
40+
```
41+
42+
Note: If running one of the other example models demonstrated in '(Optional) Try Different Models', the command might be slightly different.
43+
44+
After the firmware image burning is completed, the message "Do you want to end file transmission and reboot system? (y)" is displayed. Press the reset button on the module as per the image below.
45+
46+
![reset button](./reset_button.jpg)

‎content/learning-paths/microcontrollers/yolo-on-himax/how-to-4.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
title: Run and View Model Results
3+
weight: 3
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
10+
### Step 4.1. Connect module to PC with USB cable.
11+
12+
Exit the terminal session and connect the module to the PC via your USB-C cable.
13+
14+
### Step 4.2. Download the Himax AI web toolkit.
15+
16+
The Himax AI web toolkit enables a browser-based graphical user interface (GUI) for the live camera feed.
17+
18+
Download the Himax AI Web toolkit by clicking on this [link](https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2/releases/download/v1.1/Himax_AI_web_toolkit.zip)
19+
20+
Unzip the archived file and double click `index.html`. This will open the GUI within your default browser.
21+
22+
### Step 4.3. Connect to the Grove Vision AI
23+
24+
Select 'Grove Vision AI(V2)' in the top-right hand corner and press connect button.
25+
26+
![Himax web UI](./himax_web_ui.jpg)

‎content/learning-paths/microcontrollers/yolo-on-himax/how-to-5.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
title: (Optional) Try Different Models
3+
weight: 5
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
10+
### Modify the makefile
11+
12+
Change the directory to the where the makefile is located.
13+
14+
```bash
15+
cd $HOME/Seeed_Grove_Vision_AI_Module_V2/EPII_CM55M_APP_S/ # replace $HOME with the location of the project
16+
```
17+
18+
Using a text editor, for example visual studio code or nano, modify the `APP_TYPE` field in the makefile from the default value of `allon_sensor_tflm` to one of the values in the table below
19+
20+
21+
|APP_TYPE =|Description|
22+
|---|---|
23+
|tflm_folov8_od|Object detection|
24+
|tflm_folov8_pose|Pose detection|
25+
|tflm_fd_fm|Face detection|
26+
27+
### Regenerate the Firmware Image
28+
29+
Go back to the 'Build The Firmware' section and start from Step 3.2. to regenerate the firmware image.
30+
31+
The images below are examples images from the model.
32+
33+
#### Objection Detection
34+
![object_detection](./object_detection.jpg)
35+
36+
#### Pose Estimation
37+
![Pose estimation](./pose_estimation.jpg)
38+
39+
#### Face Detection
40+
![object_detection](./face_detection.jpg)
41+

0 commit comments

Comments
 (0)