Skip to content

Commit 833712e

Browse files
KBOUSTMKBOUSTM
committed
Update: Added new YOLO X models and updated the README of the object detection use case
Signed-off-by: khaoula boutiche <[email protected]>
1 parent 6f10965 commit 833712e

File tree

13 files changed

+440
-4
lines changed

13 files changed

+440
-4
lines changed

‎manifest.json

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -432,6 +432,14 @@
432432
"readme": "./object_detection/st_yolo_x/README.md",
433433
"visible": true
434434
},
435+
{
436+
"name": "./object_detection/st_yolo_x/ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_int8.tflite",
437+
"description": "ST yolo X (derived from YoloX) trained on COCO 2017 (Person Detection) with 1 class, dm=1.0, wm=0.25, 3 anchors",
438+
"usecase": "object_detection",
439+
"dataset_used": "COCO 2017 Person",
440+
"readme": "./object_detection/st_yolo_x/README.md",
441+
"visible": true
442+
},
435443
{
436444
"name": "./pose_estimation/movenet/Public_pretrainedmodel_custom_dataset/custom_dataset_person_17kpts/movenet_lightning_heatmaps_192/movenet_lightning_heatmaps_192_int8_pc.tflite",
437445
"description": "Movenet variant trained on COCO 2017 (Person) with 1 class",

‎object_detection/LICENSE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
| tiny_yolo_v2/ST_pretrainedmodel_public_dataset | [SLA0044](./tiny_yolo_v2/ST_pretrainedmodel_public_dataset/LICENSE.md) | STMicroelectronics |
77
| st_yolo_lc_v1/ST_pretrainedmodel_public_dataset | [SLA0044](./st_yolo_lc_v1/ST_pretrainedmodel_public_dataset/LICENSE.md) | STMicroelectronics |
88
| st_yolo_x/ST_pretrainedmodel_public_dataset | [SLA0044](./st_yolo_x/ST_pretrainedmodel_public_dataset/LICENSE.md) | STMicroelectronics |
9+
| st_yolo_x/ST_pretrainedmodel_custom_dataset | [SLA0044](./st_yolo_x/ST_pretrainedmodel_custom_dataset/LICENSE.md) | STMicroelectronics |
910
| st_ssd_mobilenet_v1/ST_pretrainedmodel_public_dataset | [SLA0044](./st_ssd_mobilenet_v1/ST_pretrainedmodel_public_dataset/LICENSE.md) | STMicroelectronics |
1011
| ssd_mobilenet_v2_fpnlite/ST_pretrainedmodel_public_dataset | [SLA0044](./ssd_mobilenet_v2_fpnlite/ST_pretrainedmodel_public_dataset/LICENSE.md) | STMicroelectronics |
1112
| yolo_v8n | [SLA0044](./yolo_v8n/LICENSE.md) | STMicroelectronics |

‎object_detection/st_yolo_x/README.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,8 @@ This is an optimized ST version of the well known yolo x, quantized in int8 form
1414
|-------------------------|-----------------|
1515
| Framework | TensorFlow Lite |
1616
| Quantization | int8 |
17-
| Provenance | TO DO |
18-
| Paper | TO DO |
17+
| Provenance | |
18+
| Paper | |
1919

2020

2121

@@ -29,7 +29,7 @@ For an image resolution of NxM and NC classes
2929

3030
| Output Shape | Description |
3131
| ----- | ----------- |
32-
| TO DO |
32+
| | |
3333

3434

3535
## Recommended Platforms
@@ -59,6 +59,7 @@ Measures are done with default STM32Cube.AI configuration with enabled input / o
5959
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_256/st_yolo_x_nano_256_0.5_0.4_int8.tflite) | COCO-Person | Int8 | 256x256x3 | STM32N6 | 971.62 | 0.0 | 2547.17 | 10.0.0 | 2.0.0 |
6060
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_320/st_yolo_x_nano_320_0.33_0.25_int8.tflite) | COCO-Person | Int8 | 320x320x3 | STM32N6 | 968.5 | 0.0 | 1028.08 | 10.0.0 | 2.0.0 |
6161
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_416/st_yolo_x_nano_416_0.33_0.25_int8.tflite) | COCO-Person | Int8 | 416x416x3 | STM32N6 | 2640.62 | 0.0 | 1027.89 | 10.0.0 | 2.0.0 |
62+
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_int8.tflite) | COCO-Person | Int8 | 480x480x3 | STM32N6 | 2418.75 | 0.0 | 1383.56 | 10.0.0 | 2.0.0 |
6263

6364
### Reference **NPU** inference time based on COCO Person dataset (see Accuracy for details on dataset)
6465
| Model | Dataset | Format | Resolution | Board | Execution Engine | Inference time (ms) | Inf / sec | STM32Cube.AI version | STEdgeAI Core version |
@@ -69,6 +70,8 @@ Measures are done with default STM32Cube.AI configuration with enabled input / o
6970
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_320/st_yolo_x_nano_320_0.33_0.25_int8.tflite) | COCO-Person | Int8 | 320x320x3 | STM32N6570-DK | NPU/MCU | 11.59 | 86.29 | 10.0.0 | 2.0.0 |
7071
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_416/st_yolo_x_nano_416_0.33_0.25_int8.tflite) | COCO-Person | Int8 | 416x416x3 |
7172
STM32N6570-DK | NPU/MCU | 17.99 | 55.59 | 10.0.0 | 2.0.0 |
73+
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_int8.tflite) | COCO-Person | Int8 | 480x480x3 |
74+
STM32N6570-DK | NPU/MCU | 32.4 | 30.8 | 10.0.0 | 2.0.0 |
7275

7376
### Reference **MCU** memory footprint based on COCO Person dataset (see Accuracy for details on dataset)
7477

@@ -94,7 +97,6 @@ STM32N6570-DK | NPU/MCU | 17.99 | 55.59 | 10.0.0
9497

9598
### AP on COCO Person dataset
9699

97-
98100
Dataset details: [link](https://cocodataset.org/#download) , License [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/legalcode) , Quotation[[1]](#1) , Number of classes: 80, Number of images: 118,287
99101

100102
| Model | Format | Resolution | AP |
@@ -109,6 +111,9 @@ Dataset details: [link](https://cocodataset.org/#download) , License [CC BY 4.0]
109111
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_320/st_yolo_x_nano_320_0.33_0.25.h5) | Float | 320x320x3 | 57.1 % |
110112
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_416/st_yolo_x_nano_416_0.33_0.25_int8.tflite) | Int8 | 416x416x3 | 62.2 % |
111113
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_416/st_yolo_x_nano_416_0.33_0.25.h5) | Float | 416x416x3 | 62.5 % |
114+
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_int8.tflite) | Int8 | 480x480x3 | 67.9 % |
115+
| [st_yolo_x_nano](ST_pretrainedmodel_public_dataset/coco_2017_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3.h5) | Float | 480x480x3 | 68.4 % |
116+
112117

113118
\* EVAL_IOU = 0.4, NMS_THRESH = 0.5, SCORE_THRESH =0.001
114119

‎object_detection/st_yolo_x/ST_pretrainedmodel_custom_dataset/LICENSE.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
This software component is provided to you as part of a software package and
2+
applicable license terms are in the Package_license file. If you received this
3+
software component outside of a package or without applicable license terms,
4+
the terms of the SLA0044 license shall apply and are fully reproduced below:
5+
6+
SLA0044 Rev5/February 2018
7+
8+
Software license agreement
9+
10+
ULTIMATE LIBERTY SOFTWARE LICENSE AGREEMENT
11+
12+
BY INSTALLING, COPYING, DOWNLOADING, ACCESSING OR OTHERWISE USING THIS SOFTWARE
13+
OR ANY PART THEREOF (AND THE RELATED DOCUMENTATION) FROM STMICROELECTRONICS
14+
INTERNATIONAL N.V, SWISS BRANCH AND/OR ITS AFFILIATED COMPANIES
15+
(STMICROELECTRONICS), THE RECIPIENT, ON BEHALF OF HIMSELF OR HERSELF, OR ON
16+
BEHALF OF ANY ENTITY BY WHICH SUCH RECIPIENT IS EMPLOYED AND/OR ENGAGED AGREES
17+
TO BE BOUND BY THIS SOFTWARE LICENSE AGREEMENT.
18+
19+
Under STMicroelectronics’ intellectual property rights, the redistribution,
20+
reproduction and use in source and binary forms of the software or any part
21+
thereof, with or without modification, are permitted provided that the following
22+
conditions are met:
23+
24+
1. Redistribution of source code (modified or not) must retain any copyright
25+
notice, this list of conditions and the disclaimer set forth below as items 10
26+
and 11.
27+
28+
2. Redistributions in binary form, except as embedded into microcontroller or
29+
microprocessor device manufactured by or for STMicroelectronics or a software
30+
update for such device, must reproduce any copyright notice provided with the
31+
binary code, this list of conditions, and the disclaimer set forth below as
32+
items 10 and 11, in documentation and/or other materials provided with the
33+
distribution.
34+
35+
3. Neither the name of STMicroelectronics nor the names of other contributors to
36+
this software may be used to endorse or promote products derived from this
37+
software or part thereof without specific written permission.
38+
39+
4. This software or any part thereof, including modifications and/or derivative
40+
works of this software, must be used and execute solely and exclusively on or in
41+
combination with a microcontroller or microprocessor device manufactured by or
42+
for STMicroelectronics.
43+
44+
5. No use, reproduction or redistribution of this software partially or totally
45+
may be done in any manner that would subject this software to any Open Source
46+
Terms. “Open Source Terms” shall mean any open source license which requires as
47+
part of distribution of software that the source code of such software is
48+
distributed therewith or otherwise made available, or open source license that
49+
substantially complies with the Open Source definition specified at
50+
www.opensource.org and any other comparable open source license such as for
51+
example GNU General Public License (GPL), Eclipse Public License (EPL), Apache
52+
Software License, BSD license or MIT license.
53+
54+
6. STMicroelectronics has no obligation to provide any maintenance, support or
55+
updates for the software.
56+
57+
7. The software is and will remain the exclusive property of STMicroelectronics
58+
and its licensors. The recipient will not take any action that jeopardizes
59+
STMicroelectronics and its licensors' proprietary rights or acquire any rights
60+
in the software, except the limited rights specified hereunder.
61+
62+
8. The recipient shall comply with all applicable laws and regulations affecting
63+
the use of the software or any part thereof including any applicable export
64+
control law or regulation.
65+
66+
9. Redistribution and use of this software or any part thereof other than as
67+
permitted under this license is void and will automatically terminate your
68+
rights under this license.
69+
70+
10. THIS SOFTWARE IS PROVIDED BY STMICROELECTRONICS AND CONTRIBUTORS "AS IS" AND
71+
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
72+
IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
73+
NON-INFRINGEMENT OF THIRD PARTY INTELLECTUAL PROPERTY RIGHTS, WHICH ARE
74+
DISCLAIMED TO THE FULLEST EXTENT PERMITTED BY LAW. IN NO EVENT SHALL
75+
STMICROELECTRONICS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
76+
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
77+
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
78+
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
79+
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
80+
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
81+
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
82+
83+
11. EXCEPT AS EXPRESSLY PERMITTED HEREUNDER, NO LICENSE OR OTHER RIGHTS, WHETHER
84+
EXPRESS OR IMPLIED, ARE GRANTED UNDER ANY PATENT OR OTHER INTELLECTUAL PROPERTY
85+
RIGHTS OF STMICROELECTRONICS OR ANY THIRD PARTY.
86+

‎object_detection/st_yolo_x/ST_pretrainedmodel_custom_dataset/st_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3.h5

6.08 MB
Binary file not shown.

‎object_detection/st_yolo_x/ST_pretrainedmodel_custom_dataset/st_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_config.yaml

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
general:
2+
project_name: st_yolo_x_480_1.0_0.25_3anchors_st
3+
model_path:
4+
model_type: st_yolo_x
5+
logs_dir: logs
6+
saved_models_dir: saved_models
7+
display_figures: false
8+
gpu_memory_limit: 24 #24
9+
num_threads_tflite: 24
10+
global_seed: 127
11+
12+
operation_mode: chain_tqeb
13+
14+
dataset:
15+
name: st_person
16+
class_names: [person]
17+
training_path: /local/data/od_st_person/train
18+
validation_path: /local/data/od_st_person/test
19+
validation_split: #0.1
20+
test_path: /local/data/od_st_person/test
21+
quantization_path: /local/data/od_st_person/train
22+
quantization_split: 0.001
23+
24+
preprocessing:
25+
rescaling:
26+
scale: 1/255
27+
offset: 0
28+
resizing:
29+
aspect_ratio: fit
30+
interpolation: nearest
31+
color_mode: rgb
32+
33+
data_augmentation:
34+
random_contrast:
35+
factor: 0.4
36+
random_brightness:
37+
factor: 0.3
38+
random_flip:
39+
mode: horizontal
40+
random_translation:
41+
width_factor: 0.15
42+
height_factor: 0.15
43+
fill_mode: reflect
44+
interpolation: nearest
45+
random_rotation:
46+
factor: 0.02
47+
fill_mode: reflect
48+
interpolation: nearest
49+
random_crop:
50+
crop_center_x: (0.25, 0.75)
51+
crop_center_y: (0.25, 0.75)
52+
crop_width: (0.6, 0.9)
53+
crop_height: (0.6, 0.9)
54+
change_rate: 0.9
55+
56+
training:
57+
model:
58+
input_shape: (480, 480, 3)
59+
depth_mul: 1.0
60+
width_mul: 0.25
61+
dropout: null
62+
batch_size: 64
63+
epochs: 550
64+
optimizer:
65+
Adam:
66+
learning_rate: 0.0025
67+
callbacks:
68+
LRWarmupCosineDecay:
69+
initial_lr: 1.0e-05
70+
warmup_steps: 20
71+
max_lr: 0.00125
72+
hold_steps: 20
73+
decay_steps: 500
74+
end_lr: 1.0e-06
75+
EarlyStopping:
76+
monitor: val_map
77+
patience: 60
78+
restore_best_weights: true
79+
verbose: 1
80+
81+
postprocessing:
82+
yolo_anchors: [0.5, 0.5, 0.07, 0.25, 0.23, 0.7]
83+
confidence_thresh: 0.001
84+
NMS_thresh: 0.5
85+
IoU_eval_thresh: 0.5
86+
plot_metrics: false
87+
max_detection_boxes: 100
88+
89+
quantization:
90+
quantizer: TFlite_converter
91+
quantization_type: PTQ
92+
quantization_input_type: uint8
93+
quantization_output_type: float
94+
export_dir: quantized_models
95+
96+
tools:
97+
stedgeai:
98+
version: 10.0.0
99+
optimization: balanced
100+
on_cloud: True
101+
path_to_stedgeai: C:/Users/<XXXXX>/STM32Cube/Repository/Packs/STMicroelectronics/X-CUBE-AI/<*.*.*>/Utilities/windows/stedgeai.exe
102+
path_to_cubeIDE: C:/ST/STM32CubeIDE_<*.*.*>/STM32CubeIDE/stm32cubeide.exe
103+
104+
benchmarking:
105+
board: STM32N6570-DK
106+
107+
mlflow:
108+
uri: ./experiments_outputs/mlruns
109+
110+
hydra:
111+
run:
112+
dir: ./experiments_outputs/st_yolox_480_1.0_0.25_3anchors_st

‎object_detection/st_yolo_x/ST_pretrainedmodel_custom_dataset/st_person/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_int8.tflite

1.53 MB
Binary file not shown.

‎object_detection/st_yolo_x/ST_pretrainedmodel_public_dataset/coco_2017_80_classes/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3.h5

6.25 MB
Binary file not shown.

‎object_detection/st_yolo_x/ST_pretrainedmodel_public_dataset/coco_2017_80_classes/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_config.yaml

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
general:
2+
project_name: st_yolo_x_480_1.0_0.25_3anchors_fullcoco
3+
model_path:
4+
model_type: st_yolo_x
5+
logs_dir: logs
6+
saved_models_dir: saved_models
7+
display_figures: false
8+
gpu_memory_limit: 24 #24
9+
num_threads_tflite: 24
10+
global_seed: 127
11+
12+
operation_mode: chain_tqeb
13+
14+
dataset:
15+
name: coco_person
16+
class_names: ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush']
17+
training_path: /local/data/od_coco_2017/train
18+
validation_path: /local/data/od_coco_2017/test
19+
validation_split: #0.1
20+
test_path: /local/data/od_coco_2017/test
21+
quantization_path: /local/data/od_coco_2017/train
22+
quantization_split: 0.001
23+
24+
preprocessing:
25+
rescaling:
26+
scale: 1/255
27+
offset: 0
28+
resizing:
29+
aspect_ratio: fit
30+
interpolation: nearest
31+
color_mode: rgb
32+
33+
data_augmentation:
34+
random_contrast:
35+
factor: 0.4
36+
random_brightness:
37+
factor: 0.3
38+
random_flip:
39+
mode: horizontal
40+
random_translation:
41+
width_factor: 0.15
42+
height_factor: 0.15
43+
fill_mode: reflect
44+
interpolation: nearest
45+
random_rotation:
46+
factor: 0.02
47+
fill_mode: reflect
48+
interpolation: nearest
49+
random_crop:
50+
crop_center_x: (0.25, 0.75)
51+
crop_center_y: (0.25, 0.75)
52+
crop_width: (0.6, 0.9)
53+
crop_height: (0.6, 0.9)
54+
change_rate: 0.9
55+
56+
training:
57+
model:
58+
input_shape: (480, 480, 3)
59+
depth_mul: 1.0
60+
width_mul: 0.25
61+
dropout: null
62+
batch_size: 64
63+
epochs: 550
64+
optimizer:
65+
Adam:
66+
learning_rate: 0.0025
67+
callbacks:
68+
LRWarmupCosineDecay:
69+
initial_lr: 1.0e-05
70+
warmup_steps: 20
71+
max_lr: 0.00125
72+
hold_steps: 20
73+
decay_steps: 500
74+
end_lr: 1.0e-06
75+
EarlyStopping:
76+
monitor: val_loss
77+
patience: 60
78+
restore_best_weights: true
79+
verbose: 1
80+
81+
postprocessing:
82+
yolo_anchors: [0.5, 0.5, 0.07, 0.25, 0.23, 0.7]
83+
confidence_thresh: 0.001
84+
NMS_thresh: 0.5
85+
IoU_eval_thresh: 0.5
86+
plot_metrics: false
87+
max_detection_boxes: 100
88+
89+
quantization:
90+
quantizer: TFlite_converter
91+
quantization_type: PTQ
92+
quantization_input_type: uint8
93+
quantization_output_type: float
94+
export_dir: quantized_models
95+
96+
tools:
97+
stedgeai:
98+
version: 10.0.0
99+
optimization: balanced
100+
on_cloud: True
101+
path_to_stedgeai: C:/Users/<XXXXX>/STM32Cube/Repository/Packs/STMicroelectronics/X-CUBE-AI/<*.*.*>/Utilities/windows/stedgeai.exe
102+
path_to_cubeIDE: C:/ST/STM32CubeIDE_<*.*.*>/STM32CubeIDE/stm32cubeide.exe
103+
104+
benchmarking:
105+
board: STM32N6570-DK
106+
107+
mlflow:
108+
uri: ./experiments_outputs/mlruns
109+
110+
hydra:
111+
run:
112+
dir: ./experiments_outputs/st_yolox_480_1.0_0.25_3anchors_fullcoco

‎object_detection/st_yolo_x/ST_pretrainedmodel_public_dataset/coco_2017_80_classes/st_yolo_x_nano_480/st_yolo_x_nano_480_1.0_0.25_3_int8.tflite

1.59 MB
Binary file not shown.

0 commit comments

Comments
 (0)