Updates:

- Minor bug fix for the Object Detection (OD) use case preprocessing.
- Updated dataset naming of the Semantic Segmentation (SS) use case in YAML files and READMEs.


Signed-off-by: khaoula boutiche <[email protected]>

Author: KBOUSTM

human_activity_recognition/src/user_config.yaml

@@ -60,12 +60,12 @@ training:
 #  trained_model_path: trained.h5   # Optional, use it if you want to save the best model at the end of the training to a path of your choice
 
 tools:
-  stm32ai:
-    version: 8.1.0
-    optimization: balanced
-    on_cloud: True
-    path_to_stm32ai: C:/Users/XXX/STM32Cube/Repository/Packs/STMicroelectronics/X-CUBE-AI/8.1.0/Utilities/windows/stm32ai.exe
-  path_to_cubeIDE: C:/ST/STM32CubeIDE_1.1.0/STM32CubeIDE/stm32cubeide.exe
+   stedgeai:
+      version: 9.1.0
+      optimization: balanced
+      on_cloud: True
+      path_to_stedgeai: C:/Users/<XXXXX>/STM32Cube/Repository/Packs/STMicroelectronics/X-CUBE-AI/<*.*.*>/Utilities/windows/stedgeai.exe
+   path_to_cubeIDE: C:/ST/STM32CubeIDE_1.15.0/STM32CubeIDE/stm32cubeide.exe
 
 benchmarking:
   board: B-U585I-IOT02A

object_detection/src/preprocessing/tiny_yolo_v2_preprocess.py

@@ -492,11 +492,12 @@ def tiny_yolo_v2_preprocess(cfg):
     # Load the quantization dataset if provided
     if quantization_path or training_path:
 
-        if Path(cfg.general.model_path).suffix =='.onnx':
-            _, ish = get_model_name_and_its_input_shape(cfg.general.model_path)
-            input_shape = [ish[1],ish[2],ish[0]]
-        else:
-            _, input_shape = get_model_name_and_its_input_shape(cfg.general.model_path)
+        if cfg.general.model_path:
+            if Path(cfg.general.model_path).suffix =='.onnx':
+                _, ish = get_model_name_and_its_input_shape(cfg.general.model_path)
+                input_shape = [ish[1],ish[2],ish[0]]
+            else:
+                _, input_shape = get_model_name_and_its_input_shape(cfg.general.model_path)
 
         img_width, img_height, _ = input_shape
         if training_path:

semantic_segmentation/pretrained_models/README.md

@@ -33,10 +33,10 @@ IoU are averaged on all classes including background.
 
 | Models                                                | Implementation | Dataset                | Input Resolution | Accuracy (%) | average IoU | Activation RAM (MiB) | Weights Flash (MiB) | STM32Cube.AI version      | Source                                                                                                                                                               |
 |-------------------------------------------------------|----------------|------------------------|------------------|--------------|-------------|----------------------|---------------------|---------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| deeplabv3_257_int8_per_tensor                         | TensorFlow     | PASCAL VOC + COCO 2012 | 257x257x3        | 88.66        | 59.06       |       25.7            |       0.86        | 9.1.0                     | Available in X-LINUX-AI package [link](https://www.st.com/en/embedded-software/x-linux-ai.html)                                                                              |
-| deeplab_v3_mobilenetv2_05_fft_float32 | Tensorflow     | PASCAL VOC + COCO 2012        | 512x512x3        | 93.29       | 73.44       | /                | /              | 9.1.0                     | [link](./deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft.h5) 
-| deeplab_v3_mobilenetv2_05_fft_per_channel | Tensorflow     | PASCAL VOC + COCO 2012        | 512x512x3        | 91.3        | 67.32       |     57.38        | 7.63             | 9.1.0                     | [link](./deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite) |
-deeplab_v3_mobilenetv2_05_fft_int8_f32_per_channel | Tensorflow     | PASCAL VOC + COCO 2012        | 512x512x3        |    92.83     |   71.93     |      55.91       |       6.2        | 9.1.0                     | [link](./deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8_f32.tflite) |
+| deeplabv3_257_int8_per_tensor                         | TensorFlow     |  COCO 2017 + PASCAL VOC 2012 | 257x257x3        | 88.66        | 59.06       |       25.7            |       0.86        | 9.1.0                     | Available in X-LINUX-AI package [link](https://www.st.com/en/embedded-software/x-linux-ai.html)                                                                              |
+| deeplab_v3_mobilenetv2_05_fft_float32 | Tensorflow     |  COCO 2017 + PASCAL VOC 2012        | 512x512x3        | 93.29       | 73.44       | /                | /              | 9.1.0                     | [link](./deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft.h5) 
+| deeplab_v3_mobilenetv2_05_fft_per_channel | Tensorflow     |  COCO 2017 + PASCAL VOC 2012        | 512x512x3        | 91.3        | 67.32       |     57.38        | 7.63             | 9.1.0                     | [link](./deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite) |
+deeplab_v3_mobilenetv2_05_fft_int8_f32_per_channel | Tensorflow     |  COCO 2017 + PASCAL VOC 2012        | 512x512x3        |    92.83     |   71.93     |      55.91       |       6.2        | 9.1.0                     | [link](./deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8_f32.tflite) |
 </details>
 
 

semantic_segmentation/pretrained_models/deeplab_v3/README.md

@@ -59,7 +59,7 @@ For an image resolution of NxM and P classes
 To train the deeplab_v3 with backbone MobileNet v2 model with pretrained weights, from scratch or fine-tune it on your own dataset, you need to configure the [user_config.yaml](../../src/user_config.yaml) file following the
 [tutorial](../../README.md) under the src section.
 
-As an example, [deeplab_v3_mobilenetv2_05_16_512_fft.yaml](./ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_config.yaml) file is used to train on PASCAL VOC + COCO 2012 dataset. You can copy its content in the [user_config.yaml](../../src/user_config.yaml) file provided under 
+As an example, [deeplab_v3_mobilenetv2_05_16_512_fft.yaml](./ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_config.yaml) file is used to train on COCO 2017 + PASCAL VOC 2012 dataset. You can copy its content in the [user_config.yaml](../../src/user_config.yaml) file provided under 
 the src section to reproduce the results presented below.
 
 ## Deployment
@@ -73,16 +73,16 @@ To deploy your trained model, you need to configure the same [user_config.yaml](
 Measures are done with default STM32Cube.AI configuration with enabled input / output allocated option.
 
 
-### Reference **MPU** inference time based on PASCAL VOC + COCO 2012  segmentation dataset 21 classes (see Accuracy for details on dataset)
+### Reference **MPU** inference time based on COCO 2017 + PASCAL VOC 2012  segmentation dataset 21 classes (see Accuracy for details on dataset)
 | Model                                                                                                                                                                                                          | Dataset     | Format | Resolution | Quantization   | Board             | Execution Engine | Frequency | Inference time (ms) | %NPU  | %GPU   | %CPU | X-LINUX-AI version |       Framework       |
 |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|--------|------------|----------------|-------------------|------------------|-----------|---------------------|-------|--------|------|--------------------|-----------------------|
-| [DeepLabV3 per tensor (no ASPP)](https://www.st.com/en/embedded-software/x-linux-ai.html)                                                                                                                       | PASCAL VOC + COCO 2012  | Int8   | 257x257x3  | per-tensor     | STM32MP257F-DK2   | NPU/GPU          | 1500  MHz | 52.75           | 99.2 | 0.80  | 0 | v5.1.0             | OpenVX                |                |       |        |      | v5.1.0 
-| [DeepLabV3 per channel](./ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite) | PASCAL VOC + COCO 2012  | Int8   | 512x512x3  | per-channel ** | STM32MP257F-DK2   | NPU/GPU          | 1500  MHz | 806.12            | 8.73| 91.27 | 0   | v5.1.0             | OpenVX                |
-| [DeepLabV3 mixed precision](./ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8_f32.tflite) | PASCAL VOC + COCO 2012  | Int8 & float32  | 512x512x3  | per-channel ** | STM32MP257F-DK2   | NPU/GPU          | 1500  MHz |  894.56  | 7.67 | 92.33 | 0  | v5.1.0             | OpenVX                |
+| [DeepLabV3 per tensor (no ASPP)](https://www.st.com/en/embedded-software/x-linux-ai.html)                                                                                                                       | COCO 2017 + PASCAL VOC 2012 | Int8   | 257x257x3  | per-tensor     | STM32MP257F-DK2   | NPU/GPU          | 1500  MHz | 52.75           | 99.2 | 0.80  | 0 | v5.1.0             | OpenVX                |                |       |        |      | v5.1.0 
+| [DeepLabV3 per channel](./ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8   | 512x512x3  | per-channel ** | STM32MP257F-DK2   | NPU/GPU          | 1500  MHz | 806.12            | 8.73| 91.27 | 0   | v5.1.0             | OpenVX                |
+| [DeepLabV3 mixed precision](./ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8_f32.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8 & float32  | 512x512x3  | per-channel ** | STM32MP257F-DK2   | NPU/GPU          | 1500  MHz |  894.56  | 7.67 | 92.33 | 0  | v5.1.0             | OpenVX                |
 
 ** **To get the most out of MP25 NPU hardware acceleration, please use per-tensor quantization**
 
-### Accuracy with PASCAL VOC + COCO 2012 
+### Accuracy with COCO 2017 + PASCAL VOC 2012
 
 Dataset details: [link](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/), License [Database Contents License (DbCL) v1.0](https://opendatacommons.org/licenses/dbcl/1-0/) , Number of classes: 21, Number of images: 11530
 Please note, that the following accuracies are evaluated on Pascal VOC 2012 validation set (val.txt), and with a preprocessing resize with interpolation method 'bilinear'.
@@ -91,17 +91,17 @@ Moreover, IoU are averaged on all classes including background.
 | Model Description                                                                                                                                            | Resolution | Format     | Accuracy | Averaged IoU |
 |--------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|------------|----------|--------------|
 | [DeepLabV3 per tensor (no ASPP)](https://www.st.com/en/embedded-software/x-linux-ai.html)                                                            | 257x257x3  | Int8       | 88.6%    | 59.33%       |
-| [DeepLabV3 float precision](./ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft.h5) | 512x512x3  | Float      | 93.29%   | 73.44%       |
-| [DeepLabV3 per channel](./ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite) | 512x512x3  | Int8       | 91.3%    | 67.32%       |
-| [DeepLabV3 mixed precision](./ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8_f32.tflite) | 512x512x3  | Int8/Float | 92.83%   | 71.93%       |
+| [DeepLabV3 float precision](./ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft.h5) | 512x512x3  | Float      | 93.29%   | 73.44%       |
+| [DeepLabV3 per channel](./ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite) | 512x512x3  | Int8       | 91.3%    | 67.32%       |
+| [DeepLabV3 mixed precision](./ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8_f32.tflite) | 512x512x3  | Int8/Float | 92.83%   | 71.93%       |
 
 ## Retraining and code generation
 
 - **DeepLabV3 per tensor**:
    This model, which does not include ASPP (Atrous Spatial Pyramid Pooling), was downloaded from the TensorFlow DeepLabV3 page on[Kaggle](https://www.kaggle.com/models/tensorflow/deeplabv3/).
 
 - **DeepLabV3 float precision**:
-   This model is the result of using the [deeplab_v3_mobilenetv2_05_16_512_fft.yaml](./ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_config.yaml) configuration file to train the model on the PASCAL VOC + COCO 2012 dataset.
+   This model is the result of using the [deeplab_v3_mobilenetv2_05_16_512_fft.yaml](./ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_config.yaml) configuration file to train the model on the COCO 2017 + PASCAL VOC 2012 dataset.
 
 - **DeepLabV3 Per channel**:
    This model is quantized `per channel` version of DeepLabV3 float precision. It is generated using the quantization service with the [the quantization_config.yaml](../../src/config_file_examples/quantization_config.yaml) configuration file.

semantic_segmentation/pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft.h5 renamed to semantic_segmentation/pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft.h5

@@ -13,13 +13,13 @@ dataset:
                 "car", "cat", "chair", "cow", "dining table", "dog", "horse", "motorbike",
                 "person", "potted plant", "sheep", "sofa", "train", "tv/monitor"]
 
-  training_path: ../datasets/VOC2012_COCO_train_val/JPEGImages
-  training_masks_path: ../datasets/VOC2012_COCO_train_val/SegmentationClassAug
-  training_files_path: ../datasets/VOC2012_COCO_train_val/ImageSets/Segmentation/trainaug.txt
+  training_path: ../datasets/COCO2017_VOC2012/JPEGImages
+  training_masks_path: ../datasets/COCO2017_VOC2012/SegmentationClassAug
+  training_files_path: ../datasets/COCO2017_VOC2012/ImageSets/Segmentation/trainaug.txt
 
-  validation_path: ../datasets/VOC2012_COCO_train_val/JPEGImages
-  validation_masks_path: ../datasets/VOC2012_COCO_train_val/SegmentationClassAug
-  validation_files_path: ../datasets/VOC2012_COCO_train_val/ImageSets/Segmentation/val.txt
+  validation_path: ../datasets/COCO2017_VOC2012/JPEGImages
+  validation_masks_path: ../datasets/COCO2017_VOC2012/SegmentationClassAug
+  validation_files_path: ../datasets/COCO2017_VOC2012/ImageSets/Segmentation/val.txt
   validation_split: 
 
   test_path: 

semantic_segmentation/pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_config.yaml renamed to semantic_segmentation/pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_config.yaml

@@ -16,7 +16,7 @@ In particular, `operation_mode` should be set to evaluation and the `benchmarkin
 
 ```yaml
 general:
-   model_path: ../pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite
+   model_path: ../pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite
 
 operation_mode: benchmarking
 ```

semantic_segmentation/pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite renamed to semantic_segmentation/pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite

@@ -1,5 +1,5 @@
 general:
-   model_path: ../pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/pascal_voc_coco_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite
+   model_path: ../pretrained_models/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512_fft/deeplab_v3_mobilenetv2_05_16_512_fft_int8.tflite
 
 operation_mode: benchmarking