Skip to content

Latest commit

 

History

History
398 lines (225 loc) · 11.6 KB

GETTING_STARTED.rst

File metadata and controls

398 lines (225 loc) · 11.6 KB

Getting Started

This document provides installation instructions and basic usage examples of the Hailo Model Zoo.

System Requirements

  • Ubuntu 22.04/24.04, 64 bit (supported also on Windows, under WSL2)
  • Python 3.10/3.11/3.12, including pip and virtualenv
  • Hailo Dataflow Compiler v5.2.0 (Obtain from hailo.ai)
  • HailoRT v5.2.0 (Obtain from hailo.ai) - required only for inference on Hailo-10H.
  • The Hailo Model Zoo supports Hailo-10H connected via PCIe only.
  • Nvidia’s Pascal/Turing/Ampere GPU architecture (such as Titan X Pascal, GTX 1080 Ti, RTX 2080 Ti, or RTX A4000)
  • GPU driver version 525
  • CUDA 12.5.1
  • CUDNN 9.10

Install Instructions

Hailo Software Suite

The model requires the corresponding Dataflow Compiler version, and the optional HailoRT version. Therefore, it is recommended to use the Hailo Software Suite, that includes all of Hailo's SW components and ensures compatibility across product versions.

The Hailo Software Suite is composed of the Dataflow Compiler, HailoRT, TAPPAS and the Model Zoo see diagram below.

Detailed block diagram of Hailo software packages

Detailed block diagram of Hailo software packages

Manual Installation

  1. Install the Hailo Dataflow Compiler and enter the virtualenv (visit hailo.ai for further instructions).

  2. Install the HailoRT - required only for inference on Hailo-10H (visit hailo.ai for further instructions).

  3. Clone the Hailo Model Zoo repo:

    git clone https://github.com/hailo-ai/hailo_model_zoo.git

  4. Run the setup script:

    cd hailo_model_zoo; pip install -e .

  5. For setting up datasets, please see DATA.

  6. Verify Hailo-10 is connected via PCIe (required only to run on Hailo-10. Full-precision / emulation runs on GPU.)

    hailortcli fw-control identify

    Note

    hailortcli is the HailoRT command-line tool for interacting with Hailo devices.

    Expected output:

    (hailo) Running command 'fw-control' with 'hailortcli'
Identifying board
Control Protocol Version: 2
Firmware Version: 4.6.0 (release,app)
Logger Version: 0
Board Name: Hailo-10H
Device Architecture: HAILO10H
Serial Number: HLUTM20204900071
Part Number: HM218B1C2FA
Product Name: Hailo-10H AI ACCELERATOR M.2 MODULE

Upgrade Instructions

If you want to upgrade to a specific Hailo Model Zoo version within a suite or on top of a previous installation not in the suite.

  1. Pull the specific repo branch:

    git clone -b v2.6 https://github.com/hailo-ai/hailo_model_zoo.git

  2. Run the setup script:

    cd hailo_model_zoo; pip install -e .

Usage

Flow Diagram

The following scheme shows a high-level view of the model-zoo evaluation process, and the different stages involved.

By default, each stage executes all of its previously necessary stages according to the above diagram. The post-parsing stages also have an option to start from the product of previous stages (i.e., the Hailo Archive (HAR) file), as explained below. The operations are configured through a YAML file that exists for each model in the cfg folder. For a description of the YAML structure, please see YAML.

NOTE: Hailo Model Zoo provides the following functionality for Model Zoo models only. If you wish to use your custom model, use the Dataflow Compiler directly.

Parsing

The pre-trained models are stored on AWS S3 and will be downloaded automatically when running the model zoo into your data directory. To parse models into Hailo's internal representation and generate the Hailo Archive (HAR) file:

hailomz parse <model_name>
  • The default compilation target is Hailo-10H. To compile for different architecture (Hailo-15H for example), use --hw_arch hailo15h as CLI argument:
hailomz parse <model_name> --hw-arch hailo15h
  • To customize the parsing behavior, use --start-node-names and/or --end-node-names flags:
hailomz parse <model_name> --start-node-names <name1> --end-node-names <name1> <name2>

Optimization

To optimize models, convert them from full precision into integer representation and generate a quantized Hailo Archive (HAR) file:

hailomz optimize <model_name>

To optimize the model starting from a previously generated HAR file:

hailomz optimize <model_name> --har /path/to/model.har

You can use your own images by giving a directory path to the optimization process, with the following supported formats (.jpg,.jpeg,.png):

hailomz optimize <model_name> --calib-path /path/to/calibration/imgs/dir/
  • This step requires data for calibration. For additional information, please see OPTIMIZATION.

To achieve the highest performance, use the performance flag:

hailomz optimize <model_name> --performance

The flag will be ignored on models that do not support this feature. The default and performance model scripts are located in hailo_model_zoo/cfg/alls/

To add input conversion to the model, use the input conversion flag:

hailomz optimize <model_name> --input-conversion nv12_to_rgb
  • Do not use the flag if an input conversion already exists in the alls or in the YAML.

To add input resize to the model, use the resize flag:

hailomz optimize <model_name> --resize 1080 1920
  • Do not use the flag if resize already exists in the alls or in the YAML.

To adjust the number of classes in post-processing configuration, use classes flag:

hailomz optimize <model_name> --classes 80
  • Use this flag only if post-process exists in the alls or in the YAML.

Profiling

To generate the model profiler report:

hailomz parse <model_name>
hailo profiler path/to/model.har
  • When profiling a Quantized HAR file (the result of the optimization process), the report contains information about your model and accuracy.
  • When profiling a Compiled HAR file (the result of the compilation process), the report contains the expected performance on the Hailo hardware (as well as the accuracy information).

Compilation

To run the Hailo compiler and generate the Hailo Executable Format (HEF) file:

hailomz compile <model_name>

By default, the compilation target is Hailo-10H. To compile for a different architecture, use the --hw-arch command line argument:

hailomz compile <model_name> --hw-arch hailo15h

To generate the HEF starting from a previously generated HAR file:

hailomz compile <model_name> --har /path/to/model.har
  • When working with a generated HAR, the previously chosen architecture will be used.

To achieve the best performance, use the performance flag:

hailomz optimize <model_name> --performance --hw-arch hardware

The flag will be ignored on models that do not support this feature. The default and performance model scripts are located in hailo_model_zoo/cfg/alls/

To add input conversion to the model, use the input conversion flag:

hailomz compile <model_name> --input-conversion nv12_to_rgb

Do not use the flag if an input conversion already exists in the alls or in the YAML.

To add input resize to the model, use the resize flag:

hailomz compile <model_name> --resize 1080 1920

Do not use the flag if resize already exists in the alls or in the YAML.

Evaluation

To evaluate models in full precision:

hailomz eval <model_name>

To evaluate models starting from a previously generated Hailo Archive (HAR) file:

hailomz eval <model_name> --har /path/to/model.har

To evaluate models with the Hailo emulator (after quantization to integer representation - fast_numeric):

hailomz eval <model_name> --target emulator

To evaluate models on Hailo-10H:

hailomz eval <model_name> --target hardware

If multiple devices are available, it's possible to select a specific one. Make sure to run on a device compatible with the compiled model.

# Device id looks something like 0000:41:00.0
hailomz eval <model_name> --target <device_id>
# This command can be used to list available devices
hailomz eval --help

To limit the number of images for evaluation, use the following flag:

hailomz eval <model_name> --data-count <num-images>

To evaluate a model with additional input conversion, use the input conversion flag:

hailomz eval <model_name> --input-conversion nv12_to_rgb

Do not use the flag if an input conversion already exists in the alls or in the YAML.

To evaluate a model with input resize, use the resize flag:

hailomz eval <model_name> --resize 1080 1920

Do not use the flag if resize already exists in the alls or in the YAML.

To evaluate a cascade model please refer to CASCADE.

To explore other options (for example: changing the default batch-size) use:

hailomz eval --help
  • Currently MZ evaluation can be done only on Hailo-10H.

Visualization

To run visualization (without evaluation) and generate the output images:

hailomz eval <model_name> --visualize

To create a video file from the network predictions:

hailomz eval <model_name> --visualize --video-outpath /path/to/video_output.mp4

Info

You can easily print information of any network that exists in the model zoo to get a sense of its input/output shape, parameters, operations, framework, etc.

To print a model-zoo network information:

hailomz info <model_name>

Here is an example for printing information about mobilenet_v1:

hailomz info mobilenet_v1

Expected output:

<Hailo Model Zoo Info> Printing mobilenet_v1 Information
<Hailo Model Zoo Info>
        task:                    classification
        input_shape:             224x224x3
        output_shape:            1x1x1001
        operations:              0.57G
        parameters:              4.22M
        framework:               tensorflow
        training_data:           imagenet train
        validation_data:         imagenet val
        eval_metric:             Accuracy (top1)
        full_precision_result:   71.02
        source:                  https://github.com/tensorflow/models/tree/v1.13.0/research/slim
        license_url:             https://github.com/tensorflow/models/blob/v1.13.0/LICENSE

Compile multiple networks together

We can use multiple disjoint models in the same binary. This is useful for running several small models on the device.

python hailo_model_zoo/multi_main.py <config_name>

TFRecord to NPY conversion

There may be situations, where one might want to convert the tfrecord file to an npy file (for example, when explicitly using the Dataflow Compiler for quantization). To do so, run the command:

python hailo_model_zoo/tools/conversion_tool.py /path/to/tfrecord_file resnet_v1_50 --npy