Issue loading converted YOLOv5 ONNX model on Ambarella CV25

[sg-sugiyama]

Hi, I'm working on deploying a YOLOv5-based model to an Ambarella CV25 board using cvtool.

The ONNX model was successfully converted and a .bin file was generated, but loading the model on the device gives the following error:

create_inference_model: Unable to load model with device ambarella-cvflow: Could not load model: Could not initialize net

I'm attaching the conversion command and part of the log. Could someone help me figure out what might be going wrong?
Are there specific recommended settings or restrictions for exporting YOLOv5 models for CV25 compatibility?

1. Model graph summary before surgery
onnx_print_graph_summary.py -p my_model.onnx

Version of the ai.onnx operator set (17) exceeds the latest stable version supported by cvtools (16). Unforeseen errors may occur, please re-export your model.
********************************************* Metadata *********************************************
ONNX Version: 1.11.0
Producer: pytorch-2.7.0
Operator Sets: ai.onnx v17
****************************************** Graph summary *******************************************
Total operators: 263
Total (non-constant) operators: 263
Primary inputs (1):
-> Tensor(images, shape=[1 x 3 x 640 x 640], dtype=float32)
Primary outputs (1):
-> Tensor(output0, shape=[1 x 25200 x 8], dtype=float32)
Unsupported tensors with rank > 4 (45):

[...]

*************************** Transform recommendations (graph_surgery.py) ***************************
Found foldable operators, hint: CVFlow
****************************************************************************************************

2. Graph surgery
graph_surgery.py onnx -m my_model.onnx -isrc "i:images|is:1,3,640,640" -t CVFlow

Applying custom transform pass: CVFlow
Applying custom transform pass: ConstantifyShapes
    [info] No eligible nodes found over 10 samplings
Post-transform loss validation: Pass
Applying custom transform pass: FoldConstants
    [info] Running onnxruntime forward pass to fetch values of fold targets
Applying custom transform pass: FoldNoOps
Post-transform loss validation: Pass

3. Model graph summary after surgery
onnx_print_graph_summary.py -p my_model_modified.onnx

Version of the ai.onnx operator set (17) exceeds the latest stable version supported by cvtools (16). Unforeseen errors may occur, please re-export your model.
********************************************* Metadata *********************************************
ONNX Version: 1.11.0
Producer: pytorch-2.7.0
Operator Sets: ai.onnx v17
****************************************** Graph summary *******************************************
Total operators: 236
Total (non-constant) operators: 236
Primary inputs (1):
-> Tensor(images, shape=[1 x 3 x 640 x 640], dtype=float32)
Primary outputs (1):
-> Tensor(output0, shape=[1 x 25200 x 8], dtype=float32)
Unsupported tensors with rank > 4 (39):

[...]

****************************************************************************************************

4. Generate image list for int8 calibration
gen_image_list.py -f images/ -o ./dra_image_bin/image_list.txt -ns -e jpg -c RGB -d int8 -r 640,640 -bf dra_image_bin/ -bo dra_image_bin/dra_bin_list.txt

5. Parse the ONNX model
onnxparser.py -m my_model_modified.onnx -isrc "i:images=dra_image_bin/dra_bin_list.txt|idf:0,0,0,0|is:1,3,640,640|iq" -o my_model -of ./outputs/ -on output0

cannot import name 'optimizer' from 'onnx' (/usr/local/lib/python3.8/dist-packages/onnx/__init__.py)
instead, import name 'onnxoptimizer' (/usr/local/lib/python3.8/dist-packages/onnxoptimizer/__init__.py)
Warning [parser]: removing existing dir /home/tatsuya/work/outputs
*** (info) importing amba_cnn_py @ /usr/local/amba-cv-tools-basic.2.5.6.0.1245.ubuntu-20.04/cv2/tv2/release/AmbaCnn/cv2.basic.2.5.6.0.1245.ubu2004/python/amba_cnn_py
*** (info) setting project cv22
opened session 6536 on libambacnn cv2.basic.2.5.6.0.1245.ubu2004 (9322cec*0), at 2025-06-05 07:08:31
libambacnn session config: target_device = cv22, gpu_id = -1, mode = default, split = False
[onnxparser] : WARNING - Model opset version [17] exceeds the maximum opset version currently supported [16]. Unforeseen errors may occur, please re-export model to a lower opset.

6. generate vas output
   vas -auto -show-progress outputs/my_model.vas
7. generate .bin file
   cavalry_gen -V 2.1.6 -d ./vas_output/ -f my_model.bin

Thank you in advance for your support!

[github-actions[bot]]

This automatically generated reply acts as a friendly reminder.

Answers to your questions will most often come from the community, from developers like yourself. You will, from time to time, find that Axis employees answers some of the questions, but this is not a guarantee. Think of the discussion forum as a complement to other support channels, not a replacement to any of them.

If your question remains unanswered for a period of time, please revisit it to see whether it can be improved by following the guidelines listed in Axis support guidelines.

[vivekatoffice]

Hi @sg-sugiyama ,
I hope you have checked this:

Image reference

Image reference

Image reference

Test your model can be used for testing your model before packaging it in the ACAP application.

Additional information based on my understanding:

Ambarella CVflow Model Deployment: Key Details

Ambarella’s SoCs like CV25 or CV28 use a proprietary CVflow hardware accelerator. Unlike general-purpose platforms, deploying neural networks here requires offline compilation into a native binary format, with tools and workflows provided only through Ambarella’s official toolchain.

1. Offline Compilation – No Runtime Interpreter

Unlike TensorFlow Lite or ONNX Runtime, Ambarella devices do not support on-device interpretation or just-in-time model loading.

What this means:

• You must compile the model offline using Ambarella’s toolchain (e.g., cvtool, ambacv).

• The compiled output is in a hardware-specific binary format (.nmx and related files).

• You cannot deploy .onnx, .tflite, or .pb files directly to the device.

• Ambarella’s toolchain is not publicly available

2. Model Format Requirements

• Preferred input model: ONNX
• Deprecated formats: TFLite, TensorFlow (.pb), Caffe

3. Limitations

• Post-training Quantization and Optimizations: handled at compile time via toolchain

4. You may need to apply the patch for YOLOv5-based model like in the YOLOv5 guide

• Use simple layers
• You can use the Ambarella SDK tool like cvtool to compare your *.bin file with these two examples for Image classification:
  
  Image reference
  

[sg-sugiyama]

Hi @vivekatoffice, thank you very much for the detailed and helpful explanation!

Yes, I had gone through the reference materials you shared.
Actually, I was experimenting with a YOLOv5 model that we had fine-tuned for our own use case and had previously deployed on ARTPEC-8, and I was curious to see if I could get it running on the CV25 as well.

But based on your guidance, it seems like sticking closely to the CV25-specific instructions is definitely the safer route.
So now I’m planning to try fine-tuning an SSD MobileNet model instead and see how that goes.

Also, I had one more question:
For models intended for CV25, is it recommended to train with TensorFlow 1 or TensorFlow 2? I’d appreciate any guidance on that as well.

Thanks again for your support!

[vivekatoffice]

Hi @sg-sugiyama ,

Regarding your question:

Should I train with TensorFlow 1 or TensorFlow 2 for models intended for CV25?

Since TensorFlow Lite (TFLite) is no longer supported on CVflow, and no TFLite delegate is available, training with either TensorFlow 1 or 2 is not relevant for deployment via TFLite.

Instead, CV25 workflows now favor ONNX models, which are compiled offline using the CVflow toolchain (such as the offline compiler for CVflow).

You may also check this 604

[sg-sugiyama]

Thank you @vivekatoffice
We’ll make sure to train with ONNX export in mind, since TFLite’s not an option for CVflow.

[vivekatoffice]

Feel free to re-open it OR share the solution/alternative/comment if already resolved👁️‍🗨️