Skip to content

Commit 9fcc378

Browse files
committed
imx500: Add instructions for the new Edge-MDT tooling
1 parent 1934985 commit 9fcc378

File tree

1 file changed

+29
-27
lines changed

1 file changed

+29
-27
lines changed

documentation/asciidoc/accessories/ai-camera/model-conversion.adoc

Lines changed: 29 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -2,46 +2,35 @@
22

33
To deploy a new neural network model to the Raspberry Pi AI Camera, complete the following steps:
44

5-
. Provide a neural network model.
6-
. Quantise and compress the model so that it can run using the resources available on the IMX500 camera module.
7-
. Convert the compressed model to IMX500 format.
5+
. Provide a floating-point neural network model (PyTorch or TensorFlow).
6+
. Run the model through Edge-MDT (Edge AI Model Development Toolkit).
7+
.. *Quantise* and compress the model so that it can run using the resources available on the IMX500 camera module.
8+
.. *Convert* the compressed model to IMX500 format.
89
. Package the model into a firmware file that can be loaded at runtime onto the camera.
910

10-
The first three steps will normally be performed on a more powerful computer such as a desktop or server. You must run the final packaging step on a Raspberry Pi.
11+
The first two steps will normally be performed on a more powerful computer such as a desktop or server. You must run the final packaging step on a Raspberry Pi.
1112

1213
=== Model creation
1314

14-
The creation of neural network models is beyond the scope of this guide. Existing models can be re-used, or new ones created using popular frameworks like TensorFlow or PyTorch.
15+
The creation of neural network models is beyond the scope of this guide. Existing models can be re-used, or new ones created using popular AI frameworks like TensorFlow or PyTorch.
1516

1617
For more information, see the official https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera[AITRIOS developer website].
1718

18-
=== Quantisation and compression
19+
=== Model compression and conversion
1920

20-
Models are quantised and compressed using Sony's Model Compression Toolkit. To install the toolkit, run the following command:
21+
==== Edge-MDT installation
2122

22-
[source,console]
23-
----
24-
$ pip install model_compression_toolkit
25-
----
26-
27-
For more information, see the https://github.com/sony/model_optimization[Sony model optimization GitHub repository].
28-
29-
The Model Compression Toolkit generates a quantised model in the following formats:
23+
The Edge-MDT (Model Development Toolkit) software package installs all the tools required to quantise, compress, and convert models to run on your IMX500 device.
3024

31-
* Keras (TensorFlow)
32-
* ONNX (PyTorch)
33-
34-
=== Conversion
35-
36-
To convert a model, first install the converter tools:
25+
The Edge-MDT package takes a parameter to select between installing the PyTorch or TensorFlow version of the tools.
3726

3827
[tabs]
3928
======
4029
TensorFlow::
4130
+
4231
[source,console]
4332
----
44-
$ pip install imx500-converter[tf]
33+
$ pip install edge-mdt[tf]
4534
----
4635
+
4736
TIP: Always use the same version of TensorFlow you used to compress your model.
@@ -50,13 +39,26 @@ PyTorch::
5039
+
5140
[source,console]
5241
----
53-
$ pip install imx500-converter[pt]
42+
$ pip install edge-mdt[pt]
5443
----
5544
======
5645

57-
If you need to install both packages, use two separate Python virtual environments. This prevents TensorFlow and PyTorch from causing conflicts with one another.
46+
If you need to install both packages, use two separate Python virtual environments. This prevents TensorFlow and PyTorch from causing conflicts with each other.
47+
48+
==== Model Optimization
5849

59-
Next, convert the model:
50+
Models are quantised and compressed using Sony's Model Compression Toolkit (MCT). This tool is automatically installed as part of the Edge-MDT installation step. For more information, see the https://github.com/sony/model_optimization[Sony model optimization GitHub repository].
51+
52+
The Model Compression Toolkit generates a quantised model in the following formats:
53+
54+
* Keras (TensorFlow)
55+
* ONNX (PyTorch)
56+
57+
=== Conversion
58+
59+
The converter is a command line application that compiles the quantised model (in .onnx or .keras formats) into a binary file that can be packaged and loaded onto the AI Camera. This tool is automatically installed as part of the Edge-MDT installation step.
60+
61+
To convert a model model:
6062

6163
[tabs]
6264
======
@@ -75,9 +77,9 @@ $ imxconv-pt -i <compressed ONNX model> -o <output folder>
7577
----
7678
======
7779

78-
Both commands create an output folder that contains a memory usage report and a `packerOut.zip` file.
80+
IMPORTANT: For optimal use of the memory available to the accelerator on the IMX500 sensor, add `--no-input-persistency` to the above commands. However, this will disable input tensor generation that may be used for debugging purposes.
7981

80-
For optimal use of the memory available to the accelerator on the IMX500 sensor, add `--no-input-persistency` to the above commands. However, this will disable input tensor generation and return to the application for debugging purposes.
82+
Both commands create an output folder that contains a memory usage report and a `packerOut.zip` file.
8183

8284
For more information on the model conversion process, see the official https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera/documentation/imx500-converter[Sony IMX500 Converter documentation].
8385

0 commit comments

Comments
 (0)