You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The process of deploying a new neural network model to the Raspberry Pi AI Camera will normally consist of the following steps:
3
+
To deploy a new neural network model to the Raspberry Pi AI Camera, complete the following steps:
4
4
5
-
. A neural network model must be provided.
6
-
. The model must be quantised and compressed so that it can be run using the resources available in the IMX500 camera.
7
-
. The compressed model must be converted to IMX500 format.
8
-
. Finally, the model must be packaged into a firmware file that can be loaded at runtime into the camera.
5
+
. Provide a neural network model.
6
+
. Quantise and compress the model so that it can run using the resources available on the IMX500 camera module.
7
+
. Convert the compressed model to IMX500 format.
8
+
. Package the model into a firmware file that can be loaded at runtime onto the camera.
9
9
10
-
The first three steps will normally be performed on a more powerful computer such as a desktop or server, whilst the final packaging step must be performed on a Raspberry Pi.
10
+
The first three steps will normally be performed on a more powerful computer such as a desktop or server. You must run the final packaging step on a Raspberry Pi.
11
11
12
12
=== Model Creation
13
13
14
14
The creation of neural network models is beyond the scope of this guide. Existing models can be re-used, or new ones created using popular frameworks like TensorFlow or PyTorch.
15
15
16
-
For more information, readers are referred to the official https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera[AITRIOS Developer] website.
16
+
For more information, see the official https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera[AITRIOS developer website].
17
17
18
18
=== Quantisation and Compression
19
19
20
-
Models are quantised and compressed using Sony's _Model Compression Toolkit_. This can be installed with
20
+
Models are quantised and compressed using Sony's Model Compression Toolkit. To install the toolkit, run the following command:
21
21
22
22
[source,console]
23
23
----
24
-
pip install model_compression_toolkit
24
+
$ pip install model_compression_toolkit
25
25
----
26
26
27
-
and information and tutorials can be found at the project's https://github.com/sony/model_optimization[GitHub page].
27
+
For more information, see the https://github.com/sony/model_optimization[Sony model optimization GitHub repository].
28
+
29
+
The Model Compression Toolkit generates a quantised model in the following formats:
30
+
31
+
* Keras (TensorFlow)
32
+
* ONNX (PyTorch)
28
33
29
-
The _Model Compression Toolkit_ will genearate a quantised model in either Keras (for TensorFlow) or ONNX (for PyTorch) format.
TIP: If you need to install both these packages, we strongly recommend doing so in separate Python virtual environments (for example, using `python -m venv <virtual-environment-name>`). This avoids any problems with TensorFlow and PyTorch causing conflicts with one another.
50
54
51
-
Next, we can convert the model. For TensorFlow, use
55
+
Next, convert the model:
52
56
57
+
[tabs]
58
+
======
59
+
TensorFlow::
60
+
+
53
61
[source,console]
54
62
----
55
-
imxconv-tf -i <compressed Keras model> -o <output folder>
63
+
$ imxconv-tf -i <compressed Keras model> -o <output folder>
In both cases, the output folder will be created containing, among other things, a memory usage report, plus a `packerOut.zip` file which is what we will need to copy to the Pi for the final step.
74
+
Both commands create an output folder that contains a memory usage report and a `packerOut.zip` file.
66
75
67
-
Again, for more information on the model conversion process, please refer to the official https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera/documentation/imx500-converter[IMX500 Converter] documentation.
76
+
For more information on the model conversion process, see the official https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera/documentation/imx500-converter[Sony IMX500 Converter documentation].
68
77
69
78
=== Packaging
70
79
71
-
The final step, which we run on a Raspberry Pi, is packaging the model into an _RPK_ file. This _RPK_ file is then uploaded to the IMX500 camera when running the neural network model. Before proceeding, we must install the necessary tools:
80
+
IMPORTANT: You must run this step on a Raspberry Pi.
81
+
82
+
The final step packages the model into an RPK file. When running the neural network model, we'll upload this file to the AI Camera. Before proceeding, run the following command to install the necessary tools:
72
83
73
84
[source,console]
74
85
----
75
86
$ sudo apt install imx500-tools
76
87
----
77
88
78
-
Now we can run
89
+
To package the model into an RPK file, run the following command:
79
90
80
91
[source,console]
81
92
----
82
93
imx500-package.sh -i <path to packerOut.zip> -o <output folder>
83
94
----
84
95
85
-
The output folder should finally contain a file `network.rpk`, the name of which is what we pass to our IMX500 camera applications.
96
+
This command should create a file named `network.rpk` in the output folder. You'll pass the name of this file to your IMX500 camera applications.
86
97
87
-
More specific instructions on all these tools, and their constraints is out of scope for this tutorial. For a more comprehensive set of instructions and further specifics on the tools used, please see the official https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera/documentation/imx500-packager[IMX500 Packager] documentation.
98
+
For a more comprehensive set of instructions and further specifics on the tools used, see the https://developer.aitrios.sony-semicon.com/en/raspberrypi-ai-camera/documentation/imx500-packager[Sony IMX500 Packager documentation].
Copy file name to clipboardExpand all lines: documentation/asciidoc/computers/raspberry-pi/pcie.adoc
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ To connect a PCIe https://datasheets.raspberrypi.com/hat/hat-plus-specification.
9
9
10
10
For more information about the PCIe FPC connector pinout and other details needed to create third-party devices, accessories, and HATs, see the https://datasheets.raspberrypi.com/pcie/pcie-connector-standard.pdf[Raspberry Pi Connector for PCIe] standards document. It should be read alongside the https://datasheets.raspberrypi.com/hat/hat-plus-specification.pdf[Raspberry Pi HAT+ Specification].
11
11
12
-
NOTE: Starting with version 2020-06-05 of the Raspberry Pi 5 bootloader, Enumeration of PCIe devices behind a switch is preliminary supported. See also this https://github.com/raspberrypi/firmware/issues/1833[issue].
12
+
NOTE: Only certain devices [support](https://github.com/raspberrypi/firmware/issues/1833[support]) enumeration of PCIe devices behind a switch.
0 commit comments