|
3 | 3 |
|
4 | 4 | === Overview
|
5 | 5 |
|
6 |
| -The Raspberry Pi AI Camera works rather differently to more traditional AI-based camera image processing systems, as shown in the diagram below. |
| 6 | +The Raspberry Pi AI Camera works differently from traditional AI-based camera image processing systems, as shown in the diagram below: |
7 | 7 |
|
8 | 8 | image::images/imx500-comparison.svg[Traditional versus IMX500 AI camera systems]
|
9 | 9 |
|
10 |
| -On the left is a diagram of a more traditional AI camera system. Here, the camera delivers only images to the Raspberry Pi. The Raspberry Pi processes the images and is then responsible for performing AI inferencing. This may use an optional external AI accelerator, as shown, or it may happen (more slowly) in the CPU. |
| 10 | +The left side demonstrates the architecture of a traditional AI camera system. In such a system, the camera delivers images to the Raspberry Pi. The Raspberry Pi processes the images and then performs AI inference. Traditional systems may use external AI accelerators (as shown) or rely exclusively on the CPU. |
11 | 11 |
|
12 |
| -On the right we have the IMX500-based system. The camera module contains a small ISP which turns the raw camera image data into an _input tensor_ which is fed directly to the AI accelerator within the camera. In turn, this produces an _output tensor_, containing the inferencing results, which is fed back to the Raspberry Pi itself. There is no need for an external accelerator, nor for the Raspberry Pi to run neural network software on the CPU. |
| 12 | +The right side demonstrates the architecture of a system that uses IMX500. The camera module contains a small Image Signal Processor (ISP) which turns the raw camera image data into an _input tensor_. The camera module sends this tensor directly into the AI accelerator within the camera, which produces an _output tensor_ that contains the inferencing results. The AI accelerator sends this tensor to the Raspberry Pi. There is no need for an external accelerator, nor for the Raspberry Pi to run neural network software on the CPU. |
13 | 13 |
|
14 |
| -Some concepts that may be helpful to understand include: |
| 14 | +To fully understand this system, familiarise yourself with the following concepts: |
15 | 15 |
|
16 |
| -==== The _Input Tensor_ |
| 16 | +The _Input Tensor_:: The part of the sensor image passed to the AI engine for inferencing. Produced by a small on-board ISP which also crops and scales the camera image to the dimensions expected by the neural network that has been loaded. The input tensor is not normally made available to applications, though it is possible to access it for debugging purposes. |
17 | 17 |
|
18 |
| -This is the part of the sensor image that is passed to the AI engine for inferencing. It is produced by a small on-board ISP which also crops and scales the camera image to the dimensions expected by the neural network that has been loaded. The input tensor is not normally made available to applications, though it is possible to access it for debugging purposes. |
| 18 | +The _Region of Interest_ (ROI):: Specifies exactly which part of the sensor image is cropped out before being rescaled to the size demanded by the neural network. Can be queried and set by an application. The units used are always pixels in the full resolution sensor output. The default ROI setting uses the full image received from the sensor, cropping no data. |
19 | 19 |
|
20 |
| -==== The _Region of Interest_ |
21 |
| - |
22 |
| -The Region of Interest (or _ROI_) specifies exactly which part of the sensor image is cropped out before being rescaled to the size demanded by the neural network. It can be queried and set by an application. The units used are always pixels in the full resolution sensor output. |
23 |
| - |
24 |
| -By default, the ROI is set to be the full image received from the sensor (that is, nothing is actally cropped out). |
25 |
| - |
26 |
| -==== The _Output Tensor_ |
27 |
| - |
28 |
| -These are the results of inferencing performed by the neural network. The precise number and shape of the outputs will depend on the neural network, and application code will need to understand how to handle them. |
| 20 | +The _Output Tensor_:: The results of inferencing performed by the neural network. The precise number and shape of the outputs depend on the neural network. Application code must understand how to handle the tensor. |
29 | 21 |
|
30 | 22 | === System Architecture
|
31 | 23 |
|
32 |
| -The diagram below shows the various camera software components (in green) used during our imaging/inference use case with the Raspberry Pi AI Camera module hardware (in red). |
| 24 | +The diagram below shows the various camera software components (in green) used during our imaging/inference use case with the Raspberry Pi AI Camera module hardware (in red): |
33 | 25 |
|
34 | 26 | image::images/imx500-block-diagram.svg[IMX500 block diagram]
|
35 | 27 |
|
|
0 commit comments