aitrios-sample-application-reference Zone Detection

Table of Contents

• aitrios-sample-application-reference Zone Detection
  • Table of Contents
  • Overview
  • System Diagram
  • Project Structure
    • Directory and File Descriptions
  • Verified Operating Systems & Dependencies
  • Getting Started
    • Prepare a device on AITRIOS Console
      • Create and deploy artifacts
      • Configure the Edge Application
    • Check prerequisites for executing the Reference Solution
      • Docker and Docker Compose
    • Start Application on GitHub Codespaces
    • Local Installation
      • Running the Application
      • Frontend
      • Backend
  • Trademark
  • Notice
    • Security

Overview

This repository provides a reference implementation of a full solution using AITRIOS™. Building up on top of an existing AI model and Edge app, this reference solution implements a zone detection solution. It consists of a Backend that interacts with Console for AITRIOS using the Console's REST API that is paired with a UI that retrieves the meta-data from the Console for AITRIOS and displays the image together with the detected objects. The edge application calculates the overlap of the detections with a configurable zone which are shown in the UI.

System Diagram

Project Structure

The repository is organized as follows:

├── .devcontainer/
├── .github/workflows/
├── .vscode/
├── ai_model/
├── edge_app/
├── backend/
├── frontend/
├── docs/

Directory and File Descriptions

.devcontainer

This directory contains the files necessary to run this project in a development container (dev container), be it in GitHub Codespaces or Visual Studio Code. More details how to use it are provided in the USAGE.md document.

.github/workflows/

Contains GitHub Actions workflows for continuous integration and deployment.

.vscode/

This directory contains the launch.json file with configuration for a debugger in Visual Studio Code.

ai_model/

Contains materials necessary to create an ai model that shall be deployed on device to enable running the reference solution. Consult model_id.md for more details.

edge_app/

This directory contains the Markdown files that explain how to create edge applications that shall be deployed on device to enable running the reference solution.

• edge_app_v1.md defines the process of obtaining edge application related artifacts for a camera with the firmware version v1 and AITRIOS Console v1.
• edge_app_v2.md defines the process of obtaining edge application related artifacts for a camera with the firmware version v2 and AITRIOS Console v2.

backend/

Contains the backend application built with FastAPI. For more details on this directory's structure, check its README.md.

frontend/

This directory contains the frontend built with React and TypeScript. For more details on its structure, check its README.md.

docs/

This directory contains project documentation.

Verified Operating Systems & Dependencies

The application has been tested and verified to work with:

• Operating Systems: Ubuntu 22.04
• Python: 3.10-3.13
• Node.js: 20

Getting Started

To get started with the Zone Detection Application, follow the steps below.

Prepare a device on AITRIOS Console

To run this reference solution you need to have a CSV26 camera to be enrolled to your project in the Aitrios Console. In addition, it should have deployed a zone detection Edge application and a compatible object detection AI model. These artifacts should first be obtained locally, then imported to the Console and finally be deployed on device.

Note
For basic device and console operation, please refer to the Console V2 Device Setup Guide and Console V2 User Manual, or Console V1 Device Setup Guide and Console V1 User Manual.

Create and deploy artifacts

The reference solution requires a model and an edge application to be deployed on the device. In some cases, the process for obtaining and deploying these artifacts varies between versions of Console for AITRIOS, V1 and V2. When applicable, such difference is directly indicated in further description.

AI Model

• In order to obtain the model, follow the description in ai_model/model.md of this repository. The AI model is obtained as a packerOut.zip or yoloXXX_imx.onnx file and needs to be imported to the AITRIOS Console and then deployed to the device.
• To import the AI model into Console, follow the Console V2 Deployment Guide for PyTorch AI Model or Console V1 PyTorch Model Deployment Guide.

Edge Application

• In order to get the Edge Application, follow the description in edge_app/edge_app_v1.md of this repository for V1 or the description in edge_app/edge_app_v2.md for V2.

  NOTE
  Note that for V1 the Edge Application artifact would be a single file of type .wasm/.aot, while V2 would result in an Edge Application Package.

• For instructions on how to import an Edge Application into Console and deploy it to the device, check the Console V2 User Manual or Console V1 User Manual.

Configure the Edge Application

v1: Import and Bind Command Parameter File

To control the device's inference operation, a Command Parameter file must be bound to the device. While a user can provide an initial command parameter file themselves through the Console, it is not necessary since the current application automatically creates a new Command Parameter file and binds it to a selected device in case when no initial is available. Console User Manual provides instructions on importing Command Parameter file into Console, and on binding Command Parameter to the device.

v2: update Edge Appplication state with DTDL

Starting from version 2, AITRIOS is using DTDL (short for "Digital Twins Definition Language") to define the Edge Application Interface. While there are numerous configurations, the reference solution requires only some of them to be specified. In particular, one has to correctly configure the port settings that define methods and a storage for the upload of the edge application output. They correspond to the content of the field edge_app.common_settings.port_settings.

To do so, you first need to obtain the ID of your project (YOUR_PROJECT_ID) from AITRIOS Portal and specify it in the "storage_name" for the "input_tensor" field. You can find YOUR_PROJECT_ID in the browser url on the website of the portal, after logging in and selecting a project. The browser url takes the form PORTAL_URL/project/YOUR_PROJECT_ID/marketplace/product.

You can then apply the following configuration to the Edge Application deployed on Console.

{
  "edge_app": {
    "common_settings": {
      "port_settings": {
        "metadata": {
          "path": "",
          "method": 0,
          "enabled": true,
          "endpoint": "",
          "storage_name": ""
        },
        "input_tensor": {
          "path": "YOUR_DEVICE_ID/image/FOLDER_NAME",
          "method": 1,
          "enabled": true,
          "endpoint": "",
          "storage_name": "YOUR_PROJECT_ID"
        }
      }
  }
}

Note
For general details on the Edge Application interface, check the Edge Application Interface JSON Specification page on the AITRIOS developer website. Pay attention to the port_settings subsection
For details on the custom settings for the zone detection Edge Application, check its README.md.

Check prerequisites for executing the Reference Solution

Ensure you have the following installed on your system:

Docker and Docker Compose

These are necessary to run the application in a containerized environment.

Below are the links to the instructions on how to install both Docker and Docker Compose in most common operating systems:

• Linux (Ubuntu): See instructions here.
• Linux (other distributions): See instructions here.
• Windows: See instructions here.
• macOS: See instructions here.

Start Application on GitHub Codespaces

1. Clone or Fork: Clone or fork this repository into your GitHub account.

2. Create a Codespace:

   • Navigate to the cloned/forked repository.
   • Click the "+" button to create a codespace.
   • For more details, see How to Create a Codespace.

3. Access and Monitor:

   • The application will start automatically once the setup completes.
   • To view deployment logs, press Ctrl + Shift + P (or Cmd + Shift + P on Mac) and select "View Creation Logs."

4. Open Application:

   • Use the URL displayed under frontend port 3000 in the "Ports" tab to access your application.

Local Installation

Clone the repository:

git clone git@github.com:SonySemiconductorSolutions/aitrios-sample-application-reference.git
cd aitrios-sample-application-reference/backend

Running the Application

To run the full application, use the provided Docker Compose setup:

docker compose --env-file backend/.env up

To run only the backend or frontend separately, execute:

docker compose --env-file backend/.env up backend

docker compose up frontend

Note
This approach doesn't support debugging. To run app in the debugging mode you should start frontend and backend separately.

Frontend

For more detailed usage instructions, see the frontend README.

Backend

For more detailed usage instructions, see the backend README.

Trademark

• Read This First

Notice

Security

Before using Codespaces, please read the Site Policy of GitHub and understand the usage conditions.