You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/ApplicationUserGuide.md
+27-31Lines changed: 27 additions & 31 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,30 +14,29 @@ When the ndp120 detects a ML feature in the data it interrupts the RA6 MCU and p
14
14
1. Optionally send a MQTT telemetry message to the target cloud implementation if enabled (see the ```[Cloud Connectivity]``` section below).
15
15
16
16
## Data Collection for Developing/Testing New ML Models
17
-
Avnet, Syntiant, Renesas and Edge Impulse have all worked together to enable RASynBoard ML model development in [Edge Impulse Studio](https://www.edgeimpulse.com/). Edge Impulse Studio is a free on-line toolkit that enables data ingestion from development boards that can be used to develop/test/deploy ML models. Additionally, Edge Impulse will generate the \*.syspkg files that can be used on the RASynBoard with the OOB application.
17
+
Avnet, Syntiant, Renesas and Edge Impulse have all worked together to enable RASynBoard ML model development in [Edge Impulse Studio](https://www.edgeimpulse.com/). Edge Impulse Studio is a free on-line toolkit that enables data ingestion from development boards that can be used to develop/test/deploy ML models. Additionally, Edge Impulse can generate the \*.syspkg files that can be used on the RASynBoard with the OOB application.
18
18
19
-
**Note: RASynBoard Edge Impulse support is scheduled to be released in late October 2023**
20
-
21
-
**Note: Edge Impulse will release a custom RASynBoard application to interface with the Edge Impulse Studio to develop/test/deploy/test on device in conjunction with the late October 2023 release**
19
+
**Note: RASynBoard Edge Impulse support has been released!
22
20
23
21
The OOB application also implements multiple ways to collect data from the on-board sensors
24
22
1. Audio data can be recorded to the microSD card
25
23
1. 6-Axis IMU data can be written to the microSD card and/or streamed to the debug UART
26
24
1. (Future enablement) Using additional external sensors
27
25
28
26
### Flash microSD card images to core board SPI Flash
29
-
When developing a RASynBoard ML application it's common to use the microSD card to manage the config.ini and \*.synpkg files. However if users get to a point where they want to deploy the RASynBoard core board without the I/O board, they can move the config.ini and \*.synpkg files to the on-board SPI flash by pressing the user button for > 3 seconds.
27
+
When developing a RASynBoard ML application it's common to use the microSD card to manage the config.ini and \*.synpkg files. However if users get to a point where they want to deploy the RASynBoard core board without the I/O board, they can move the config.ini settings and \*.synpkg files to the on-board SPI flash by pressing the user button for > 3 seconds.
30
28
31
29
Once the > 3 second button press is detected the application will . . .
32
30
1. Erase the SPI Flash region that holds the config.ini and \*.synpkg files
33
31
1. Concatenate all the files into a single binary blob
34
32
1. Flash the binary blob to the SPI Flash
33
+
1. Copy all core board only relevant config.ini settings to SPI Flash
35
34
36
35
Once the images have been moved to SPI Flash, the application will run from SPI Flash if . . .
37
36
1. The microSD card is removed from the I/O board
38
37
1. The I/O board is separated from the core board
39
38
40
-
![](./assets/writeToFlash.jpg"")
39
+
![](./assets/images/writeToFlash.jpg"")
41
40
42
41
**Note: Use caution when separating and re-attaching the core board from/to the I/O board as the interconnect pins have a fine pitch and are easy to damage.**
43
42
@@ -52,6 +51,8 @@ The OOB application reads application configuration data from the config.ini fil
52
51
### Syntiant Binary Images
53
52
The microSD card is also used to add syntiant binary images that are used by the NDP120. These files end with the \*.synpkg file extension. There are three \*.synpkg files for any given ML model. Using this approach, ML models can be added to the OOB application without having to rebuild the application. The user defines which \*.synpkg files get loaded onto the NDP120 with config.ini configuration changes (details below).
54
53
54
+
**Note:** The *.synpkg files are specific to the Syntiant SDK version. Always use the *.synpkg files included with any specific OOB release to maintain compatability
55
+
55
56
### Data Recording
56
57
The other main use of the microSD card is to capture training data from either the on-board TDK digital microphone, or the TDK 6-Axis IMU sensor.
57
58
@@ -81,7 +82,7 @@ The OOB features are configured by editing the config.ini file located on the mi
81
82
82
83
The application will output the current configuration.
83
84
84
-
![](./assets/configOutput.jpg"")
85
+
![](./assets/images/configOutput.jpg"")
85
86
86
87
## ndp120 Configuration
87
88
The ndp120 is configured by identifying the firmware images that will be loaded onto the device. There are three images referred to as the \*.synpkg files. These files are usually delivered together and should only be used together. New model files and ndp120 firmware can be copied to the microSD card and configured using the config.ini file. The application looks for the ndp120 files in the root directory of the microSD card.
@@ -101,12 +102,12 @@ Each ```[Function_x]``` block defines a description, and the three required ndp1
101
102
-```DSP``` refers to the dsp firmware that runs on the ndp120. This firmware does data pre-processing plus other functions.
102
103
-```DNN``` refers to the Nerual Network Parameters and ML model. This firmware implements the ML model.
103
104
104
-
![](./assets/ndpConfig.jpg"")
105
+
![](./assets/images/ndpConfig.jpg"")
105
106
106
107
## LED control
107
108
The ```[LED]``` block allows the user to assign different RGB LED colors to each inference index. When the ndp120 detects a feature in the data the inference results are passed to the application. The application uses the index of the inference result [0 - n] to identify how to light the RGB LED on the I/O board. The comments in the config.ini file associate each index with the 5-keyword model, but this feature works with any model that's loaded.
108
109
109
-
![](./assets/ledConfig.jpg"")
110
+
![](./assets/images/ledConfig.jpg"")
110
111
111
112
## Debug Print
112
113
The application can be configured to output debug messages to either the UART exposed on the PMOD connector (I/O Board), or the USB-C UART on the core board (the smaller board).
@@ -117,23 +118,23 @@ The application can be configured to output debug messages to either the UART ex
117
118
-```Port=2``` Outputs debug to the UART associated with the core board USB-C connector
118
119
- This selection allows developers to access debug when the core board is running without the I/O board
119
120
120
-
**Note the application must bring this UART connection on-line when the application starts. Because of this delay, most of the startup debug will not be displayed on the UART.**
121
+
**Note the application must bring this UART connection on-line when the application starts. Because of this delay, most of the startup debug will not be displayed on the UART. Release v1.4.0 added a serial command ```log``` that will output the early debug when using the core board USB-C com port**
121
122
122
-
![](./assets/debugConfig.jpg"")
123
+
![](./assets/images/debugConfig.jpg"")
123
124
124
125
## Recording Period
125
126
The application can record audio or IMU data.
126
127
127
128
1.```[Recording Period]-->Recording_Period``` defines how long the application collects/records data in units of seconds.
128
129
129
-
![](./assets/recordingPeriodConfig.jpg"")
130
+
![](./assets/images/recordingPeriodConfig.jpg"")
130
131
131
132
## 6-Axis IMU Data Stream Features
132
133
The OOB application can capture 6-Axis IMU data either to a \*.csv file on the microSD card, and/or stream data out the debug UART.
133
134
134
135
**Note: in order to capture 6-Axis IMU data the active ```[Function_x]``` block must contain ```Button_shift=imu``` within the block. This is true for any active ```[Function_x]``` block even if the block defines audio detection files.**
135
136
136
-
![](./assets/imuButtonShift.jpg"")
137
+
![](./assets/images/imuButtonShift.jpg"")
137
138
138
139
**Note: There is currenty a bug if you try to use the circular motion ```[Function_3]``` block to capture 6-Axis IMU data. If you need to capture 6-Axis IMU data please use the ```[Function_1]``` block. (AAGBT-82)**
139
140
@@ -147,20 +148,7 @@ The OOB application can capture 6-Axis IMU data either to a \*.csv file on the m
147
148
-```[IMU data stream]-->Print_to_terminal=1``` tells the application to stream IMU data to the debug UART for ```[Recording Period]-->Recording_Period``` seconds when the user button is pressed. This data can be used in conjunction with the [Edge Impulse Data Forwarder](./EdgeImpulseDataForwarder.md) to stream 6-Axis IMU data directly to Edge Impulse.
148
149
-```[IMU data stream]-->Print_to_terminal=0``` disables this feature
149
150
150
-
![](./assets/imuDataConfig.jpg"")
151
-
152
-
## Low Power Mode
153
-
The OOB application supports low-power mode. When in the low-power state, the RA6 device is put into sleep mode, and the DA16600 WiFi/BLE device is held in reset. The low power mode configuration section defines when the applicatoin enters and exits low power mode.
154
-
155
-
1.```[Low Power Mode]-->Power_Mode=x```
156
-
157
-
-```[Low Power Mode]-->Power_Mode=0```: When in ```Power_Mode=0``` the application will enter a low-power state when the application detects the "down" \<pause for < 3.6 seconds\> "down" sequence. The device will stay in a low-power state until any other inference detection event occurs. When the next event is detected the device will exit the low-power state not enter low-power state again until the "down" \<pause for < 3.6 seconds\> "down" sequence is detected.
158
-
159
-
-```[Low Power Mode]-->Power_Mode=1```: When in ```Power_Mode=1``` the application will enter low-power state on power-up and will come out of low-power state when any inference detection occurs. As soon as the new inference detection logic is completed, the application will automatically re-enter the low-power state.
160
-
161
-
**Note: The data recording feature is disabled when ```[Low Power Mode]-->Power_Mode=1```**
162
-
163
-
![](./assets/lowPowerConfig.jpg"")
151
+
![](./assets/images/imuDataConfig.jpg"")
164
152
165
153
## BLE Mode
166
154
@@ -171,7 +159,7 @@ The application can broadcast inference results over BLE that can be displayed o
171
159
-```[BLE Mode]-->BLE_Enabled=0```: Disable the BLE functionality (default)
172
160
-```[BLE Mode]-->BLE_Enabled=1```: Enable the BLE functionality
173
161
174
-
![](./assets/bleConfig.jpg"")
162
+
![](./assets/images/bleConfig.jpg"")
175
163
176
164
## Cloud Connectivity
177
165
The OOB application implements logic to send telemetry to a cloud provider. The first release connects to the Avnet IoTConnect Cloud solution built on AWS. In the future, the application will also interface directly with AWS or Azure.
@@ -185,7 +173,15 @@ There are multiple configuration items required to establish an MQTT connection.
185
173
1.```[WIFI]-->Access_Point=<Your AP SSID Here>```: Used to set your WiFi access point SSID
186
174
1.```[WIFI]-->Access_Point_Password=<Your AP Password Here>```: Used to set your WiFi Access Point password
187
175
188
-
![](./assets/wifiConfig.jpg"")
176
+
![](./assets/images/wifiConfig.jpg"")
177
+
178
+
### WiFi Access Point Configuration Source
179
+
Sometimes the user may want to dynamically change the WiFi Acess point configuration. For instance when deploying a solution on the core board only, it may be necessary to change the WiFi credentials when moving the device to a new location.
180
+
181
+
1.```[WIFI]-->Use_Config_AP_Details=0```: Don't use the config.ini WiFi configuration. This assumes that the user will use the free Renesas Wi-Fi Provisioning Tool avalible from your mobile device's App store
182
+
1. ```[WIFI]-->Use_Config_AP_Details=1```: Use the config.ini wifi access point configuration
The application currently only supports IoTConnect on AWS, however we have feature enhancement tickets written to add AWS and Azure implementations. The ```[Cloud Connectivity]-->Target_Cloud``` configuration item is used to define which service the device should connect to.
@@ -200,7 +196,7 @@ The application currently only supports IoTConnect on AWS, however we have featu
200
196
- ```[Cloud Connectivity]-->Target_Cloud=2```: Connect to AWS (Not currently implemented)
201
197
- ```[Cloud Connectivity]-->Target_Cloud=3```: Connect to Azure (Not currently implemented)
202
198
203
-
![](./assets/cloudConfiguratioin.jpg"")
199
+
204
200
205
201
### IoTConnect
206
202
Please see the [IoTConnect](./IoTConnect.md) document for details on how to . . .
@@ -212,5 +208,5 @@ Please see the [IoTConnect](./IoTConnect.md) document for details on how to . .
212
208
- ```[IoTConnect]-->Device_Unique_ID=<your device id here>```: Enter the Unique Device ID as defined in your IoTConnect account. - ```[IoTConnect]-->CPID=<your IoTConnect company ID here>```: Enter the 32 character company ID
213
209
- ```[IoTConnect]-->Environment=<your IoTConnect environment varaible```: Enter the environment here i.e., poc, prod
On the right side of the screen, there's a `Collect Data` pane. This is where you specify the label for the data being collected, along with the sample size. In this case, `circle` has been entered for the label and a sample size of 5000 ms will be collected.
149
-
![Collect Data Pane](assets/acq_collect_data.png"Collect Data Pane")
149
+
![Collect Data Pane](./assets/images/acq_collect_data.png"Collect Data Pane")
150
150
151
151
Click the `Start sampling` button and press the user button on the RASynBoard. At this point, the forwarder will start sampling the sensors on the device.
152
152
153
153
Once the sample period has completed, the data should be uploaded to Edge Impulse and you should see the samples appear:
Copy file name to clipboardExpand all lines: docs/EdgeImpulseUploader.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -86,23 +86,23 @@ Create a free account with Edge Impulse and create a new project if one isn't al
86
86
87
87
Press the user button to start printing IMU data to the serial port:
88
88
89
-
![](../images/UserButton.jpg"")
89
+
![](assets/images/UserButton.jpg"")
90
90
91
91
Navigate back to the projects page and select your project. In the project, select the `Data Acqusition page. Startup the CSV Wizard by clicking CSV Wizard:
You'll be presented with a pane to specify the type of data. Ensure that the time series option is slected and the option stating each column contains a reading and fill row contains a sample:
0 commit comments