From d5be1db424aa421f305720e5351df9376281837b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christopher=20M=C3=A9ndez?= <49886387+mcmchris@users.noreply.github.com> Date: Wed, 21 Aug 2024 10:48:58 -0400 Subject: [PATCH 1/3] Nicla Vision Added to requirements --- .../tutorials/speech-recognition-engine/content.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md b/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md index 2c503e7d00..86e9305626 100644 --- a/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md +++ b/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md @@ -57,6 +57,7 @@ In case you are interested in unlocking the full potential of the tool, the tuto To use the Arduino Speech Recognition Engine, you will need one of the following boards: - [Arduino Portenta H7 (any variant)](https://store.arduino.cc/portenta-h7) + Portenta Vision Shield ([LoRa](https://store.arduino.cc/products/arduino-portenta-vision-shield-lora%C2%AE) or [Ethernet](https://store.arduino.cc/products/arduino-portenta-vision-shield-ethernet)) +- [Arduino Nicla Vision](https://store.arduino.cc/products/nicla-vision) - [Arduino Nano 33 BLE Sense Rev 1](https://store.arduino.cc/products/arduino-nano-33-ble-sense) - [Arduino Nano 33 BLE Sense Rev 2](https://store.arduino.cc/products/nano-33-ble-sense-rev2) - [Arduino Nano RP2040](https://store.arduino.cc/products/arduino-nano-rp2040-connect) From 326ed5be6b5aa32bba63648fe7d96c5eb182a2f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christopher=20M=C3=A9ndez?= <49886387+mcmchris@users.noreply.github.com> Date: Thu, 22 Aug 2024 17:27:07 -0400 Subject: [PATCH 2/3] chris modification --- .../speech-recognition-engine/content.md | 113 ++++++++++++++++-- 1 file changed, 104 insertions(+), 9 deletions(-) diff --git a/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md b/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md index 86e9305626..33a457ab23 100644 --- a/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md +++ b/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md @@ -134,7 +134,7 @@ Once everything is ready, click on the **submit** button to get your license, it }; ``` -**Remember to replace the g_lpdwLicense values with your license ones.** +***Remember to replace the `g_lpdwLicense` values with your license ones.*** Now switch back to the `VoiceRecognition` tab and upload the sketch. @@ -150,7 +150,7 @@ Please note that the sketch and license that you are using are a "demo" version ### Customized Commands -To expand the features and human interaction of your projects, you can create your own voice commands with the Arduino Speech Recognition Engine. In this section, you will learn about how to use the _Cyberon Model Configuration_ webpage to create a new project with custom voice commands. +To expand the features and human interaction of your projects, you can create your own voice commands with the Arduino Speech Recognition Engine. In this section, you will learn about how to use the [Cyberon Model Configuration](https://tool.cyberon.com.tw/ArduinoDSpotterAuth/CTMain.php) webpage to create a new project with custom voice commands. ***These steps are compatible with the trial license, used in this tutorial, and the paid license*** @@ -170,18 +170,18 @@ Click next, you will see a new page to: ***Warning: each project is bound to a single Arduino board with the declared Serial Number. If you would like to use the Arduino Speech Recognition Engine on another Arduino board, you need to create a new project from scratch and assign it to the new Serial Number.*** -To create a new project first you need to select the desired language for the speech recognition. Once is set, click **create**. +To create a new project first you need to select the desired language for the speech recognition. Once is set, click **Create**. ![Cyberon, New Project](assets/newProject.png) Now you need to configure the following steps to create a new trigger: * Create the **Input Trigger word**, for example "Hey Arduino". - The **Input Trigger word** will trigger the device, to let the board know that you are going to say a command after that. - ![Cyberon Adding the Input trigger](assets/newProjectTrigger.png) + The **Input Trigger word** will trigger the device to let the board know that you are going to say a command after that. +![Cyberon Adding the Input trigger](assets/newProjectTrigger.png) * Add the **Command** list. - These commands will be used to do tasks on your sketch. For example, if you have a command which is "Play", later you will be able to get that command and proceed with some job inside the sketch easily. - ![Cyberon Adding the Command list](assets/newProjectCommands.png) + These commands will be used to do tasks on your sketch. For example, if you have a command which is `share`, later you will be able to get that command and proceed with some job inside the sketch easily. +![Cyberon Adding the Command list](assets/newProjectCommands.png) * The next step is just to confirm that the data written is correct. Click on **Next** to finish. On the next page you will see all the configurations already set. Check it out to confirm that everything is correct. In case something is wrong, click on **Back** to fix it. @@ -202,9 +202,9 @@ On your sketch directory, paste the files you have got in your e-mail inbox befo * `CybLicense_.h` * `Model_.h` -Now to implement the **Input Trigger Command** and the **Command List** open the sketch and change the following `#include` +Now to implement the **Input Trigger Command** and the **Command List** open the sketch and change the following headers with the downloaded ones: -```cpp +```arduino ... #include "CybLicense.h" -> #include "CybLicense_.h" @@ -221,6 +221,101 @@ Now to implement the **Input Trigger Command** and the **Command List** open the At this point, the project is set to be used. Upload the sketch and open the Serial Monitor. Now you can test your new **Input Trigger Word** and the **Command** list that you have created. Pronounce the new trigger words out loud, you will see the recognized words on the **Serial Monitor**. +#### Modify the Example Codes + +If you want to execute custom actions with the **trigger** and **commands** you defined, you can do it inside the `VRCallback` function. + +First you need to know your trigger and commands IDs. You can find them listed in your Serial Monitor, in this case are the following: + +| **Keyword** | **ID** | +| :---------: | :----: | +| Hey Arduino | 100 | +| read | 10000 | +| upload | 10001 | +| share | 10002 | + +Here is the `VRCallback` function modified so you can easily give functionality to the commands detection: + +```arduino +// Callback function for VR engine +void VRCallback(int nFlag, int nID, int nScore, int nSG, int nEnergy) +{ + if (nFlag==DSpotterSDKHL::InitSuccess) + { + //ToDo + } + else if (nFlag==DSpotterSDKHL::GetResult) + { + /* + When getting an recognition result, + the following index and scores are also return to the VRCallback function: + nID The result command id + nScore nScore is used to evaluate how good or bad the result is. + The higher the score, the more similar the voice and the result command are. + nSG nSG is the gap between the voice and non-command (Silence/Garbage) models. + The higher the score, the less similar the voice and non-command (Silence/Garbage) models are. + nEnergy nEnergy is the voice energy level. + The higher the score, the louder the voice. + */ + //ToDo + switch(nID) + { + case 100: + Serial.println("The Trigger was heard"); + // Add your own code here + break; + case 10000: + Serial.println("The Command -read- was heard"); + // Add your own code here + break; + case 10001: + Serial.println("The Command -upload- was heard"); + // Add your own code here + break; + case 10002: + Serial.println("The Command -share- was heard"); + // Add your own code here + break; + default: + break; + } + + } + else if (nFlag==DSpotterSDKHL::ChangeStage) + { + switch(nID) + { + case DSpotterSDKHL::TriggerStage: + LED_RGB_Off(); + LED_BUILTIN_Off(); + break; + case DSpotterSDKHL::CommandStage: + LED_BUILTIN_On(); + break; + default: + break; + } + } + else if (nFlag==DSpotterSDKHL::GetError) + { + if (nID == DSpotterSDKHL::LicenseFailed) + { + //Serial.print("DSpotter license failed! The serial number of your device is "); + //Serial.println(DSpotterSDKHL::GetSerialNumber()); + } + g_oDSpotterSDKHL.Release(); + while(1);//hang loop + } + else if (nFlag == DSpotterSDKHL::LostRecordFrame) + { + //ToDo + } +} +``` +We added a `switch...case` that evaluates the `nID` variable and compares it with the different IDs. +Inside each case add your custom code. + + #### Unlock Limitations (License) In case you want to further customize your model or add additional commands or trigger words, we have the right licenses for you: From 5f62d89034d9a43284cff75685573b877d664578 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christopher=20M=C3=A9ndez?= <49886387+mcmchris@users.noreply.github.com> Date: Fri, 23 Aug 2024 10:02:54 -0400 Subject: [PATCH 3/3] finished --- .../speech-recognition-engine/content.md | 29 ++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md b/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md index 33a457ab23..412abef41d 100644 --- a/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md +++ b/content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md @@ -313,7 +313,34 @@ void VRCallback(int nFlag, int nID, int nScore, int nSG, int nEnergy) } ``` We added a `switch...case` that evaluates the `nID` variable and compares it with the different IDs. -Inside each case add your custom code. +Inside each case add your custom code to be executed with the trigger or commands. + +```arduino +... + switch(nID) + { + case 100: + Serial.println("The Trigger was heard"); + // Add your own code here + break; + case 10000: + Serial.println("The Command -read- was heard"); + // Add your own code here + break; + case 10001: + Serial.println("The Command -upload- was heard"); + // Add your own code here + break; + case 10002: + Serial.println("The Command -share- was heard"); + // Add your own code here + break; + default: + break; + } +... + +``` #### Unlock Limitations (License)