Skip to content

Commit 30e85b0

Browse files
Tinyu-Zhaolbuque
Tinyu-Zhao
authored and
lbuque
committed
lib/unit: Add Unit ASR Support.
Signed-off-by: Tinyu-Zhao <[email protected]>
1 parent 846870b commit 30e85b0

File tree

8 files changed

+571
-1
lines changed

8 files changed

+571
-1
lines changed

‎docs/en/refs/unit.asr.ref‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
2+
.. |ASRUnit| image:: https://m5stack-doc.oss-cn-shenzhen.aliyuncs.com/635/U914_01.webp
3+
:target: https://docs.m5stack.com/en/unit/asr
4+
:height: 200px
5+
:width: 200px
6+
7+
.. |init.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/init.png
8+
.. |_handler.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/_handler.png
9+
.. |get_received_status.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/get_received_status.png
10+
.. |send_message.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/send_message.png
11+
.. |get_current_raw_message.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/get_current_raw_message.png
12+
.. |get_current_command_word.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/get_current_command_word.png
13+
.. |get_current_command_num.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/get_current_command_num.png
14+
.. |get_command_handler.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/get_command_handler.png
15+
.. |add_command_word.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/add_command_word.png
16+
.. |event.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/event.png
17+
.. |remove_command_word.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/remove_command_word.png
18+
.. |search_command_num.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/search_command_num.png
19+
.. |search_command_word.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/search_command_word.png
20+
.. |get_command_list.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/get_command_list.png
21+
22+
.. |example.png| image:: https://static-cdn.m5stack.com/mpy_docs/unit/asr/example.png
23+
24+
.. |asr_cores3_example.m5f2| raw:: html
25+
26+
<a
27+
href="https://uiflow2.m5stack.com/?example=https://raw.githubusercontent.com/m5stack/uiflow-micropython/develop/examples/unit/asr/asr_cores3_example.m5f2"
28+
target="_blank"
29+
>
30+
asr_cores3_example.m5f2
31+
</a>
32+

‎docs/en/units/asr.rst‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
2+
ASR Unit
3+
=========
4+
.. sku:U194
5+
.. include:: ../refs/unit.asr.ref
6+
7+
**Unit ASR** is an **AI** offline speech recognition unit, featuring the built-in AI smart offline speech module **CI-03T**. This unit offers powerful functions such as speech recognition, voiceprint recognition, speech enhancement, and speech detection. It supports AEC (Acoustic Echo Cancellation) to effectively eliminate echoes and noise interference, improving the accuracy of speech recognition. Additionally, it supports mid-speech interruption, allowing for flexible interruption during the recognition process and quick response to new commands. The product is pre-configured with wake-up words and feedback commands at the factory. The device uses **UART** serial communication for data transmission and also supports waking up the device via UART or voice keywords. This unit supports user customization of the **wake-up** recognition word and can recognize up to 300 command words. It is equipped with a **microphone** for clear audio capture and includes a **speaker** for high-quality audio feedback. This product is widely used in AI assistants, smart homes, security monitoring, automotive systems, robotics, smart hardware, healthcare, and other fields, making it an ideal choice for realizing smart voice interactions.
8+
9+
Support the following products:
10+
11+
|ASRUnit|
12+
13+
Micropython Example:
14+
15+
.. literalinclude:: ../../../examples/unit/asr/asr_cores3_example.py
16+
:language: python
17+
:linenos:
18+
19+
UIFLOW2 Example:
20+
21+
|example.png|
22+
23+
.. only:: builder_html
24+
25+
|asr_cores3_example.m5f2|
26+
27+
class ASRUnit
28+
-------------
29+
30+
Constructors
31+
------------
32+
33+
.. class:: ASRUnit(id, port)
34+
35+
Initialize the ASRUnit object with UART configuration and set up the command handler.
36+
37+
:param int id: The UART port ID for communication, default is 1.
38+
:param port: A list or tuple containing the TX and RX pins for UART communication.
39+
40+
UIFLOW2:
41+
42+
|init.png|
43+
44+
45+
Methods
46+
-------
47+
48+
.. method:: ASRUnit.get_received_status()
49+
50+
Get the status of the received message.
51+
52+
:returns: True if a message is received, False otherwise.
53+
54+
UIFLOW2:
55+
56+
|get_received_status.png|
57+
58+
.. method:: ASRUnit.send_message(command_num)
59+
60+
Send a message with a specified command number via UART.
61+
62+
:param int command_num: The command number to send in the message.
63+
64+
UIFLOW2:
65+
66+
|send_message.png|
67+
68+
.. method:: ASRUnit.get_current_raw_message()
69+
70+
Get the raw message received in hexadecimal format.
71+
72+
73+
:returns: The raw message as a string in hexadecimal format.
74+
75+
UIFLOW2:
76+
77+
|get_current_raw_message.png|
78+
79+
.. method:: ASRUnit.get_current_command_word()
80+
81+
Get the command word corresponding to the current command number.
82+
83+
84+
:returns: The command word as a string.
85+
86+
UIFLOW2:
87+
88+
|get_current_command_word.png|
89+
90+
.. method:: ASRUnit.get_current_command_num()
91+
92+
Get the current command number.
93+
94+
:returns: The command number.
95+
96+
UIFLOW2:
97+
98+
|get_current_command_num.png|
99+
100+
.. method:: ASRUnit.get_command_handler()
101+
102+
Check if the current command has an associated handler.
103+
104+
:returns: True if a handler exists for the current command,
105+
106+
UIFLOW2:
107+
108+
|get_command_handler.png|
109+
110+
.. method:: ASRUnit.add_command_word(command_num, command_word, event_handler)
111+
112+
Add a new command word and its handler to the command list.
113+
114+
:param int command_num: The command number (must be between 0 and 255).
115+
:param str command_word: The command word to associate with the command number.
116+
:param event_handler: An optional event handler function to be called for the command.
117+
118+
UIFLOW2:
119+
120+
|add_command_word.png|
121+
122+
|event.png|
123+
124+
.. method:: ASRUnit.remove_command_word(command_word)
125+
126+
Remove a command word from the command list by its word.
127+
128+
:param str command_word: The command word to remove.
129+
130+
UIFLOW2:
131+
132+
|remove_command_word.png|
133+
134+
.. method:: ASRUnit.search_command_num(command_word)
135+
136+
Search for the command number associated with a command word.
137+
138+
:param str command_word: The command word to search for.
139+
140+
:returns: The command number if found, otherwise -1.
141+
142+
UIFLOW2:
143+
144+
|search_command_num.png|
145+
146+
.. method:: ASRUnit.search_command_word(command_num)
147+
148+
Search for the command word associated with a command number.
149+
150+
:param int command_num: The command number to search for.
151+
152+
:returns: The command word if found, otherwise "Unknown command
153+
154+
UIFLOW2:
155+
156+
|search_command_word.png|
157+
158+
.. method:: ASRUnit.get_command_list()
159+
160+
Get the list of all commands and their associated handlers.
161+
162+
:returns: A dictionary of command numbers and their corresponding command words and handlers.
163+
164+
UIFLOW2:
165+
166+
|get_command_list.png|
167+
168+
.. method:: ASRUnit.check_tick_callback()
169+
170+
Check if a handler is defined for the current command and schedule its execution.

‎docs/en/units/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ Unit
1212
ain4.rst
1313
angle.rst
1414
angle8.rst
15+
asr.rst
1516
bps.rst
1617
button.rst
1718
buzzer.rst

‎examples/unit/asr/asr_cores3_example.m5f2‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"version":"V2.0","versionNumber":"V2.2.1","type":"cores3","components":[{"name":"screen","type":"screen","layer":0,"screenId":"builtin","screenName":"","id":"__cores3_screen","createTime":1738984206533,"x":0,"y":0,"width":320,"height":240,"backgroundColor":"#222222","size":0,"isSelected":true},{"name":"title0","type":"title","layer":1,"screenId":"builtin","screenName":"","id":"lHDFJlvUVPDKu#oE","createTime":1738987578355,"x":0,"y":0,"color":"#ffffff","backgroundColor":"#0000FF","text":"UnitASR M5CoreS3 Example","textOffset":3,"font":"Widgets.FONTS.DejaVu18","isSelected":false},{"name":"label0","type":"label","layer":2,"screenId":"builtin","screenName":"","id":"lh3@VMN=YN%jIUPQ","createTime":1738987596235,"x":0,"y":51,"color":"#ffffff","backgroundColor":"#222222","text":"msg:","engine":"gfx","font":"Widgets.FONTS.DejaVu18","rotation":0,"isSelected":false},{"name":"label1","type":"label","layer":3,"screenId":"builtin","screenName":"","id":"qb7`E!xf%-#yrXYt","createTime":1738987598852,"x":0,"y":93,"color":"#ffffff","backgroundColor":"#222222","text":"rec cmd num:","engine":"gfx","font":"Widgets.FONTS.DejaVu18","rotation":0,"isSelected":false},{"name":"label2","type":"label","layer":4,"screenId":"builtin","screenName":"","id":"p8On-oS$5RBen$tz","createTime":1738987601502,"x":0,"y":134,"color":"#ffffff","backgroundColor":"#222222","text":"rec cmd word:","engine":"gfx","font":"Widgets.FONTS.DejaVu18","rotation":0,"isSelected":false},{"name":"label3","type":"label","layer":5,"screenId":"builtin","screenName":"","id":"mVd*nC3=WQFN4CHw","createTime":1738987604932,"x":0,"y":178,"color":"#ffffff","backgroundColor":"#222222","text":"rec cmd handler state:","engine":"gfx","font":"Widgets.FONTS.DejaVu18","rotation":0,"isSelected":false}],"resources":[{"hardware":["hardware_button","hardware_pin_button","imu","speaker","touch","als","mic"]},{"unit":["unit_asr"]}],"units":[{"type":"unit_asr","name":"asr_0","portList":["A","B","C","Custom"],"portType":"A","userPort":[22,21],"id":"jKk&28cMoM6zop1-","createTime":1739442702079,"initBlockId":"2H(Y/6[$Y.^DcbG1({HV"}],"hats":[],"bases":[],"i2cs":[],"blockly":"<block type=\"basic_on_setup\" id=\"setup_block\" deletable=\"false\" x=\"50\" y=\"50\"><mutation isBegin=\"true\"></mutation><field name=\"UPDATEOP\">true</field><statement name=\"FUNC\"><block type=\"system_m5_begin\" id=\"system_m5_begin\"><next><block type=\"unit_asr_init\" id=\"2H(Y/6[$Y.^DcbG1({HV\"><field name=\"NAME\">asr_0</field><value name=\"UART\"><shadow type=\"unit_asr_id_option\" id=\"VHXDfq_R!LV=6Rt7smUB\"><field name=\"VALUE\">2</field></shadow></value><next><block type=\"text_print\" id=\"MAkprDFcP)8C+ZSE+5+o\"><value name=\"TEXT\"><shadow type=\"text\" id=\"*9:/ve|MTjrM]k~2sXov\"><field name=\"TEXT\">hello M5</field></shadow><block type=\"unit_asr_search_command_num\" id=\"vmD1PWn.M=6+p`3L~e?b\"><field name=\"NAME\">asr_0</field><value name=\"VALUE\"><shadow type=\"text\" id=\"NryDWH9p,NZs2*i-F4A0\"><field name=\"TEXT\">hello</field></shadow></value></block></value><next><block type=\"text_print\" id=\"r`@.Ic^qtJ2rVGNmGGck\"><value name=\"TEXT\"><shadow type=\"text\" id=\"*9:/ve|MTjrM]k~2sXov\"><field name=\"TEXT\">hello M5</field></shadow><block type=\"unit_asr_search_command_word\" id=\"`6u,M{I8Vx8(?*P8(BO-\"><field name=\"NAME\">asr_0</field><value name=\"VALUE\"><shadow type=\"math_number\" id=\"K?#e0]wkml^|I{-BRq-!\"><mutation max=\"Infinity\" min=\"-Infinity\" precision=\"0\"></mutation><field name=\"NUM\">0x32</field></shadow></value></block></value><next><block type=\"unit_asr_add_command_word\" id=\"})a6y0OFLy/E:iV)xWh=\"><mutation wordValue=\"hello\"></mutation><field name=\"NAME\">asr_0</field><field name=\"WORD\">hello</field><value name=\"NUM\"><shadow type=\"math_number\" id=\"W/-=n%i1t-an0@F(nK5Y\"><mutation max=\"Infinity\" min=\"-Infinity\" precision=\"0\"></mutation><field name=\"NUM\">0x32</field></shadow></value></block></next></block></next></block></next></block></next></block></statement></block><block type=\"basic_on_loop\" id=\"loop_block\" deletable=\"false\" x=\"50\" y=\"310\"><mutation isUpdate=\"true\"></mutation><field name=\"UPDATEOP\">true</field><statement name=\"FUNC\"><block type=\"system_m5_update\" id=\"system_m5_update\"><next><block type=\"controls_if\" id=\"42hyED.UX0ra^2hIawfV\"><value name=\"IF0\"><block type=\"unit_asr_get_received_status\" id=\"^tl7o2;s01Pnkg9B}8PR\"><field name=\"NAME\">asr_0</field></block></value><statement name=\"DO0\"><block type=\"label_set_text\" id=\"+j5TmnvhX*Qn1*;Hy$]y\"><field name=\"NAME\">label0</field><value name=\"TEXT\"><shadow type=\"text\" id=\"5:n4~i:8iNIE5WH6q~sm\" disabled=\"true\"><field name=\"TEXT\">Label</field></shadow><block type=\"text_add_str\" id=\"Si:^Fw.flP65;;=Z%+Lj\"><value name=\"VALUE1\"><shadow type=\"text\" id=\"A-C5FNQJ%[FW?:/KxX(Q\"><field name=\"TEXT\">msg:</field></shadow></value><value name=\"VALUE2\"><block type=\"unit_asr_get_current_raw_message\" id=\"m9vZyFLx=.ucD9L:0pe0\"><field name=\"NAME\">asr_0</field></block></value></block></value><next><block type=\"label_set_text\" id=\"t2g_.)rt?Bw-OdtC9V%[\"><field name=\"NAME\">label1</field><value name=\"TEXT\"><shadow type=\"text\" id=\"5:n4~i:8iNIE5WH6q~sm\"><field name=\"TEXT\">Label</field></shadow><block type=\"text_add_str\" id=\"^UYvPnn%TT:T0sgdp]Yr\"><value name=\"VALUE1\"><shadow type=\"text\" id=\"h}|3[X*`2s7.J^3J`-g4\"><field name=\"TEXT\">rec cmd num:</field></shadow></value><value name=\"VALUE2\"><block type=\"unit_asr_get_current_command_num\" id=\"wLnH=u[VK@*LM}V_GD~X\"><field name=\"NAME\">asr_0</field></block></value></block></value><next><block type=\"label_set_text\" id=\"eIvB}F.Br4gFMgfBkzru\"><field name=\"NAME\">label2</field><value name=\"TEXT\"><shadow type=\"text\" id=\"f,b}wonBAU#J!7P8q|rv\"><field name=\"TEXT\">Label</field></shadow><block type=\"text_add_str\" id=\"koY5r8`xXbQYrx~ZM1LH\"><value name=\"VALUE1\"><shadow type=\"text\" id=\"*9-nV#y37,%DB*t%N]38\"><field name=\"TEXT\">rec cmd word:</field></shadow></value><value name=\"VALUE2\"><block type=\"unit_asr_get_current_command_word\" id=\"!S#!nt9)N;6q9;l`ipL:\"><field name=\"NAME\">asr_0</field></block></value></block></value><next><block type=\"label_set_text\" id=\"Hf1j|d!=9~Mn![0hGSJp\"><field name=\"NAME\">label3</field><value name=\"TEXT\"><shadow type=\"text\" id=\"w{ooZcY8:40ef0q/#pIJ\"><field name=\"TEXT\">Label</field></shadow><block type=\"text_add_str\" id=\"?1CWEq;n[IJlqGY5!D%C\"><value name=\"VALUE1\"><shadow type=\"text\" id=\"gk;f=mYO}4EI@0]:cL-:\"><field name=\"TEXT\">rec cmd handler state:</field></shadow></value><value name=\"VALUE2\"><block type=\"unit_asr_get_command_handler\" id=\"*!d2=[`y-IYis6F`b7G9\"><field name=\"NAME\">asr_0</field></block></value></block></value></block></next></block></next></block></next></block></statement></block></next></block></statement></block><block type=\"unit_asr_event\" id=\"+Ux*CDyrg@1S)vtrN4{X\" x=\"50\" y=\"650\"><field name=\"NAME\">asr_0</field><value name=\"WORD\"><shadow type=\"text\" id=\"tpbJ7sXM(Hgq5Y^RjdVZ\"><field name=\"TEXT\">hello</field></shadow></value><statement name=\"FUNC\"><block type=\"text_print\" id=\"*%%Q8puF.vW|z{2+Fbx`\"><value name=\"TEXT\"><shadow type=\"text\" id=\")Y%O0xWE=.d7:-wlCwj2\"><field name=\"TEXT\">Rec Hello</field></shadow></value></block></statement></block>","screen":[{"simulationName":"Built-in","type":"builtin","width":320,"height":240,"scale":0.78,"screenName":"","blockId":"","screenColorType":0,"id":"builtin","createTime":1738984206533}],"logicWhenNum":0,"customList":[]}

‎examples/unit/asr/asr_cores3_example.py‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
2+
#
3+
# SPDX-License-Identifier: MIT
4+
5+
import os, sys, io
6+
import M5
7+
from M5 import *
8+
from unit import ASRUnit
9+
10+
11+
title0 = None
12+
label0 = None
13+
label1 = None
14+
label2 = None
15+
label3 = None
16+
asr_0 = None
17+
18+
19+
def asr_0_hello_event(args):
20+
global title0, label0, label1, label2, label3, asr_0
21+
print("Rec Hello")
22+
23+
24+
def setup():
25+
global title0, label0, label1, label2, label3, asr_0
26+
27+
M5.begin()
28+
Widgets.fillScreen(0x222222)
29+
title0 = Widgets.Title(
30+
"UnitASR M5CoreSe Example", 3, 0xFFFFFF, 0x0000FF, Widgets.FONTS.DejaVu18
31+
)
32+
label0 = Widgets.Label("msg:", 0, 51, 1.0, 0xFFFFFF, 0x222222, Widgets.FONTS.DejaVu18)
33+
label1 = Widgets.Label("rec cmd num:", 0, 93, 1.0, 0xFFFFFF, 0x222222, Widgets.FONTS.DejaVu18)
34+
label2 = Widgets.Label(
35+
"rec cmd word:", 0, 134, 1.0, 0xFFFFFF, 0x222222, Widgets.FONTS.DejaVu18
36+
)
37+
label3 = Widgets.Label(
38+
"rec cmd handler state:", 0, 178, 1.0, 0xFFFFFF, 0x222222, Widgets.FONTS.DejaVu18
39+
)
40+
41+
asr_0 = ASRUnit(2, port=(1, 2))
42+
print(asr_0.search_command_num("hello"))
43+
print(asr_0.search_command_word(0x32))
44+
asr_0.add_command_word(0x32, "hello", asr_0_hello_event)
45+
46+
47+
def loop():
48+
global title0, label0, label1, label2, label3, asr_0
49+
M5.update()
50+
if asr_0.get_received_status():
51+
label0.setText(str((str("msg:") + str((asr_0.get_current_raw_message())))))
52+
label1.setText(str((str("rec cmd num:") + str((asr_0.get_current_command_num())))))
53+
label2.setText(str((str("rec cmd word:") + str((asr_0.get_current_command_word())))))
54+
label3.setText(str((str("rec cmd handler state:") + str((asr_0.get_command_handler())))))
55+
56+
57+
if __name__ == "__main__":
58+
try:
59+
setup()
60+
while True:
61+
loop()
62+
except (Exception, KeyboardInterrupt) as e:
63+
try:
64+
from utility import print_error_msg
65+
66+
print_error_msg(e)
67+
except ImportError:
68+
print("please update to latest firmware")

‎m5stack/libs/unit/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
"AMeterUnit": "ameter",
1414
"AngleUnit": "angle",
1515
"Angle8Unit": "angle8",
16+
"ASRUnit": "asr",
1617
"BLDCDriverUnit": "bldc_driver",
1718
"BPSUnit": "bps",
1819
"ButtonUnit": "button",
@@ -73,7 +74,6 @@
7374
"LIMITUnit": "limit",
7475
"LoRaE220433Unit": "lora_e220_433",
7576
"LoRaE220JPUnit": "lora_e220_jp",
76-
"LoRaWANUnit_RUI3": "lorawan_rui3",
7777
"LoRaWANUnit": "lorawan",
7878
"MIDIUnit": "midi",
7979
"MiniOLEDUnit": "minioled",

0 commit comments

Comments
 (0)