You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 4, 2025. It is now read-only.
To use the Googla Coral USB Accelerator, first flash the Pi SD card with the image found in the [AIY Maker Kit](https://aiyprojects.withgoogle.com/maker/)
9
-
([Download as of 2022-08-05](https://github.com/google-coral/aiy-maker-kit-tools/releases/download/v20220518/aiy-maker-kit-2022-05-18.img.xz))
11
+
# Robotics development framework
12
+
This platform was built to modularize robotics development and experimentation with Python/C++ using Raspberry Pi/Jetson nano and Arduino.
10
13
11
-
(I attempted to install the required software from the coral [getting started guide](https://coral.ai/docs/accelerator/get-started#1-install-the-edge-tpu-runtime) but I was unable to get past an error relating to grpico "GLIBC_2.29' not found")
14
+
## Coral USB Accelerator
12
15
13
-
Alternatively, set Config.vision.tech to `opencv` for the original (slower) facial recognition. I am not updating this anymore so you may find some integration issues.
16
+
To use the Googla Coral USB Accelerator, first reprint the Pi SD card with the image included in the AIY Maker Kit.
14
17
15
-
## Installation
18
+
(I tried to install the required software included in the Coral getting started guide, but was unsuccessful due to an error that "GLIBC_2.29" was not found.)
19
+
20
+
Alternatively, you can opt for the original (slower) facial recognition process by setting Config.VISION_TECH to opencv. I'm no longer updating this section, so you may encounter some integration issues.
21
+
22
+
## Setup
16
23
```
17
24
chmod 777 install.sh
18
25
./install.sh
19
26
```
20
27
21
-
Disable audio (see Neopixels section below)
28
+
To use neopixels on Raspberry Pi, you need to disable the sound (see neopixel section).
22
29
23
-
## Running
30
+
## Operation
24
31
```
25
32
./startup.sh
26
33
```
27
-
To execute manual control via keyboard:
34
+
35
+
For manual control via keyboard
28
36
```
29
37
./manual_startup.sh
30
38
```
31
-
To execute startup including a preview of the video feed (not available via SSH):
39
+
40
+
Contains a preview of the video feed to get started (not available via SSH)
An RCWL-0516 microwave radar sensor is equipped on GPIO 13
82
+
### Motion sensor
83
+
An RCWL-0516 microwave radar sensor is attached to GPIO 13. This sensor, with its piservo module, will automatically scan 360 degrees like a radar when it enters a night loop
84
+
85
+
note: it contains two different piservos, one for nltk and one for motion sensor
74
86
75
-
### Stereo MEMS Mics
76
-
GPIO 18, 19 and 20 allow stereo MEMS microphones as audio input
87
+
### NLTK
88
+
NLTK analyzes a text and evaluates the degree to which the text is positive or negative. The antenna again uses the piservo control to perform an animation of this evaluation.
89
+
90
+
### Stereo MEMS Microphones
91
+
GPIO 18, 19 and 20 are used to use stereo MEMS microphones as audio input.
77
92
```
78
-
Mic 3V to Pi 3.3V
79
-
Mic GND to Pi GND
80
-
Mic SEL to Pi GND (this is used for channel selection, connect to either 3.3V or GND)
81
-
Mic BCLK to BCM 18 (pin 12)
82
-
Mic DOUT to BCM 20 (pin 38)
83
-
Mic LRCL to BCM 19 (pin 35)
93
+
Mic 3V - Connects to Pi 3.3V.
94
+
Mic GND - Connects to Pi GND.
95
+
Mic SEL - Pi connects to GND (this is used for channel selection, can be connected to 3.3V or GND).
_Note:_ See below for additional configuration to support voice recognition
114
+
_Note:_ See additional configuration below to support voice recognition.
115
+
116
+
### Hotword
117
+
The original hot word detection used the now deprecated Snowboy. The files are still available in this repository:
118
+
119
+
https://github.com/dmt-labs/modular-biped/blob/feature/PR29_review/modules/hotword.py contains the module from this framework.
120
+
121
+
https://github.com/dmt-labs/modular-biped/tree/feature/PR29_review/modules/snowboy includes original snowboy functionality.
122
+
123
+
https://github.com/dmt-labs/modular-biped/tree/feature/PR29_review/modules/snowboy/resources/models contains trained models that can be used as keywords. It may be possible to find more on the internet.
124
+
125
+
A guide to training new models is available here, but there are resources for training new models on a Linux device.
100
126
101
127
### Speech Recognition
102
-
Trigger word for voice recognition (currently not used):
128
+
Trigger word for voice recognition (not currently used):
103
129
https://snowboy.kitt.ai/
104
130
105
-
Speech recognition is enabled whenever a face is visible.
106
-
Ensure that the `device_index` specified in `modules/speechinput.py` matches your microphone.
131
+
Speech recognition is activated when a face is seen.
132
+
Make sure that the `device_index`value specified in the `modules/speechinput.py`file matches your microphone.
107
133
108
-
See `scripts/speech.py` to list input devices and test. See below for MEMS microphone configuration
134
+
See `scripts/speech.py` to list and test input devices. See below for MEMS microphone configuration.
135
+
### MEMS Microphone configuration for Speech Recognition
109
136
110
-
### MEMS Microphone configuration for speech recognition
137
+
By default, the Adafruit I2S MEMS Microphone Breakout does not work with voice recognition.
111
138
112
-
By default the Adafruit I2S MEMS Microphone Breakout does not work with speech recognition.
113
-
114
-
To support voice recognition on the MEMS microphone(s) the following configuration changes are needed.
139
+
The following configuration changes must be made to support voice recognition on MEMS microphone(s).
115
140
116
141
`sudo apt-get install ladspa-sdk`
142
+
Create `/etc/asound.conf` file with the following content:
117
143
118
-
Create `/etc/asound.conf` with the following content:
119
-
120
-
```
144
+
```
121
145
pcm.pluglp {
122
-
type ladspa
123
-
slave.pcm "plughw:0"
124
-
path "/usr/lib/ladspa"
125
-
capture_plugins [
126
-
{
127
-
label hpf
128
-
id 1042
129
-
}
130
-
{
131
-
label amp_mono
132
-
id 1048
133
-
input {
134
-
controls [ 30 ]
135
-
}
136
-
}
137
-
]
146
+
type ladspa
147
+
slave.pcm "plughw:0"
148
+
path "/usr/lib/ladspa"
149
+
capture_plugins [
150
+
{
151
+
label hpf
152
+
id 1042
153
+
}
154
+
{
155
+
label amp_mono
156
+
id 1048
157
+
input {
158
+
controls [ 30 ]
159
+
}
160
+
}
161
+
]
138
162
}
139
163
140
164
pcm.lp {
141
-
type plug
142
-
slave.pcm pluglp
165
+
type plug
166
+
slave.pcm pluglp
143
167
}
144
168
```
145
169
146
-
This enables the device 'lp' to be referenced in voice recognition. Shown with index `18` in the example below.
170
+
This ensures that the 'lp' device can be referenced in voice recognition. In the example below it is shown at index `18`.
147
171
148
-
Sample rate should also be set to `16000`
172
+
For example, the sample rate should be set to `16000`.
In order to use the Raspberry Pi’s serial port, we need to disable getty (the program that displays login screen)
186
+
To use the Raspberry Pi's serial port, getty (the program that displays the login screen) must be disabled.
163
187
164
-
`sudo raspi-config -> Interfacing Options -> Serial -> "Would you like a login shell to be accessible over serial" = 'No'. Restart`
188
+
`sudo raspi-config -> Interfacing Options -> Serial -> "Would you like a login shell to be accessible over serial" = 'No'. Restart`
165
189
166
190
#### Connection via serial pins
167
-
Connect the Pi GPIO 14 & 15 (tx & rx) to the arduino tx & rx (tx -> rx in both directions!) via a logic level shifter, as the Pi is 3v3 and the arduino is (most likely) 5v.
191
+
Connect GPIO pins 14 and 15 (tx and rx) of the Raspberry Pi to the Arduino tx and rx pins (tx -> rx in both direction and direction reversal!) via a logic level shifter, since Raspberry Pi 3v3 and Arduino (probably) It is 5v.
168
192
169
-
####Upload to Arduino over serial pins
170
-
To upload over serial pins, press the reset button on the Arduino at the point that the IDE starts 'uploading' (after compile), otherwise a sync error will display.
193
+
#### You can follow the steps below to upload to Arduino via serial pins:
194
+
195
+
To upload via serial pins, press the reset button on the Arduino at the point where the IDE starts 'uploading' (after compiling); otherwise a synchronization error will be displayed.
171
196
172
197
### Neopixel
173
198
174
-
WS1820B support is included via the Pi GPIO pin 12. Unfortunately to support this you must disable audio on the Pi.
199
+
WS1820B support is provided via Pi GPIO pin 12. Unfortunately, audio on the Pi must be disabled to support this.
175
200
176
201
```
177
202
sudo vim /boot/config.txt
178
203
#dtparam=audio=on
179
204
```
180
205
181
-
This is also why the application must be executed with `sudo`
206
+
Therefore, the application must be run with `sudo`.
207
+
208
+
If you want to use neopixels without using the driver (and no audio output), set pin to 12 in the configuration and set i2c to False. The application must also be run with sudo.
0 commit comments