UnoGuide is a smart guide cane prototype built on Arduino® UNO Q (Linux MPU + STM MCU) with Edge Impulse vision inference. It provides traffic‑light recognition and obstacle awareness, delivering voice feedback and buzzer alerts as the primary user interface for visually impaired users.
- 1. Overview
- 2. System Architecture
- 3. Hardware
- 4. Software
- 5. Get the Model from Edge Impulse
- 6. Retrain the Model Using the Dataset
- 7. Build and Download Deployable Model
- 8. Setup Network, SSH, and Edge Impulse on Arduino® UNO Q
- 9. Setup Arduino® App Lab and Deploy Application
- 10. Project Structure
- 11. Setup Instructions
- 12. Running the Application
- 13. Deployment Mode and Debug Feature Control
- 14. Testing Guide
- 15. Demo Output
A Smart Guide Cane prototype designed to assist visually impaired users with traffic‑light awareness, upper‑body obstacle detection, and voice feedback. The system combines Arduino® UNO Q (Linux MPU + STM MCU) with an Edge Impulse vision model. AI inference runs on the Linux side and writes stable results to label.txt; the MCU polls this label to drive the core assistive feedback (voice + buzzer).
UnoGuide enhances a traditional guide cane with:
- 🚦 AI‑based traffic light recognition (red / yellow / green)
- 📏 Upper‑body obstacle detection using an ultrasonic sensor
- 🔔 Assistive feedback for safe navigation
- 🗣️ Voice feedback (primary user feedback)
- 🔊 Buzzer alerts for close‑range obstacles
- 🧪 Debug & development interfaces (can be disabled)RGB LEDs
- 8×13 LED matrix
- Web preview (MJPEG + JSON)
- 🔌 Fully on‑device processing (offline capable)
flowchart LR
%% --- Inputs ---
CAM[USB UVC Camera\n/dev/videoX] -->|frames| LNX
US[Ultrasonic Sensor\nHC-SR04] -->|echo timing| MCU
%% --- Linux MPU ---
subgraph LNX["Linux (MPU) on Arduino UNO Q"]
RUN[Edge Impulse Runner\nInference + Anti-flicker + OFF debounce]
LAB["(label.txt)"]
VOICE[Voice Feedback\nWAV playback via aplay]
RUN -->|stable label\nred/yellow/green/off| LAB
LAB -->|label change| VOICE
end
%% --- Bridge ---
LAB -->|RouterBridge\nget_ai_label| MCU
%% --- MCU STM ---
subgraph MCU["MCU (STM) on Arduino UNO Q"]
POLL[Poll label every 100ms]
BUZ[Buzzer Alert\n< 50cm]
LED[RGB LEDs\nDebug only]
MAT[8×13 LED Matrix\nDebug only]
POLL --> LED
POLL --> MAT
end
%% --- Outputs ---
MCU -->|core| BUZ
VOICE -->|core| SPK[USB Headset / Speaker]
%% --- Debug-only web preview ---
RUN -. debug .-> WEB["Web Preview (MJPEG)\n/stream.mjpg + /state.json"]
%% --- Styling ---
classDef core fill:#e8fff2,stroke:#2e7d32,stroke-width:1px;
classDef debug fill:#fff7e6,stroke:#b26a00,stroke-width:1px,stroke-dasharray: 4 3;
class VOICE,BUZ,SPK,RUN,LAB,POLL core;
class LED,MAT,WEB debug;
- Runs Edge Impulse Linux SDK for camera‑based inference (unoguide_runner.py)
- Performs anti‑flicker, OFF debouncing, and stable label decision.
- Writes stable results to label.txt (red_light, yellow_light, green_light, off).
- Voice feedback plays WAV prompts on stable label changes.
- Optional Web preview (MJPEG) for debugging (can be disabled in deployment).
- Implemented in the Linux app (main.py)
- Monitors label.txt at 10 Hz
- Triggers WAV audio prompts when the label changes
- Uses ALSA
aplayfor audio output - Includes:
- Cooldown (default
2500 ms) - Repeat playback (default
3×with gap)
- Cooldown (default
✅ This is the main feedback channel for visually impaired users
- Polls get_ai_label every 100 ms via RouterBridge.
- Controls buzzer (primary alert) and RGB LEDs (debug only).
- Reads ultrasonic distance and triggers buzzer when < 50 cm.
- 8×13 LED matrix displays distance (debug only).
- 🔊 Buzzer
- Activated when obstacle distance < 50 cm
- RGB LEDs — traffic‑light state visualization (debug only)
- 8×13 LED matrix — distance display (debug only)
Linux → File (
label.txt) → MCU is used as a simple and robust IPC mechanism.
- Arduino® UNO Q
- USB camera (x1)
- USB-C® hub adapter with external power (x1)
- A power supply (5V, 3A) for the USB hub (e.g., a phone charger)
- Personal computer with internet access.
- HC-SR04 ultrasonic sensor
- Buzzer (core assistive output)
- USB headset / speaker (for voice feedback)
- RGB LEDs (debug only)
| Function | Pin | Notes |
|---|---|---|
| Ultrasonic Trig | D11 | Core sensor |
| Ultrasonic Echo | D12 | Core sensor |
| Buzzer | D5 | Primary user alert |
| Red LED | D6 | Debug only |
| Yellow LED | D7 | Debug only |
| Green LED | D8 | Debug only |
- Edge Impulse
- Arduino® App Lab
- Python 3 (Linux side)
- ALSA (aplay) for audio playback
python3 -m venv ~/venvs/ei
source ~/venvs/ei/bin/activate
pip install edge_impulse_linux opencv-python-headless six pyaudio
sudo apt install -y portaudio19-dev
- ALSA aplay (/usr/bin/aplay)
- Optional audio device selection via:
export APLAY_DEVICE="<alsa_device_name>"
The traffic‑light recognition model is built and managed using Edge Impulse. You may either reuse an existing public project, or create your own dataset by capturing and labeling traffic‑light images.
- Go to https://studio.edgeimpulse.com/
- Sign up for a free Edge Impulse account
- Create a new project (e.g. UNOGuide)
- Image: 320*320
- Resize mode: Squash
- Object Detection
- Public academic / open‑source traffic‑light datasets
- Dash‑cam or street‑scene datasetsLabels must be consistent:
- red_light
- yellow_light
- green_light
- Capture traffic‑light images using a USB or phone camera
- Upload images to Edge Impulse Studio
- Label bounding boxes for red / yellow / green lights
- Include environmental variations (day/night, distance, backlight)
✅ Self‑collected data significantly improves real‑world robustness
- Use Object Detection pipeline (e.g. YOLO‑based)
- Train and evaluate in Edge Impulse Studio
- Go to Deployment in Edge Impulse Studio
- Select Linux (AArch64) as the target
- Build and download the
.eimmodel (e.g.unoguide-linux-aarch64-v1.eim)
This section covers network setup, SSH access, and Edge Impulse installation on the Arduino® UNO Q, preparing the device for model deployment and on-device inference.
Once installed, confirm your UNO Q is connected by running:
adb devicesIf the device appears, log in:
adb shellConnect to Wi-Fi:
sudo nmcli dev wifi connect <WiFi-SSID> password <WiFi-password>Check IP address:
hostname -Isudo apt install openssh-server -y
sudo systemctl enable ssh
sudo systemctl stop sshd
sudo ssh-keygen -A
sudo systemctl start sshdNow you’re ready to connect from your local machine using the terminal:
ssh arduino@<arduino ip address>sudo apt update
curl -sL https://deb.nodesource.com/setup_20.x | sudo bash -
sudo apt install -y gcc g++ make build-essential nodejs sox gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps
sudo npm install edge-impulse-linux -g --unsafe-permPlug in your camera,speaker then run:
edge-impulse-linuxFollow the wizard to log in and select your project.
To switch projects:
edge-impulse-linux --cleanPlease refer to the Edge Impulse with Arduino UNO Q.
Arduino® App Lab lets you build and deploy apps directly on the Arduino® UNO Q, which combines a microcontroller and a Linux-based processor. It runs on Windows, macOS, and Linux, is pre-installed on the UNO Q, and updates automatically.
For detailed steps, refer to the documentation: Set up Arduino® App Lab
From your local machine, connect to the device using:
ssh arduino@<arduino ip address>Clone the repository and copy the required source code to the device’s application directory. Make sure the device is connected via SSH or ADB before proceeding:
cd /home/Arudino
git clone -n --depth=1 --filter=tree:0 https://github.com/qualcomm/Startup-Demos.git
cd Startup-Demos
git sparse-checkout init --no-cone
git sparse-checkout set Others/IoT-Robotics/Smart_Guide_Cane
git checkout
cp -R Smart_Guide_Cane/ArduinoApps/unoguide /home/Arduino/ArduinoApps/
cp Smart_Guide_Cane/unoguide/unoguide_runner.py /home/Arduino/
After building your model in Edge Impulse, download the .eim file (e.g. unoguide-linux-aarch64-v1.eim) to your local machine.
Use scp to transfer the model file to the Arduino® UNO Q:
scp unoguide-linux-aarch64-v1.eim arduino@<arduino-ip-address>:/home/Arduino/- Ensure the destination directory exists before copying files
- Use sudo if permission errors occur
- Verify the source path if files are not found
ArduinoApps/
└── unoguide/
├── python/
│ ├── label.txt # Shared AI state
│ ├── main.py # Voice feedback + label monitor
│ └── audio/ # Voice prompts
├── sketch/
│ ├── sketch.ino # MCU firmware
└───└── sketch.yaml
/home/arduino/unoguide_runner.py # AI inference + web preview (optional)
/home/arduino/unoguide-linux-aarch64-v1.eim
- Install required Arduino® libraries in Arduino® App lab:
- MsgPack 0.4.2
- DebugLog 0.8.4
- ArxContainer 0.7.0
- ArxTypeTraits 0.3.1
- Upload sketch.ino to the Arduino® UNO Q (STM MCU)
- (Optional) Disable RGB LED / LED matrix related code paths for deployment
- Copy the Edge Impulse deployable model to the device:
/home/arduino/unoguide-linux-aarch64-v1.eim - Ensure application directory exists:
/home/arduino/ArduinoApps/unoguide/python/ - Verify camera device:
ls /dev/video* - (Optional – Deployment mode) Run without web preview to disable debug UI
WAV mapping (defined in main.py):
- red_light → red_light.wav
- yellow_light → yellow_light.wav
- green_light → green_light.wav
- off → idle.wav
When running UnoGuide on Arduino® UNO Q (embedded Linux), audio playback is handled via ALSA (aplay). Unlike desktop Linux, there is usually no default audio device configured.
-
Identify the USB Audio Device
aplay -lExample output: card 1: II [Jabra EVOLVE 30 II], device 0: USB Audio [USB Audio]
-
Use plughw Instead of hw
export APLAY_DEVICE=plughw:1,0 aplay -D plughw:1,0 green_light.wav -
Application Integration
export APLAY_DEVICE=plughw:1,0 export APP_VOICE=1
- COOLDOWN_MS – debounce interval between voice prompts
- REPEAT_COUNT – number of repetitions per trigger
- REPEAT_GAP_MS – gap between repeated playbacks
Ensure all WAV files are placed under:
/app/python/audio/
-
Connect to the Arduino® UNO Q via Arduino® App Lab over Wi‑Fi
-
Click the Run button in Arudino app lab
-
Execute unoguide_runner.py
From your local machine, connect to the device using:
ssh arduino@<arduino ip address>
cd /home/arduino/ export APP_LABEL_FILE=/home/arduino/ArduinoApps/unoguide/python/label.txt export APP_AUDIO_DIR=/home/arduino export APP_VOICE=1 python3 unoguide_runner.py \ --model /home/arduino/unoguide-linux-aarch64-v1.eim \ --camera /dev/video0 \ --preview --preview-port 8080
-
Preview (debug only): http://<UNOQ_IP>:8080/
export APP_DEPLOYMENT=1
- Web preview disabled automatically
- Debug logs disabled
- Voice feedback remains enabled
#define DEPLOYMENT_MODE 1
- RGB LEDs ❌
- LED matrix ❌
- Buzzer ✅
- DBG_RGB_LED (MCU)
- DBG_LED_MATRIX (MCU)
- APP_WEBPREVIEW (Linux)
- APP_DEBUG (Linux)
- Voice + buzzer are the only required outputs in real‑world deployment.
- Debug interfaces can be disabled to reduce power consumption and distraction.
- Adjust thresholds, cooldowns, and repeat counts to tune UX.
- Point the camera at traffic lights and observe voice prompts.
- Move an object within 50 cm to trigger the buzzer.
- Use web preview only during development.
Traffic-light recognition and obstacle awareness run at the same time. They were recorded separately, each with a different focus.
Please follow Section 12. Running the Application to run UnoGuide.
