Skip to content

Commit aac727c

Browse files
committed
Add README
1 parent 4545e1c commit aac727c

File tree

1 file changed

+281
-0
lines changed

1 file changed

+281
-0
lines changed
Lines changed: 281 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,281 @@
1+
# Coffee Voice Agent UI
2+
3+
A comprehensive PyQt-based monitoring dashboard for the Coffee Voice Agent system. This package provides real-time visualization and control of voice agent status, emotions, tool usage, conversation flow, and system analytics.
4+
5+
![Coffee Voice Agent Monitor](https://img.shields.io/badge/ROS2-Jazzy-blue) ![PyQt5](https://img.shields.io/badge/UI-PyQt5-green) ![License](https://img.shields.io/badge/License-TODO-red)
6+
7+
## 🎯 Overview
8+
9+
The Coffee Voice Agent UI is a standalone ROS2 application that provides comprehensive monitoring and control capabilities for the coffee robot's voice interaction system. It connects to the voice agent bridge to display real-time status information, conversation transcripts, emotional states, and system performance metrics.
10+
11+
### Key Features
12+
13+
- **🤖 Real-time Agent Status**: Monitor connection state, behavioral mode, and session information
14+
- **🎭 Emotion Visualization**: Track current emotions, transitions, and emotional journey
15+
- **💬 Live Conversation Flow**: Real-time transcript with user speech and agent responses
16+
- **🔧 Tool Activity Monitor**: Function tool execution tracking with performance metrics
17+
- **📊 Analytics Dashboard**: Usage statistics, performance trends, and system health
18+
- **⚙️ Manual Controls**: Testing interface for virtual requests and debug commands
19+
20+
## 📋 Prerequisites
21+
22+
- **ROS2 Jazzy** (or compatible version)
23+
- **Python 3.8+**
24+
- **PyQt5** (installed via `python_qt_binding`)
25+
- **coffee_voice_agent_msgs** package
26+
- **coffee_voice_agent** package (for full system operation)
27+
28+
## 🚀 Installation
29+
30+
### 1. Clone and Build
31+
32+
```bash
33+
# Navigate to your ROS2 workspace
34+
cd ~/your_ros2_ws/src
35+
36+
# Build the package
37+
cd ..
38+
colcon build --packages-select coffee_voice_agent_ui
39+
40+
# Source the workspace
41+
source install/setup.bash
42+
```
43+
44+
### 2. Verify Installation
45+
46+
```bash
47+
# Check if the package is available
48+
ros2 pkg list | grep coffee_voice_agent_ui
49+
50+
# Check available executables
51+
ros2 pkg executables coffee_voice_agent_ui
52+
```
53+
54+
## 🎮 Usage
55+
56+
### Option 1: Standalone Monitor (Recommended)
57+
58+
Launch just the monitoring UI to connect to an existing voice agent system:
59+
60+
```bash
61+
# Direct execution
62+
ros2 run coffee_voice_agent_ui voice_agent_monitor
63+
64+
# Or using launch file with parameters
65+
ros2 launch coffee_voice_agent_ui monitor.launch.py \
66+
ui_node_name:=voice_monitor \
67+
log_level:=debug
68+
```
69+
70+
### Option 2: Full System Launch
71+
72+
Launch both the voice agent bridge and monitor UI together:
73+
74+
```bash
75+
ros2 launch coffee_voice_agent_ui full_system.launch.py \
76+
voice_agent_host:=localhost \
77+
voice_agent_port:=8080 \
78+
enable_ui:=true
79+
```
80+
81+
### Option 3: RQT Plugin (Development)
82+
83+
For development purposes, the UI can also be loaded as an RQT plugin:
84+
85+
```bash
86+
# Load in rqt
87+
rqt
88+
89+
# Navigate to: Plugins → Coffee Robot → Voice Agent Monitor
90+
# Or run standalone rqt plugin
91+
rqt --standalone coffee_voice_agent_ui
92+
```
93+
94+
## 🏗️ Architecture
95+
96+
### System Overview
97+
98+
```
99+
Voice Agent ←→ Voice Agent Bridge ←→ ROS2 Topics ←→ Monitor UI
100+
```
101+
102+
### Component Architecture
103+
104+
```
105+
VoiceAgentMonitorApp (QMainWindow)
106+
├── Agent Status Widget (connection, state, metrics)
107+
├── Emotion Display Widget (current emotion, transitions, timeline)
108+
├── Conversation Widget (live transcript, timeouts, turn tracking)
109+
├── Tool Monitor Widget (function tool usage, performance)
110+
├── Analytics Widget (usage trends, performance metrics)
111+
└── Controls Widget (manual testing, debug commands)
112+
```
113+
114+
### ROS2 Integration
115+
116+
The monitor subscribes to these topics:
117+
- `voice_agent/status` (AgentStatus) - Unified agent state information
118+
- `voice_agent/tool_events` (ToolEvent) - Function tool execution events
119+
- `voice_agent/user_speech` (String) - User speech transcriptions
120+
- `voice_agent/connected` (Bool) - Connection status updates
121+
122+
The monitor publishes to these topics:
123+
- `voice_agent/virtual_requests` (String) - Test virtual coffee requests
124+
- `voice_agent/commands` (String) - Debug and control commands
125+
126+
## 📊 Dashboard Layout
127+
128+
```
129+
┌─────────────────┬─────────────────┬─────────────────┐
130+
│ 🤖 Agent Status │ 💬 Conversation │ 📊 Analytics │
131+
│ • State & Mode │ Flow │ • Performance │
132+
│ • Connection │ • Live Chat │ • Usage Stats │
133+
│ • Session Info │ • Timeouts │ • Trends │
134+
├─────────────────┼─────────────────┼─────────────────┤
135+
│ 🎭 Emotion │ 🔧 Tool Monitor │ ⚙️ Controls │
136+
│ Center │ • Active Tools │ • Virtual Reqs │
137+
│ • Current │ • Recent Calls │ • Commands │
138+
│ • Transitions │ • Statistics │ • Debug Tools │
139+
└─────────────────┴─────────────────┴─────────────────┘
140+
```
141+
142+
## 🎛️ Features Guide
143+
144+
### Agent Status Panel
145+
- **Real-time state indicator**: Visual status with color coding
146+
- **Connection monitoring**: WebSocket connection health
147+
- **Session tracking**: Active conversation duration and metrics
148+
- **Phase indication**: Current conversation phase (greeting, discussion, etc.)
149+
150+
### Emotion Center
151+
- **Current emotion display**: Large visual indicator with emoji
152+
- **Transition tracking**: Previous → current emotion flow
153+
- **Emotion timeline**: Visual history of emotional journey
154+
- **Eye animation preview**: How emotions translate to robot expressions
155+
156+
### Conversation Flow
157+
- **Live transcript**: Real-time user and agent messages
158+
- **Tool integration**: Function tool calls inline with conversation
159+
- **Timeout monitoring**: User response countdown with visual indicators
160+
- **Turn tracking**: Conversation metrics and response times
161+
162+
### Tool Monitor
163+
- **Active tracking**: Currently executing tools with duration
164+
- **Recent activity**: Last 10 tool calls with status and timing
165+
- **Usage statistics**: Success rates, execution times, frequency
166+
- **Performance metrics**: Tool reliability and efficiency tracking
167+
168+
### Analytics
169+
- **Session performance**: Daily conversation count, success rates
170+
- **Popular interactions**: Most requested functions and topics
171+
- **Emotion trends**: Emotional state patterns over time
172+
- **System metrics**: Message rates, connection health, queue status
173+
174+
### Controls Panel
175+
- **Quick actions**: End conversation, reset state, pause wake word
176+
- **Virtual request testing**: Simulate coffee orders with different priorities
177+
- **Command interface**: Send debug commands with parameters
178+
- **Debug tools**: Connection testing, log export, tool triggers
179+
180+
## 📁 Launch Files
181+
182+
### `monitor.launch.py`
183+
Launches only the monitor UI for connecting to existing voice agent systems.
184+
185+
**Parameters:**
186+
- `ui_node_name` (default: "voice_agent_monitor") - Name for the UI node
187+
- `log_level` (default: "info") - Logging level
188+
- `window_title` (default: "Coffee Voice Agent Monitor") - Window title
189+
190+
### `full_system.launch.py`
191+
Launches both the voice agent bridge and monitor UI for complete system operation.
192+
193+
**Parameters:**
194+
- `voice_agent_host` (default: "localhost") - Voice agent WebSocket host
195+
- `voice_agent_port` (default: "8080") - Voice agent WebSocket port
196+
- `enable_ui` (default: "true") - Enable/disable monitor UI
197+
- `log_level` (default: "info") - Logging level for all nodes
198+
199+
## 🔧 Configuration
200+
201+
### Environment Variables
202+
- `QT_AUTO_SCREEN_SCALE_FACTOR=1` - Qt high DPI scaling
203+
- `ROS_DOMAIN_ID` - ROS2 domain for multi-robot systems
204+
205+
### Parameters
206+
The monitor can be configured through ROS2 parameters:
207+
- `use_sim_time` (default: false) - Use simulation time
208+
- `window_title` - Custom window title
209+
210+
## 🐛 Troubleshooting
211+
212+
### Common Issues
213+
214+
**1. "No module named 'coffee_voice_agent_msgs'"**
215+
```bash
216+
# Ensure the messages package is built and sourced
217+
colcon build --packages-select coffee_voice_agent_msgs
218+
source install/setup.bash
219+
```
220+
221+
**2. Qt High DPI Warnings**
222+
```bash
223+
# Set environment variable before running
224+
export QT_AUTO_SCREEN_SCALE_FACTOR=1
225+
ros2 run coffee_voice_agent_ui voice_agent_monitor
226+
```
227+
228+
**3. No Data in UI Panels**
229+
- Check if voice agent bridge is running: `ros2 node list | grep voice_agent_bridge`
230+
- Verify topic publications: `ros2 topic list | grep voice_agent`
231+
- Check topic data: `ros2 topic echo voice_agent/status`
232+
233+
**4. UI Won't Start**
234+
```bash
235+
# Check PyQt installation
236+
python3 -c "from python_qt_binding.QtWidgets import QApplication; print('PyQt OK')"
237+
238+
# Check ROS2 dependencies
239+
ros2 pkg deps coffee_voice_agent_ui
240+
```
241+
242+
### Debug Mode
243+
244+
Run with debug logging for detailed information:
245+
```bash
246+
ros2 run coffee_voice_agent_ui voice_agent_monitor --ros-args --log-level debug
247+
```
248+
249+
## 🔗 Related Packages
250+
251+
- **coffee_voice_agent** - Core voice agent implementation
252+
- **coffee_voice_agent_msgs** - ROS2 message definitions
253+
- **coffee_vision_ui** - Camera monitoring interface
254+
- **coffee_expressions** - Emotion and expression control
255+
256+
## 📝 Development
257+
258+
### Adding New Widgets
259+
260+
1. Create widget in `coffee_voice_agent_ui/widgets/`
261+
2. Inherit from `QWidget` and implement required interfaces
262+
3. Add to main layout in `voice_agent_monitor_app.py`
263+
4. Connect to ROS2 data streams as needed
264+
265+
### Extending Analytics
266+
267+
1. Add data collection in existing widgets' `get_analytics_data()` methods
268+
2. Update `AnalyticsWidget` to process new data types
269+
3. Create visualizations in analytics panel
270+
271+
## 📄 License
272+
273+
TODO: License declaration
274+
275+
## 🤝 Contributing
276+
277+
This package is part of the Coffee Robot project. Please follow the project's coding conventions and submit pull requests for review.
278+
279+
---
280+
281+
**For support or questions about the Coffee Voice Agent UI, please refer to the main project documentation or contact the development team.**

0 commit comments

Comments
 (0)