A comprehensive, production-ready health monitoring and diagnostics system for ROS 2 robotics applications. This system provides real-time monitoring of ROS graph health, topic statistics, system metrics, and centralized logging with alerting capabilities.
The ROS 2 Health Monitoring System is designed to provide enterprise-grade observability for robotics applications. It integrates seamlessly with ROS 2 Humble and provides:
- Real-time ROS Graph Monitoring - Detect missing nodes, orphaned topics, QoS mismatches
- Topic Statistics - Monitor message rates, latency, and deadline compliance
- System Metrics - CPU, memory, disk, network via Telegraf
- Centralized Logging - Aggregate ROS logs with Fluent Bit and Loki
- Visualization - Pre-configured Grafana dashboards
- System Control - Remote shutdown/reboot via ROS 2 services
- Alerting - Slack/Teams notifications for critical issues
┌─────────────────────────────────────────────────────────────────────────────┐
│ ROS 2 NETWORK │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Your │ │ Your │ │ Your │ │ Your │ │
│ │ Node 1 │ │ Node 2 │ │ Node 3 │ │ Node N │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │
│ └─────────────┴──────┬──────┴─────────────┘ │
│ │ │
│ ┌─────────────────────────┴─────────────────────────┐ │
│ │ HEALTH MONITORING STACK │ │
│ │ ┌─────────────────┐ ┌─────────────────────────┐ │ │
│ │ │ rosgraph_monitor│ │ diagnostic_bridge │ │ │
│ │ │ - Node health │ │ - /diagnostics → DB │ │ │
│ │ │ - Topic health │ └───────────┬─────────────┘ │ │
│ │ │ - QoS checks │ │ │ │
│ │ └────────┬────────┘ │ │ │
│ │ │ │ │ │
│ │ ┌────────┴────────┐ ┌───────────┴─────────────┐ │ │
│ │ │ telegraf_bridge │ │ system_control_node │ │ │
│ │ │ - Topic stats │ │ - Shutdown/Reboot API │ │ │
│ │ └────────┬────────┘ └─────────────────────────┘ │ │
│ └───────────┼───────────────────────────────────────┘ │
└──────────────┼──────────────────────────────────────────────────────────────┘
│
┌──────────────┼──────────────────────────────────────────────────────────────┐
│ │ OBSERVABILITY STACK (Docker) │
│ ┌───────────▼───────────┐ │
│ │ Telegraf │──────────┐ │
│ │ - System metrics │ │ │
│ │ - ROS topic stats │ │ │
│ └───────────────────────┘ │ │
│ ▼ │
│ ┌───────────────────────┐ ┌───────────────┐ ┌───────────────────────┐ │
│ │ Fluent Bit │ │ InfluxDB │ │ Grafana │ │
│ │ - ROS log collection │ │ - Metrics DB │◄─┤ - Dashboards │ │
│ └───────────┬───────────┘ └───────────────┘ │ - Alerts │ │
│ │ └───────────────────────┘ │
│ ▼ ▲ │
│ ┌───────────────────────┐ │ │
│ │ Loki │───────────────────────────────┘ │
│ │ - Log aggregation │ │
│ └───────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
| Feature | Description |
|---|---|
| 🔍 Graph Monitoring | Real-time detection of missing nodes, leaf topics, dead sinks |
| 📊 Topic Statistics | Message rates, latency percentiles, deadline compliance |
| 🖥️ System Metrics | CPU, memory, disk, network, GPU (Jetson) |
| 📝 Log Aggregation | Centralized ROS logs with search and filtering |
| 📈 Dashboards | Pre-built Grafana dashboards for all metrics |
| 🔔 Alerting | Slack/Teams notifications for anomalies |
| 🔄 System Control | Remote shutdown/reboot via ROS 2 service |
| 🐳 Docker Ready | Single-command deployment with Docker Compose |
| 🤖 Jetson Support | Optimized for NVIDIA Jetson platforms |
- ROS 2 Humble
- Docker & Docker Compose
- Python 3.10+
# Clone the repository
git clone https://github.com/jj7258/ros_diag_ws.git
cd ros_diag_ws
# Build the workspace
colcon build --symlink-install
# Source the workspace
source install/setup.bash
# Set RMW environment (required)
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export RMW_IMPLEMENTATION_WRAPPER=rmw_stats_shim
# Deploy the monitoring stack
cd docker
./deploy.sh| Service | URL | Credentials |
|---|---|---|
| Grafana | http://localhost:3000 | admin / admin |
| InfluxDB | http://localhost:8086 | admin / admin |
| Loki | http://localhost:3100 | - |
# Launch all monitoring nodes
ros2 launch ros_health_app health_monitoring.launch.py
# With rosbag recording enabled
ros2 launch ros_health_app health_monitoring.launch.py enable_rosbag:=trueDetailed documentation is available in the docs/ directory:
| Document | Description |
|---|---|
| Architecture | System design and component details |
| Installation | Detailed setup instructions |
| Configuration | Configuration options and tuning |
| API Reference | ROS 2 interfaces and services |
| Deployment | Docker and production deployment |
| Troubleshooting | Common issues and solutions |
ros_diag_ws/
├── config/ # Shared ROS configuration files
│ ├── rosgraph_config.yaml # Graph monitor configuration
│ └── heartbeat_config.yaml # Heartbeat settings
├── docker/ # Docker infrastructure
│ ├── compose/ # Docker Compose files
│ ├── ros/ # ROS container files
│ ├── telegraf/ # Telegraf configuration
│ ├── grafana/ # Grafana provisioning
│ ├── fluentbit/ # Fluent Bit configuration
│ └── deploy.sh # Deployment script
├── docs/ # Documentation
├── src/
│ ├── interfaces/ # ROS 2 message/service definitions
│ │ ├── system_interfaces/ # System control interfaces
│ │ ├── rosgraph_monitor_msgs/
│ │ └── ...
│ ├── core/ # Core monitoring components
│ │ ├── rosgraph_monitor/ # ROS graph health monitor
│ │ ├── telegraf_bridge/ # Topic stats to Telegraf
│ │ ├── diagnostic_bridge/ # Diagnostics to InfluxDB
│ │ └── rmw_stats_shim/ # RMW statistics wrapper
│ ├── apps/ # Application packages
│ │ └── ros_health_app/ # Main launch files
│ └── rmw_implementation/ # Custom RMW wrapper
└── README.md
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Need help? Open an issue or check the Troubleshooting Guide.