Commit f5e2cd6
Add structured logging, metrics, comprehensive testing, and Loki stack
Major improvements to observability, testing, and operational reliability:
## 🔍 Structured Logging with Tracing
- Replace all eprintln! calls with structured tracing macros across all services
- Add environment-based log level control via RUST_LOG
- Implement tracing-subscriber with env-filter, fmt, and json support
- Add tracing-log bridge for backward compatibility with existing log:: usage
- Services updated: common, weatherust, healthmon, speedynotify, updatemon, updatectl
Benefits:
- Structured logging with key-value pairs for better filtering
- Dynamic log level control without recompilation
- Consistent logging format across all services
- Better debugging with contextual information
## 📊 Prometheus Metrics Infrastructure
- Create comprehensive metrics module in common crate
- Add metrics crate dependency across all services
- Implement 12 different metric types:
* Notification delivery tracking (success/failure rates)
* Operation execution timing (weather API, speedtests, updates)
* Server operation tracking (multi-server operations)
* Container health metrics (CPU, memory, health status)
* Update availability gauges (OS and Docker updates)
* Speedtest result metrics (download, upload, ping, degradation)
* Weather API performance (call count, response time)
* Cleanup operation metrics (items removed, bytes reclaimed)
* Webhook request metrics (endpoint, status, duration)
Benefits:
- Full operational visibility
- Performance monitoring
- Alerting capabilities (when integrated with Prometheus)
- Historical trend analysis
## 🧪 Comprehensive Testing Suite
- Add 29 new tests across 3 packages (20 → 49 total tests, +145%)
- weatherust: 13 tests for ZIP detection, city parsing, location normalization
- speedynotify: 7 tests for JSON parsing, speed degradation detection
- common: 9 tests for all metrics recording functions
- All tests passing with 100% pass rate
Benefits:
- Prevents regressions during refactoring
- Documents expected behavior
- Enables confident code changes
- Faster debugging of parsing issues
## 🔧 Loki Logging Stack (Optional)
- Add docker-compose.logging.yml for centralized log aggregation
- Configure Loki 3.0 for log storage (7-day retention)
- Configure Promtail 3.0 for automatic Docker log collection
- Configure Grafana 11.0 with Loki pre-configured as datasource
- Traefik integration for HTTPS access to Grafana
- Comprehensive LOGGING_SETUP.md guide
Benefits:
- Centralized log viewing across all containers
- Rich query capabilities with LogQL
- Time-based log exploration
- Export and analysis features
## 📝 Documentation
- Add IMPROVEMENTS_2025.md with complete implementation summary
- Add LOGGING_SETUP.md with Loki stack setup and usage guide
- Update .env.example and .env.example.short with Grafana configuration
- Migration guide for users and developers
## 🔧 Dependencies Added
Runtime:
- tracing 0.1
- tracing-subscriber 0.3 (env-filter, fmt, json)
- tracing-log 0.2
- metrics 0.23
- metrics-exporter-prometheus 0.15 (updatectl only)
Dev:
- tokio-test 0.4
- mockito 1.5 (common only)
- axum-test 15 (updatectl only)
## ✅ Quality
- All workspace crates compile successfully
- All 49 tests passing (0 failures)
- Fully backward compatible
- No breaking changes
## 🚀 Usage
Set log level: RUST_LOG=debug weatherust --zip 52726
Deploy logging: docker compose -f docker-compose.yml -f docker-compose.logging.yml up -d
View logs: https://grafana.js-node.app (or your configured domain)
🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>1 parent bcf2407 commit f5e2cd6
File tree
25 files changed
+2222
-125
lines changed- common
- src
- healthmon
- src
- speedynotify
- src
- src
- updatectl
- src
- updatemon
- src
25 files changed
+2222
-125
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
343 | 343 | | |
344 | 344 | | |
345 | 345 | | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
0 commit comments