Skip to content

Latest commit

 

History

History
269 lines (218 loc) · 8.57 KB

LOG_NOTIFICATION_ENHANCEMENT.md

File metadata and controls

269 lines (218 loc) · 8.57 KB

Log Monitoring Notification Enhancement - Implementation Summary

Issue Addressed

GitHub Issue #113: The code was not creating notification entries upon updates about log changes. Users needed visibility into three situations:

  1. Changes in log entries since last check (RED)
  2. Can't determine changes in log entries (AMBER)
  3. No change in log entries (GREEN)

Additionally, the agent requested enhanced debug logging to help troubleshoot issues like "no notification was received."

Changes Implemented

1. Enhanced Debug Logging in log_monitor.py

When log level is set to "maximal", the system now logs comprehensive comparison details:

# Time range information
- Current check time
- Lookback period (in hours)
- Comparison window: from timestamp to timestamp

# Entry counts
- Number of current error entries found
- Number of previous error entries loaded
- Whether changes can be determined (baseline available)

# Sample entries
- Up to 5 sample entries from current errors
- Up to 5 sample entries from previous errors
- Truncated to 150 characters for readability

# Comparison results
- Number of new errors detected
- Number of resolved errors
- Number of persistent errors

Example Debug Output:

============================================================
LOG COMPARISON DETAILS
============================================================
Current check time: 2024-12-30 14:30:00
Lookback period: 24 hours
Comparing logs from: 2024-12-29 14:30:00 to 2024-12-30 14:30:00
Current errors found: 5
Previous errors loaded: 3
Previous log data available - can determine changes
Sample of current error entries (up to 5):
  1. 2024-12-30 14:25:00 ERROR homeassistant.components.mqtt: Connection failed...
  2. 2024-12-30 14:20:00 WARNING homeassistant.loader: Integration deprecated...
  ...
============================================================
Comparison results:
  New errors: 2
  Resolved errors: 0
  Persistent errors: 3

2. Always Create Notifications - Three Status States

The _report_log_analysis() method in sentry_service.py now always creates a notification with one of three distinct states:

🟢 GREEN Status - All Clear

Triggered when:

  • Previous log data is available (can_determine_changes = True)
  • No new errors detected (new_error_count = 0)
  • No resolved errors (resolved_error_count = 0)
  • Severity is 'none'

Notification Title: "✅ Home Assistant Log Monitor - All Clear"

Message includes:

  • Status: GREEN
  • Summary: "No changes in log entries since last check"
  • System logs are stable
  • Analysis method (AI or Heuristics)
  • Check timestamp
  • Lookback period

🟡 AMBER Status - Cannot Determine

Triggered when:

  • No previous log data available (can_determine_changes = False)
  • This is the first run or previous logs were cleared

Notification Title: "⚠️ Home Assistant Log Monitor - Unable to Compare"

Message includes:

  • Status: AMBER
  • Explanation: First run or previous data unavailable
  • Current errors found count
  • Explanation that baseline is being established
  • What to expect on next check

🔴 RED Status - Changes Detected

Triggered when:

  • Previous log data is available (can_determine_changes = True)
  • Any new errors detected OR severity is not 'none'

Notification Title: "🔴 Home Assistant Log Monitor - Changes Detected" (Severity-specific emoji may be used: 🟠 for high, 🟡 for medium, 🟢 for low)

Message includes:

  • Status: RED
  • Severity level (low/medium/high/critical)
  • Summary of changes
  • Statistics: new and resolved error counts
  • Up to 5 significant errors (truncated for readability)
  • Recommendations
  • Next steps
  • Analysis method (AI or Heuristics)
  • Check timestamp
  • Lookback period

3. New Analysis Field: can_determine_changes

Added to the analysis dictionary returned by check_logs():

analysis['can_determine_changes'] = len(previous_errors) > 0

This flag distinguishes between:

  • True: Previous baseline exists, can compare logs
  • False: No previous baseline, first run or data cleared

4. Notification Logic Changes

Before:

if severity == 'none':
    logger.debug("No significant log changes to report")
    return  # ← No notification created

After:

# Always create notification - determine which state applies
if not can_determine_changes:
    # AMBER notification
elif severity == 'none' and new_count == 0 and resolved_count == 0:
    # GREEN notification
else:
    # RED notification
# ← Notification ALWAYS created

Testing

Test Coverage

  1. test_log_monitor.py (7 tests - all existing tests pass)

    • LogMonitor initialization
    • Error filtering
    • Error signature extraction
    • Log comparison
    • Heuristic analysis
    • Save and load logs
    • Obfuscation in logs

  2. test_log_notification_states.py (4 new tests)

    • AMBER notification state (first run)
    • GREEN notification state (no changes)
    • RED notification state (changes detected)
    • Enhanced debug logging flag

  3. demo_notification_states.py (demonstration script)

    • Shows all three notification states in action
    • Simulates different scenarios
    • Validates notification logic

Test Results

Running Log Monitor tests...
✓ LogMonitor initialization test passed
✓ Error filtering test passed
✓ Error signature extraction test passed
✓ Log comparison test passed
✓ Heuristic analysis test passed
✓ Save and load logs test passed
✓ Obfuscation in logs test passed
==================================================
Tests completed: 7 passed, 0 failed

Running Log Notification States tests...
✓ AMBER notification state test passed
✓ GREEN notification state test passed
✓ RED notification state test passed
✓ Enhanced debug logging test passed
==================================================
Tests completed: 4 passed, 0 failed

User Impact

Before Changes

  • Notifications only created when severity != 'none'
  • No notification on first run
  • No notification when logs are stable
  • Difficult to debug why no notification was received
  • Users couldn't tell if monitoring was working

After Changes

  • Notification always created (green/amber/red)
  • Clear status indication on first run (AMBER)
  • Positive feedback when logs are stable (GREEN)
  • Clear indication of changes (RED with severity)
  • Comprehensive debug logging when log level is maximal
  • Users can verify monitoring is working

Configuration

No configuration changes required. The feature works with existing settings:

  • monitor_logs_after_update: true - Enable log monitoring
  • log_check_lookback_hours: 24 - Set lookback period
  • Log level set to "maximal" enables enhanced debug output

Debugging Aid

When log_level: maximal is set in config.yaml, administrators can now see:

  1. Exactly which time range is being compared
  2. How many log entries were found
  3. Sample entries from both current and previous checks
  4. Comparison results (new/resolved/persistent)
  5. Whether a baseline is available
  6. Why a specific notification state was chosen

This makes it much easier to diagnose issues like:

  • "Why didn't I get a notification?"
  • "What logs is it comparing?"
  • "Is the monitoring working?"

Files Modified

  1. ha_sentry/rootfs/app/log_monitor.py (+50 lines)

    • Enhanced debug logging in check_logs()
    • Added can_determine_changes flag to analysis

  2. ha_sentry/rootfs/app/sentry_service.py (+103 lines, -52 lines)

    • Completely rewrote _report_log_analysis()
    • Implemented three notification states
    • Removed early return for severity 'none'

  3. tests/test_log_notification_states.py (new file, +187 lines)

    • Tests for all three notification states
    • Tests for can_determine_changes flag

  4. tests/demo_notification_states.py (new file, +135 lines)

    • Demonstration script showing all states
    • Helpful for understanding the feature

Backward Compatibility

✅ Fully backward compatible

  • All existing tests pass
  • No breaking changes to existing functionality
  • No new configuration required
  • Enhanced logging only activates at DEBUG level
  • Existing notification behavior improved, not changed

Summary

This implementation successfully addresses the issue by:

  1. ✅ Always creating notifications (no silent failures)
  2. ✅ Clearly indicating three states (green/amber/red)
  3. ✅ Providing comprehensive debug logging for troubleshooting
  4. ✅ Making it clear what is being compared and when
  5. ✅ Helping users understand if monitoring is working correctly

The changes are minimal, focused, and enhance the user experience without breaking existing functionality.