Skip to content

fix: re-alert only on the hour #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 9, 2025
Merged

fix: re-alert only on the hour #84

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3c80a4a
fix: re-alert only on the hour
cctdaniel Jan 8, 2025
0a53dc2
bump
cctdaniel Jan 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions pyth_observer/dispatch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,13 +197,16 @@ async def process_zenduty_events(self, current_time):
to_remove.append(identifier)
# Raise alert if failed > $threshold times within the last 5m window
# or if already alerted and not yet resolved.
# Re-alert every 5 minutes but not more often.
# Re-alert at the start of each hour but not more often.
elif (
info["failures"] >= alert_threshold or (info["sent"] and not resolved)
) and (
not info.get("last_alert")
or current_time - datetime.fromisoformat(info["last_alert"])
> timedelta(minutes=5)
not info.get("last_alert") # First alert - send immediately
or ( # Subsequent alerts - send at the start of each hour
current_time - datetime.fromisoformat(info["last_alert"])
> timedelta(minutes=5)
and current_time.minute == 0 # Only alert at the start of each hour
)
):
logger.debug(f"Raising Zenduty alert {identifier}")
self.open_alerts[identifier]["sent"] = True
Expand Down
Loading