Can a Durable Object have an alarm for a time in the past that will never fire?

[johtso]

Existing documentation URL(s)

https://developers.cloudflare.com/durable-objects/api/alarms/

What changes are you suggesting?

The documentation describes that there is guaranteed at-least-once execution of alarms.

I've experienced situations where getAlarm() returns a time in the past, but there are no unhandled errors to be seen and days have past.

I'm using a Durable Object as a pseudo long running process by triggering an alarm every 10 seconds. The previous version of my code had a CRON trigger every minute that would check if an alarm was set and if not, would schedule a new alarm (as there should always be an alarm).

That's how I noticed the issue, my DO had been doing nothing all weekend. getAlarm was returning a days old alarm time, but it wasn't being triggered.

I've now modified my code to also check if the current alarm is greater than a certain time in the past, and if so reset it.

If the system is designed in such a way that an old alarm time will never be triggered (regardless of how that situation arises), then that should be documented. Someone else mentioned they've been experiencing similar issues with alarms set only a short amount of time in the future (less than 10 seconds). Is there an undocumented limitation here?

Sorry if this is a bit of an odd mix of documentation request and bug report, but it feels like there is some undocumented behaviour here.

Additional information

No response

[vy-ton]

@johtso This is more likely a bug. I'm not aware of an undocumented limit here.

Could you email vy@cloudflare.com with your account id, DO namespace id, and object ids where you see this alarm behavior so our team can investigate.

[vy-ton]

@johtso We fixed an issue that was preventing your alarm from running as scheduled. Alarm should run as expected now. I'm going to close this issue.

[johtso]

@vy-ton that's wonderful, thanks! Are you able to provide any more details? Under what circumstances might this bug have been seen? How has the behaviour changed? I'm assuming if someone else was experiencing a similar issue this fix should also help them also?

[vy-ton]

@johtso There was a bug preventing some alarms in SQLite backed DOs from successfully running. These alarms would be continuously retried. These repeated retries led our scheduler to lag substantially behind due to retries taking precedence over new alarms in some regions.

[johtso]

@vy-ton thank you for getting back to me with those extra details, glad the cause of the issue could be found!