Fix race condition causing API keys to deplete to 0 near reset time

[Sukuna0007Abhi]

The issue is exhausted from a timing mismatch where the local system assumed reset was imminent (within ~2 minutes), but GitHub hadn't actually refreshed the key yet, leading to rapid cycling in a tight loop. To fix this, I've implemented a configurable 120-second buffer in the __handle_github_ratelimit_response method of github_data_access.py, ensuring keys stay expired long enough for GitHub to catch up.

Enhancement:

The buffer uses max(epoch_when_key_resets, time.time() + GITHUB_KEY_RESET_BUFFER) to only delay when necessary, preventing unnecessary waits while avoiding the race. This resolves the depletion issue without impacting normal operation, and the buffer is adjustable via the GITHUB_KEY_RESET_BUFFER environment variable for flexibility.So it does not stick to hardcoded the 120 seconds. It allows the buffer value to be configured via an environment variable...

How the buffer fixes it:

By setting expire_time = max(epoch_when_key_resets, time.time() + 120), the key stays expired for at least 120 seconds from now.

This ensures:

If reset is imminent, the key waits extra time for GitHub to catch up.
The loop is broken because the key isn't available for reuse too soon.

Why 120 seconds:

Matches the "within 2 minutes" window from the issue where the race occurs. It's conservative but not excessive.

Changes

• Added a configurable 120-second buffer in __handle_github_ratelimit_response.
• Keys now stay expired for at least 2 minutes when reset is imminent, preventing premature reuse.
• Buffer is set via GITHUB_KEY_RESET_BUFFER env var (default 120s).

Credits

I used contributor @guptapratyksh's approach, thanks to him, but a little enhancement has done more to it.... As I fixed the hardcoded 120 seconds part..
& Thanks to @MoralCode and @cdolfi for flagging this issue..

I enhanced the contributor's 120-second buffer approach by making it conditional (only applies when reset is imminent via max()), configurable via env var, and defined as a maintainable constant—keeping it efficient and flexible without over-delaying keys.

This PR fixes #3598

Signed commits

• Yes, I signed my commits.

[guptapratykshh]

seems specific changes, as mentioned in the issue thread(#3598 (comment)), nothing is missed

[sgoggins]

LGTM: Approving with the caveat that we need to have a plan for populating config items that are new.

[sgoggins]

Using a default solves for there not being an entry in the config. I think we still need PRs that add config items to provide a pathway for those config items to make it into either the augur.json or the table.

[MoralCode]

Im a little concerned about whether this is the right solution and want some time to think about it in more depth after CHAOSScon