Skip to content

fix(docs): Implement exponential backoff for GitHub API to resolve HTTP 429/403 crashes#1181

Open
williamcancodee wants to merge 1 commit intofury-gl:v2from
williamcancodee:fix/doc-build-429-rate-limit
Open

fix(docs): Implement exponential backoff for GitHub API to resolve HTTP 429/403 crashes#1181
williamcancodee wants to merge 1 commit intofury-gl:v2from
williamcancodee:fix/doc-build-429-rate-limit

Conversation

@williamcancodee
Copy link

Description

The PR attempts to resolve the documentation generation crash caused by GitHub API rate limiting (HTTP Error 429/403), as indicated by issue #924.

In the previous implementation, when an HTTP error occurred, fetch_url() would return an empty dictionary. Consequently, when get_paged_request() tried to access f.status, an AttributeError would occur. However, even if this did not occur, the 5-attempt retry loop would terminate instantaneously without a delay.

Changes Made:

  • Fixed Error Handling: Modified fetch_url() to return the real HTTPError object (which naturally has a .code/.status attribute in recent versions of Python) or a dummy object for other types of exceptions to avoid AttributeError crashes.
  • Exponential Backoff: Introduced a time.sleep() for 429 (Rate Limit) and 403 (Forbidden/Abuse) status codes.
  • The script will now pause for progressively longer durations (1s, 2s, 4s, 8s, 16s) between attempts, giving the API quota window a chance to reset.
  • Added print statements to inform the user that the script is deliberately pausing rather than freezing.

Fixes #924

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Doc Generation Error - HTTP Error 429

1 participant