Move GitHub contributors fetching to scheduled data-layer task#17174
Draft
pettinarip wants to merge 11 commits intodevfrom
Draft
Move GitHub contributors fetching to scheduled data-layer task#17174pettinarip wants to merge 11 commits intodevfrom
pettinarip wants to merge 11 commits intodevfrom
Conversation
Replace per-request GitHub API calls with pre-fetched data stored in Netlify Blobs. This eliminates hundreds of API calls during page renders. - Add fetchGitHubContributors scheduled task (weekly) - Use git/trees API for single-request content listing - Fetch commits in parallel batches (20 concurrent) - Store results in data-layer for fast retrieval - Update contributors.ts to read from data-layer - Bump @trigger.dev/sdk to 4.3.3
pettinarip
requested review from
corwintines,
minimalsm and
wackerow
as code owners
github-actions
bot
added
dependencies 📦
✅ Deploy Preview for ethereumorg ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
The previous commit moved GitHub contributor fetching to the data-layer. This removes the now-unused fetchAndCacheGitHubContributors and fetchWithRateLimit functions from gh.ts, along with their imports. Also consolidates CONTENT_DIR and OLD_CONTENT_DIR imports in the new fetcher to use the shared constants.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
fetchGitHubContributorsscheduled task running weekly to fetch all contributor datacontributors.tsto read from data-layer instead of making live API calls@trigger.dev/sdkto 4.3.3Test plan