Skip to content

Background GitHub stars sync with Vercel Cron #21

@lovincyrus

Description

@lovincyrus

Problem

GitHub stars sync is fully synchronous — the POST handler blocks until all GitHub API pages are fetched and inserted before responding. A user with 5,000 stars triggers ~50 sequential API calls, which can take 10-30s. On Vercel hobby tier (10s timeout), this can fail silently.

There's also no automatic re-syncing — users must manually visit Settings and click Sync each time.

Proposed Solution

waitUntil + Vercel Cron (zero new dependencies, free on hobby tier)

Manual sync (UX improvement)

  • Return { message: "Sync started" } immediately using Next.js waitUntil() to decouple the response from the work
  • The sync continues running in the background after the response is sent

Scheduled sync (new)

  • Add a GET /api/cron/sync-github-stars route triggered daily by Vercel Cron
  • Protect the route with CRON_SECRET header verification
  • Iterates over all users with a configured GitHubStarsSync record and syncs each
  • ETag caching means unchanged syncs cost 1 API call (~100ms), so daily runs are cheap

Cost

  • Free on Vercel hobby tier (2 cron jobs included, 1/day minimum)
  • waitUntil is built into Next.js
  • No external services or additional API keys

Notes

  • Existing ETag + skipDuplicates logic already handles idempotency
  • GITHUB_TOKEN env var (optional) increases rate limit from 60 to 5,000 req/hr
  • If timeout becomes a problem for very large star counts (3,000+), chunked pagination can be added later

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions