Skip to content

Deal with rate limits better #46

@aspyker

Description

@aspyker

In updating the 192 public and private repos for Netflix (likely also due the large number of issues on some projects), what I see is most get processed in about 16 minutes and then the rest an hour later. I believe this is due to the github access library just looping for an hour waiting for the rate limits to reset. While this works, it isn't the best. It would be better to just quit once rate limits are exhausted as the scraper is written to be be run multiple times and we, at Netflix, run it three times per day.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions