React to throttling by google

[algebravic]

I gave pdf2bib to a directory with a lot of pdf files, eventually it started getting errors because google was throttling the requests. It should react to that by quitting (since all further tries will fail). It would be nice if it could be given a list of files to act on, instead of going through a whole directory.

I noticed the following advice:

Check the Retry-After Header: Look for this header in the 429 response; it tells you exactly how long to wait (in seconds).

Implement Exponential Backoff: If no Retry-After header is present, increase your wait time exponentially (1s, 2s, 4s, 8s...) between retries.

Use Rate Limiting: Implement your own rate limiting on your application side