Improve extractor retries #449
Description
Is your feature request related to a problem? Please describe.
If an API call fails with a temporary error while running an extractor, we do not retry it. It could be improved.
Describe the solution you'd like
We should provide optional retries for extractors similar to sinks where the extractor call will be retried if the error returned is an instance of plugins.RetryError.
Describe alternatives you've considered
Each extractor could manage its own retries. But this does not seem to be a clean approach and is also likely to get missed when a new extractor is being added.
Additional context
We could make the retrier implementation available to extractors so that they can retry a specific intermediate step with limited retries without restarting the entire extract process.