A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

aws_s3 source doesn't have any handling or backoff for SQS errors like IAM or 5xx. It will just retry the whole logic without any backoff in a loop, generating a flood of traffic. As this will be a CPU-intensive task, it will also trigger autoscaling if Vector is deployed with it, scaling up the problem 😄 Please see the code below:

In case of error, it will just return an empty Vec, go quickly through the rest of the code, and retry the whole process in a loop (no backoff).

Configuration

Version

0.46.1

Debug Output

Example Data

No response

Additional Context

No response

References

No response