-
Notifications
You must be signed in to change notification settings - Fork 926
Open
Labels
bughancementIt's kind of a bug, but mostly an enhancement.It's kind of a bug, but mostly an enhancement.outputsAny tasks or issues relating specifically to outputsAny tasks or issues relating specifically to outputsux
Description
DynamoDB batch writes are limited to 25 items, as per the documentation here. The benthos dynamo writer currently falls back to one-by-one writes in the case of any errors (such as exceeding this 25 item limit), which has a huge impact on throughput in the case of large output batch sizes.
It would be nice if the dynamo writer would internally limit each batch call to 25 items, making this invisible to the user and avoiding the need to restrict output batch settings to 25 or fewer in benthos config.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bughancementIt's kind of a bug, but mostly an enhancement.It's kind of a bug, but mostly an enhancement.outputsAny tasks or issues relating specifically to outputsAny tasks or issues relating specifically to outputsux