Kinesis ingest #14
Merged
Kinesis ingest #14
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rswalkden
reviewed
Dec 8, 2025
src/lua_resty_netacea.lua
Outdated
| local res, err = client:put_records(records) | ||
| if err then | ||
| ngx.log( ngx.ERR, "NETACEA BATCH - error sending batch to Kinesis: ", err ); | ||
| -- TODO: Consider implementing retry logic or dead letter queue |
Contributor
There was a problem hiding this comment.
What do you want to do about this? Maybe raise a ticket in the backlog? Our other integrations do retry these batches
rswalkden
reviewed
Dec 8, 2025
Comment on lines
+661
to
+662
| -- TODO | ||
| BytesReceived = vars.bytes_received or 0, -- Doesn't seem to work |
Contributor
There was a problem hiding this comment.
What is the issue here? Bytes received can be a nuisance, most of the time we just take the value of the content-length header I think
rswalkden
approved these changes
Dec 8, 2025
Contributor
rswalkden
left a comment
rswalkden
left a comment
There was a problem hiding this comment.
LGTM so long as we remember about those TODOs
Contributor
Author
Good shout on the dead letter queue. I've added it in. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update this integration to work with Netacea Ingest v2 - based around AWS Kinesis streams.