Skip to content

Add batch_size to Kafka Destination#3549

Open
itsarijitray wants to merge 3 commits intomainfrom
expose-kafka-batch-size
Open

Add batch_size to Kafka Destination#3549
itsarijitray wants to merge 3 commits intomainfrom
expose-kafka-batch-size

Conversation

@itsarijitray
Copy link
Contributor

@itsarijitray itsarijitray commented Jan 28, 2026

This pull request adds support for configuring the maximum batch size for Kafka messages, which is especially useful for RETL sources with specific batching requirements. The changes introduce a new batch_size parameter to both the payload interface and the action definition, allowing users to specify this value and providing a sensible default.

image

Testing

Include any additional information about the testing you have completed to
ensure your changes behave as expected. For a speedy review, please check
any of the tasks you completed below during your testing.

  • Added unit tests for new functionality
  • Tested end-to-end using the local server
  • [If destination is already live] Tested for backward compatibility of destination. Note: New required fields are a breaking change.
  • [Segmenters] Tested in the staging environment
  • [Segmenters] [If applicable for this change] Tested for regression with Hadron.

Security Review

Please ensure sensitive data is properly protected in your integration.

  • Reviewed all field definitions for sensitive data (API keys, tokens, passwords, client secrets) and confirmed they use type: 'password'

New Destination Checklist

  • Extracted all action API versions to verioning-info.ts file. example

@itsarijitray itsarijitray marked this pull request as ready for review February 5, 2026 06:52
@itsarijitray itsarijitray requested a review from a team as a code owner February 5, 2026 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants