Skip to content

Conversation

@Ankith-Confluent
Copy link
Member

@Ankith-Confluent Ankith-Confluent commented Sep 22, 2025

This PR refactors the librdkafka test suite to improve cloud compatibility by adding configurable options in test.conf. This enables better test portability and maintainability across different Kafka environments.

Changes

1. Configurable Test Framework

  • test.skip.idempotent: Skip tests requiring idempotent producer (replaces -I flag)
  • test.supported.acks: Configure supported acknowledgment values (e.g., -1,0,1)
  • test.sleep.multiplier: Configurable sleep multiplier for different environments. Multiplies explicit sleep_for() delays for cluster state propagation (set 0 to skip sleeps), different from test.timeout.multiplier which multiplies API operation timeouts (can't be disabled).
  • test.timeout.multiplier: Unified timeout scaling (existing feature, now consistently used)
  • test.skip.numbers: Skip specific tests by number (comma-separated list)

2. Topic Cleanup

  • Many tests were creating topics, but not deleting them.
  • Added topic delete utility.

3. Replication Factor

  • Changed hardcoded replication factors to use broker defaults (-1)

4. Preprocessor Directives for Version Compatibility

  • Added preprocessor directives to support librdkafka versions 2.0 through 2.11

🔧 Configuration Examples

test.timeout.multiplier=2.0
test.sleep.multiplier=3.0
test.supported.acks=-1,1
test.skip.idempotent=true
test.skip.numbers=0011,0055,0063,0068,0081,0086

…djusting timeout handling for cloud environments. Added checks for partition existence and refined error handling for GROUP resource configurations, ensuring compatibility with managed Kafka services.
Copy link
Contributor

@milindl milindl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks OK, left some minor comments.

tests/test.c Outdated
* @brief Sleep with configurable multiplier (only if multiplier > 0)
* @param wait_time Sleep time in seconds
*/
void sleep_for(int wait_time) {
Copy link
Contributor

@milindl milindl Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename this to test_, see the other comments made on other test files regarding this function


RdKafka::MessageTimestamp ts = msg.timestamp();
if (ts.type != RdKafka::MessageTimestamp::MSG_TIMESTAMP_CREATE_TIME)
if (ts.type != RdKafka::MessageTimestamp::MSG_TIMESTAMP_CREATE_TIME &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a confirmed issue in K2? If so add a comment. AK behaves as expected, if we have timestamp type set as create time correctly, then it will return it correctly also.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes its confirmed bug in K2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants