Skip to content

feat: Add rate limiting and backoff for Kafka Connect client#1723

Open
skecskes wants to merge 1 commit intokafbat:mainfrom
skecskes:issues/1710-kc-rate-limiting-and-backoff
Open

feat: Add rate limiting and backoff for Kafka Connect client#1723
skecskes wants to merge 1 commit intokafbat:mainfrom
skecskes:issues/1710-kc-rate-limiting-and-backoff

Conversation

@skecskes
Copy link

Summary

  • Add concurrency limits to KafkaConnectService connector data scraping
  • Implement transient error retries with backoff in RetryingKafkaConnectClient
  • Extend ClustersProperties with KafkaConnect configuration options (maxConcurrentRequests, retry settings)

Changes

  • RetryingKafkaConnectClient — added retry logic for transient errors
  • KafkaConnectService — added concurrency limiting for connector scraping
  • ClustersProperties — new KafkaConnect config options
  • KafkaClusterFactory — wiring of new config
  • Tests for retry logic, concurrency handling, and config properties

Closes #1710

…scraping

- feat: Implement transient error retries in RetryingKafkaConnectClient
- feat: Extend ClustersProperties with KafkaConnect configuration options
- test: add tests for KafkaConnect client retry logic and service concurrency handling
@skecskes skecskes requested review from a team as code owners March 13, 2026 19:22
@kapybro kapybro bot added status/triage Issues pending maintainers triage status/triage/manual Manual triage in progress status/triage/completed Automatic triage completed and removed status/triage Issues pending maintainers triage labels Mar 13, 2026
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi skecskes! 👋

Welcome, and thank you for opening your first PR in the repo!

Please wait for triaging by our maintainers.

Please take a look at our contributing guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status/triage/completed Automatic triage completed status/triage/manual Manual triage in progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler burst DDos-ing kafka connectors

1 participant