-
Couldn't load subscription status.
- Fork 1.4k
feat(autoRestart): make max backoff time configurable #11946
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat(autoRestart): make max backoff time configurable #11946
Conversation
Add support for `maxBackoffMinutes` property to configure the maximum backoff cap for automatic restarts of failed connectors and tasks. Default remains 60 minutes if not set. Signed-off-by: rodrigo-molina <[email protected]>
Signed-off-by: rodrigo-molina <[email protected]>
a297e6d to
72c8219
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #11946 +/- ##
============================================
+ Coverage 67.59% 67.60% +0.01%
- Complexity 7102 7103 +1
============================================
Files 574 574
Lines 28179 28185 +6
Branches 3199 3199
============================================
+ Hits 19047 19054 +7
+ Misses 7805 7802 -3
- Partials 1327 1329 +2
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR. However, thsi should probably have a proposal first given it is an APi change with unclear use-case?
|
Hey @scholzj, Thanks for the quick reply! Sure thing, what’s the preferred channel for the proposal? GitHub Issues, Discussions, or can we continue here in the PR? The main motivation for this change is to shorten the maximum retrial interval to align it with the retry policies we use on our platform. Using today's AutoRestart features, a faulty connector that self-recovers after a couple of hours may still wait up to an hour before being restarted. Adding a configurable I also found it interesting that the connector resiliency needs to be handled outside of Kafka Connect itself. We are experiencing some Kafka Connect tasks giving up even with "forever retries", when setting |
|
@rodrigo-molina Sorry, I forgot the details. The proposals can be opened here as a PR: https://github.com/strimzi/proposals ... this is where they will be discussed and voted on. There is a template with a structure outline. There are also many other proposals you can check out to see what they do. I guess in this case it does not need too much stuff on the implementation details. But it should cover the API, the use case(s), etc. |
Thanks for the detail. |
|
@rodrigo-molina thank you for taking care about this! Can you please move this PR to draft to make it clear it's not open for reviews yet because of the proposal which needs to be written and discussed. |
Type of change
Description
Currently, the Kafka Connect connector autoRestart configuration has a fixed max backoff cap of 60 minutes.
This PR introduces a
maxBackoffMinutesproperty to make the cap configurable. If not set, it defaults to 60 minutes, preserving backward compatibility.Checklist
make all, but they pass when run individually.