Add documentation for MilliSecondsBehindSource growing exponentially. #700
Conversation
Add documentation for MilliSecondsBehindSource growing exponentially.
|
|
||
| In the latter case it is also necessary to configure `connector.client.config.override.policy=ALL` option in Kafka Connect worker config file `connect-distributed.properties`. For Debezium `connect` Docker image the environment variable `CONNECT_CONNECTOR_CLIENT_CONFIG_OVERRIDE_POLICY` can be used to configure the option. | ||
|
|
||
| == Why MilliSecondsBehindSource is growing exponentially ? |
There was a problem hiding this comment.
Thanks for this change, this sounds a bit concerning though. Can you provide some more context about this situation: what's your change event rate and your set-up in general? I don't quite agree to the statement on roundtrip times below; the connection throughput may be a limiting factor, but latency itself should be constant, also with larger roundtrip times between the DB and the Kafka Connect host.
There was a problem hiding this comment.
Hi @gunnarmorling ,
Thanks for your comments.
MilliSecondsBehindSource growing exponentially issue occurred in our environment and I had reached out to Jiri Pechanec about this issue. Link to Gitter Thread with details - https://gitter.im/debezium/user?at=6050ec1ed1aee44e2def7b89
Created this pull request as Jiri had recommended contributing to the documentation.
Please let me know if you need more details.
There was a problem hiding this comment.
Hi @rnatarajan so I definitely think adding an FAQ entry, particularly when MilliSecondsBehindSource is high is worthwhile given the scenario you describe, but I think perhaps the way this is conveyed, particularly with the use of "exponentially" gives a bad impression.
Perhaps, this could be reworded to something like "High MilliSecondsBehindSource when connector deployed with WAN connection to database" or similar. I think speaking in terms of higher than usual latency when using a WAN connection rather than a LAN connection is what we should strive for on this PR.
There was a problem hiding this comment.
@Naros Thanks for the comment. I can reword the README.
There was a problem hiding this comment.
Awesome, once you have, if you could just request a re-review of the PR that'd be wonderful.
|
❌ Developer Certificate of Origin (DCO) check failed. Hi @rnatarajan, please sign off all commits with: If pull request commits are not signed off, the pull request cannot be merged. For more information about why this is required, please see our blog about contribution requirement changes. |
Signed-off-by: Rajkamal Natarajan <[email protected]>
|
⚡️ Deploying PR Preview... |
|
Hi @rnatarajan please remove the merge commit. |
Working on removing the merge commit. |
|
❌ Developer Certificate of Origin (DCO) check failed. Hi @rnatarajan, please sign off all commits with: If pull request commits are not signed off, the pull request cannot be merged. For more information about why this is required, please see our blog about contribution requirement changes. |
1 similar comment
|
❌ Developer Certificate of Origin (DCO) check failed. Hi @rnatarajan, please sign off all commits with: If pull request commits are not signed off, the pull request cannot be merged. For more information about why this is required, please see our blog about contribution requirement changes. |
|
@Naros , Rather than using the command line, I had used Git Web rebase + sign-off which has caused the Merge commit. |
3 participants
No description provided.