fix: update database connection info bugs#40
Merged
jason-lynch merged 3 commits intomainfrom Jun 3, 2025
Merged
Conversation
Instance IPs can change when their containers restart, so we need to update their connection info on each refresh. PLAT-98
Refactors the `orch.GetInstanceConnectionInfo` and the `ConnectionInfo` usage to deduplicate code and behavior. PLAT-98
This adds a short initial in `WaitForService` and adds some additional checks to verify that the service has hit its desired state. See the inline comment for more details about this change. It also adds some tolerance for transient connection errors from the Patroni API. This lengthens the time to detect a failed instance in exchange for more durability when the Patroni reload operation causes a brief unavailibility. PLAT-98
jason-lynch
force-pushed
the
fix/PLAT-98/update-connection-info
branch
from
4a8a63c to
d09ddd5
Compare
tsivaprasad
requested changes
Jun 3, 2025
| return fmt.Errorf("failed to get cluster status: %w", err) | ||
| } | ||
| continue | ||
| } |
Contributor
There was a problem hiding this comment.
Should we reset errCount to 0 after a successful connection attempt?
Member
Author
There was a problem hiding this comment.
I don't think so. The danger of doing that is that if the service is "flapping", we could end up stuck in an infinite loop. This is meant to be a very small tolerance to handle cases where Patroni becomes unavailable after we call its reload endpoint.
Does that make sense to you too?
tsivaprasad
approved these changes
Jun 3, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This resolves some issues related to updating the instance connection information when the instance IP addresses change after a restart:
WaitForService.WaitForServicewaits for the service to hit the desired state, so a stale read can cause it to exit before the service has started updating.PLAT-98