commands: invalidate cached config when node is not ctdb leader #152
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When the `samba-container config-update --watch` command is run on a CTDB enabled server we avoid updating the local config unless the node is the ctdb leader. However, the update function was returning the current config which the watch loop assigned to the previous var, meaning that the current and previous config would be equal on future trips around the loop. Therefore once the node became the leader it would not update the config even if it differed from the config in samba. This is a problem on start up because the loop initially runs before ctdb is ready and the node leadership is not known. Change it so that when the node is not a leader it returns None and "unsets" the previous config. This could lead to extra unneeded updates but should avoid missing updates that are needed. Signed-off-by: John Mulligan <[email protected]>
|
What is the abnormality seen with the current code? Did we observe a cluster stuck in UNHEALTHY or some other undesired state? |
|
The observed issue was that on the Ceph SMB system there's a brief race condition where if you create shares while the ceph orchestration is still rolling out the first set of containers it will apply the "initial config" to smb but the config watch sidecar will not do anything. There's a few more details in the commit message. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When the
samba-container config-update --watchcommand is run on a CTDB enabled server we avoid updating the local config unless the node is the ctdb leader. However, the update function was returning the current config which the watch loop assigned to the previous var, meaning that the current and previous config would be equal on future trips around the loop. Therefore once the node became the leader it would not update the config even if it differed from the config in samba. This is a problem on start up because the loop initially runs before ctdb is ready and the node leadership is not known. Change it so that when the node is not a leader it returns None and "unsets" the previous config. This could lead to extra unneeded updates but should avoid missing updates that are needed.