Skip to content

Commit a14697d

Browse files
Fixing README instructions for split-brain
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
1 parent ff0a4a6 commit a14697d

File tree

2 files changed

+12
-12
lines changed

2 files changed

+12
-12
lines changed

charts/redis-ha/README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -463,15 +463,15 @@ Should your Pod require additional egress rules, define them in a `egressRules`
463463

464464
## Sentinel and redis server split brain detection
465465

466-
Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the <https://github.com/DandyDeveloper/charts/issues/121>.
466+
Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the #121.
467467

468468
The proposed solution is currently implemented as a sidecar container that runs a bash script with the following logic:
469469

470-
1. Every `splitBrainDetection.interval` seconds a master (as known by sentinel) is determined
471-
1. If it is the current node: ensure the redis server's role is master as well.
472-
1. If it is not the current node: ensure the redis server also replicates from the same node.
473-
474-
If any of the checks above fails - the redis server reinitialisation happens (it regenerates configs the same way it's done during the pod init), and then the redis server is instructed to shutdown. Then kubernetes restarts the container immediately.
470+
1. At intervals defined by splitBrainDetection.interval, the sidecar checks which node is recognized as master by Sentinel.
471+
2. If the current pod is the master according to Sentinel, it verifies that the local Redis server is also running as master.
472+
3. If the current pod is not the master, it ensures the local Redis server is replicating from the correct master node.
473+
4. If any of these checks fail, the sidecar will retry the check at intervals defined by splitBrainDetection.retryInterval.
474+
5. If the checks continue to fail after the retry attempts, the sidecar triggers a reinitialization: it regenerates the Redis configuration and instructs the Redis server to shut down. Kubernetes will then automatically restart the container.
475475

476476
# Change Log
477477

charts/redis-ha/README.md.gotmpl

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -214,15 +214,15 @@ Should your Pod require additional egress rules, define them in a `egressRules`
214214

215215
## Sentinel and redis server split brain detection
216216

217-
Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the <https://github.com/DandyDeveloper/charts/issues/121>.
217+
Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the #121.
218218

219219
The proposed solution is currently implemented as a sidecar container that runs a bash script with the following logic:
220220

221-
1. Every `splitBrainDetection.interval` seconds a master (as known by sentinel) is determined
222-
1. If it is the current node: ensure the redis server's role is master as well.
223-
1. If it is not the current node: ensure the redis server also replicates from the same node.
224-
225-
If any of the checks above fails - the redis server reinitialisation happens (it regenerates configs the same way it's done during the pod init), and then the redis server is instructed to shutdown. Then kubernetes restarts the container immediately.
221+
1. At intervals defined by splitBrainDetection.interval, the sidecar checks which node is recognized as master by Sentinel.
222+
2. If the current pod is the master according to Sentinel, it verifies that the local Redis server is also running as master.
223+
3. If the current pod is not the master, it ensures the local Redis server is replicating from the correct master node.
224+
4. If any of these checks fail, the sidecar will retry the check at intervals defined by splitBrainDetection.retryInterval.
225+
5. If the checks continue to fail after the retry attempts, the sidecar triggers a reinitialization: it regenerates the Redis configuration and instructs the Redis server to shut down. Kubernetes will then automatically restart the container.
226226

227227

228228
# Change Log

0 commit comments

Comments
 (0)