Reset of election timeout when handling requestVote RPC

[Vshishkar]

Hi there,

while checking out Raft.go code I stumbled across this line where electionTimeOut is reset every time Candidate requests vote.

In original Raft paper you are supposed to reset electionTimeout only if you granted vote, which makes sense, because outdated candidates (with different term, outdated log) can still call and timer is going to be reset even though they will never win the election.
This will increase leader election process and raft is unavailable for longer period of time.

I was wondering whether I'm missing something or is this a mistake and there should be a PR with fix.
Thanks

https://github.com/nats-io/nats-server/blob/main/server/raft.go#L3918

[mprimi]

Hello @Vshishkar, thank you for bringing this up.

electionTimeOut is reset every time Candidate requests vote

<pedantic>the timer is reset every time a node receives a vote request (the cause of which is a candidate sending it out sometimes before)</pedantic>.

As you suggest, this could increase the time it takes to replace a failed leader.
However it could also result in less contention and interference when another candidate is trying to get elected.

So overall, it's hard to say wether this is a net benefit or not. Depends on the situation.

We are indeed not compliant to what the paper suggests (that is: reset for valid AppendEntry and granted VoteRequest).
But i hope we agree that the difference does not compromise the safety and integrity of the protocol.
It only changes the timing of the election and only in a specific sub-case (another candidate is active).

Personally, I agree we should change this to only reset the timer on granted vote.
For sure we should not reset on VoteRequest with a stale Term (which we are doing right now!)

Please do send a PR if you want. But be aware that this part of the system is a bit in-flux right now, and we may not be able to take it as-is in the near future. We'll make sure to give you credit for pointing this out one way or another.

[Vshishkar]

Hello @mprimi, thanks for the reply.

I will create a PR with the fix.

Could you also please provide further detail on this point:

However it could also result in less contention and interference when another candidate is trying to get elected.

As I understand it, there are only two cases where we wouldn't want to reset the election timeout: when candidate A is calling node B with a smaller term, or when B has already voted for C. If we reset it in either of these cases, we would prolong the election period, thereby reducing the availability of Raft.

But i hope we agree that the difference does not compromise the safety and integrity of the protocol.

Regarding this point, I agree that, generally, it does not compromise the safety and integrity of the protocol. However, my concern lies in scenarios where there are leader failures and a new leader needs to be elected quickly, while some nodes are outdated. This could potentially lead to significant wait times.

For example, in the scenario provided in the image, if s5 were to attempt to be elected (assuming s1 is down), neither s5 nor s4 would ever win the election (election restriction), but they would reset the timeouts of s2 and s3. It's worth noting that when s5 initiates the election, it also resets its own timeout. Even though Raft uses randomized timeouts, s5 has a higher chance of triggering another election after not winning the previous one.