Re-adding a "permanently removed" member?

[erikschul]

Scenario: Cluster of 3 nodes (s1,s2,s3). One node fails (s3). The node is removed using etcdctl member remove, and a new node is added using etcdctl member add, using the same name (s3). The new node is using the same certificates and name as the failed node (s3). But the new node reports a failure in the log: the member has been permanently removed from the cluster

Question: How can a "permanently removed" node be added? Does it require new certificates, or a different name? Can this be overridden?

[jmhbnz]

Hi @erikschul - Thanks for your question. Can you please provide the full startup logs for the node failing to start and also confirm the version of etcd you are using?

From memory I believe you will need to start the new member with a fresh/new data directory.

[erikschul]

Hmm, I got it working now.

I think the procedure must be:

• stop service + empty data dir / rebuild vm
• remove member
• add member
  this registers the node as "unstarted" in etcdctl member list
• run with --initial-cluster-state existing