pfsd: Key sharing reliability #4
Description
If a node is unable to save a key piece given to it by another node pnetserver.SendKeyPiece should return an error. Currently some of these errors are logged and others cause fatals.
When a node is locking itself, it should attempt to be sure a majority of nodes have received it's key pieces before locking. If a majority of nodes have not received it's key pieces and correctly saved them, it will not be able to retrieve it's data. There are tossups with security here, as after some time a node must lock even it is unable to contact a majority of nodes. In this case maybe the node could wipe its data and logs and raft information so it catches up in the future. Of course if a majority of nodes have the same problem, all the data would be gone in this case.