PoA - Boot nodes - What's the proper way of configuring them?

[inglkruiz]

PoA - Boot nodes - What's the proper way of configuring them?

Description

Imagine the following deployment flow:

1. I deploy a Polygon Edge PoA Network with one Validator node, therefore, the bootnodes array in the genesis.json file contains its multiaddr.
2. I deploy a 2nd Validator node: Using the 1st Validator I vote to add it as a Validator and start the node, the node syncs, and now the Network has 2 Validators. Consider this: I did not modify the genesis.json file.
3. I remove the 1st Validator node: Using both nodes, I vote to remove the 1st Validator, it goes down, and now the network has 1 Validator. Consider this: I did not modify the genesis.json file.
4. I deploy a new Node (it does not matter the type), the node uses the genesis.json file and find that the bootnode (the 1st Validator we just removed) is not answering, so it cannot join the network.

Why am I exposing this case?

Well, it's a thing that could happen, it's possibly an edge case.

I'm running PoA network deployments using Kubernetes and nodes can join and leave the network dynamically, meaning that the bootstrap node can leave the network.

OPEN QUESTIONS:

• Is it a good practice to modify the genesis.json file?
• Wouldn't be better to have a flag for bootnodes when starting the node? Or something like static-nodes in Hyperledger Besu?
• I see there's a --join flag where you pass a multiaddr, would it be correct to use it? But what are the repercussions?

Your environment

• Kubernetes
• version of the Polygon Edge > 0.2.1

Steps to reproduce

See the description.

Expected behaviour

I would expect to have a way for configuring the bootnodes that are up when starting a new node.

Actual behaviour

What I do right now is: modify the genesis.json file, of course only the bootnodes Array, and start the new node.

Logs

Not needed, it's easy to reproduce.

Proposed solution

Possibly adopting what Hyperledger Besu does is the way to go?

[lazartravica]

Hello @inglkruiz thank you for raising this issue!

It's important to note that the list of bootnodes, e.g. the nodes a single node will attempt to connect to is completely separate from the validator set that changes with the existing validators voting a validator in or out.

As every Polygon Edge can serve as a bootnode (inherently), my suggestion is to spin up a set of non-validator nodes that will always be active and use them as bootnodes in your local client configurations.

To answer your open questions:

Is it a good practice to modify the genesis.json file?

genesis.json, apart from holding the initial blockchain state of the genesis block (in this case just the initial validator set) is also used for local client configuration, for example the list of bootnodes this client will attempt to connect to when starting up. You can freely change the local configuration parameters, but don't change anything under the genesis key in the genesis.json as that would change the contents of the genesis block and break the blockchain effectively.

Wouldn't be better to have a flag for bootnodes when starting the node? Or something like static-nodes in Hyperledger Besu?

Polygon Edge supports the --join parameter when you are running the server command. Please take a look at CLI Commands. You will also find here that the genesis command takes in the --bootnode parameter as well, this is just used to generate the local JSON config which is also a part of the genesis.json and ease the setup process. Changing the array of bootnodes in the genesis.json itself later will not affect the blockchain state as the bootnodes is just a local configuration parameter.

I see there's a --join flag where you pass a multiaddr, would it be correct to use it? But what are the repercussions?

The repercussions are that the bootnode in that case is not persisted anywhere and you would need to supply it each time that the node is starting. If it suits your workflow, feel free to use it.

---

Please let me know if I managed to help you and covered all of your questions. Thanks

[inglkruiz]

Hi @lazartravica thanks for your quick response.

Based on your answers, I consider that the best approach for such case is to make every node a bootnode (Validators and Non-Validators), meaning that I'll be modifying the bootnodes Array of the genesis.json file. So what I did is correct.

Now, the --join flag for it sounds like a flag I can use if one Organization wants to have a second node, not sure if that statement is correct.

One last thing to mention, I run a scenario where a Non-Validator is left alone in the network, then new blocks won't be mine (All Validators voted for leaving the network). I wanted to restart the mining process, adding a new validator did not work. I imagine it's not possible to restart the mining. Kind of silly but it could happen.

[ZeljkoBenovic]

I'm moving this Issue to Discussion