Checking my understanding in creating a gossip peer to peer network

[eranrund]

Hello!

I am trying to write an application that establishes a p2p network and broadcasts messages around. Ideally, each message should make it to all available peers. There should also be a way for a node to connect to the network and learn the state of all currently present peers. The general idea is that the peers each hold a list of items (e.g. files) and I want each file to find its way to all other peers, and peers can join and leave the network as they wish. The amount of nodes is not going to be enormous - probably dozen or so.

My understanding so far is that these are the steps I need to take:

1. Create a custom behaviour that contains Kademlia, Gossipsub, Identify
2. Subscribe to some arbitrary Gossipsub topic, this will allow me to publish new items to all nodes
3. Call kademlia.add_address and swarm.dial to my bootstrap peer(s)
4. Call kademlia.start_providing on some arbitrary key, this should let nodes find peers by querying who provides this key (and they all will be providing it). Do this periodically.
5. When I discover a new provider for this shared key, call swarm.dial on each of the providers addresses
6. When the identify behaviour returns a Identify event, call kademlia.add_address on all listen_addrs provided
7. Periodically call kademlia.bootstrap()

Does that sound right for achieving robust peer discovery and interconnected-ness? It would probably be nice if I wasn't forming full mesh networks, but that seems the most reliable way unless I am missing something.

I am still not sure how I achieve the initial state sync upon discovering a new peer. Perhaps I add a custom protocol/behaviour for that, or use kademlia somehow?

I couldn't find an example that does all of this in one application - I only found bits and pieces and hammered them together.

Thank you so much! This is a super cool project.

[thomaseizinger]

7. Periodically call `kademlia.bootstrap()`

You should only need to do this once.

Does that sound right for achieving robust peer discovery and interconnected-ness? It would probably be nice if I wasn't forming full mesh networks, but that seems the most reliable way unless I am missing something.

Yes. You probably also want to think about storing addresses of nodes on disk and connecting to those at start-up again

I am still not sure how I achieve the initial state sync upon discovering a new peer. Perhaps I add a custom protocol/behaviour for that, or use kademlia somehow?

Yes, you'd define a custom protocol for that. Look at the file-sharing example for some inspiration.

I am trying to write an application that establishes a p2p network and broadcasts messages around. Ideally, each message should make it to all available peers. There should also be a way for a node to connect to the network and learn the state of all currently present peers.

In terms of the message design, you could take inspiration from how blockchains do this, i.e. messages are identified by some ID, each nodes serves messages upon request. No need for PoW if your nodes can trust each other so this would be reasonable cheap. Something to think about is how far back into the past a new node should be able to learn the state.

Let me know if this helps :)

[eranrund]

Thank you for taking a look and answering my questions!
Yeah, it helps - good to know I am in the right direction :)
This work will eventually be open-sourced, looking forward to sharing it once I am able to.