Alignment with Gossipsub v1.1 – Envelope & Peer Record Compliance in py-libp2p

[seetadev]

Hi all,

Following a thorough review of the go-libp2p and js-libp2p implementations by @mystical-prog (Vraj), it’s been observed that our current py-libp2p Gossipsub v1.1 implementation is not conforming to the standard envelope and peer-record structure as defined in the spec. This could lead to interop compatibility issues—particularly in scenarios involving peer exchange and signed records validation.

Big thanks to @mystical-prog for catching this.

To move forward, wish to have thoughts, feedback, pointers on:

• Specific deviations from the envelope and peer-record handling compared to the reference implementations.
• A proposed path to align with the libp2p peer record specification and associated protobuf message formats. Please also visit https://www.npmjs.com/package/@libp2p/peer-record and RFC: Signed Address Records specs#217 for reference.
• Whether we should draft a short compatibility checklist or delta doc to guide development milestones and future reviews.

Would be great to hear thoughts from other libp2p maintainers and anyone working on or using py-libp2p with Gossipsub.

Looking forward to collaborating on resolving this. CCing @pacrob, @sumanjeet0012 and @Khwahish29.

[Khwahish29]

Hi all,

Following a thorough review of the go-libp2p and js-libp2p implementations by @mystical-prog (Vraj), it’s been observed that our current py-libp2p Gossipsub v1.1 implementation is not conforming to the standard envelope and peer-record structure as defined in the spec. This could lead to interop compatibility issues—particularly in scenarios involving peer exchange and signed records validation.

Big thanks to @mystical-prog for catching this.

To move forward, wish to have thoughts, feedback, pointers on:

• Specific deviations from the envelope and peer-record handling compared to the reference implementations.
• A proposed path to align with the libp2p peer record specification and associated protobuf message formats. Please also visit https://www.npmjs.com/package/@libp2p/peer-record and RFC: Signed Address Records specs#217 for reference.
• Whether we should draft a short compatibility checklist or delta doc to guide development milestones and future reviews.

Would be great to hear thoughts from other libp2p maintainers and anyone working on or using py-libp2p with Gossipsub.

Looking forward to collaborating on resolving this. CCing @pacrob, @sumanjeet0012 and @Khwahish29.

Having worked on the Peer Exchange and BackOff PR (#690),I can confirm that we consciously adapted the implementation to fit py-libp2p's current PeerInfo abstraction, while keeping in mind the eventual transition to standardized PeerRecord and signed envelope formats. The goal was to enable dynamic peer updates without blocking progress on feature parity.

I agree that aligning with the spec is essential for long-term interop, especially as other libp2p implementations increasingly rely on signed peer records. A compatibility checklist or delta doc, as suggested, would definitely help guide the migration and identify any lingering gaps.

Looking forward to collaborating on the proposed roadmap and ensuring py-libp2p maintains full interop with Go and JS stacks in future releases.

[mystical-prog]

Adding to what already has been shared @Khwahish29 , here's the summary of approach as requested by @seetadev -

Summary of Approach: Peer Exchange and Backoff in Gossipsub v1.1

Peer Exchange (PX)

• Goal: Enable nodes to share information about other peers in the network, allowing for faster mesh recovery and improved robustness.
• Implementation:
  • Extended the ControlPrune protobuf message to include a repeated list of PeerInfo objects.
  • When a node sends a PRUNE message (removing another peer from its mesh), it can also include a list of alternative peers (PX) that the pruned peer can connect to.
  • Added logic to serialize and deserialize PeerInfo objects between nodes.
  • The receiver of PX info attempts to connect to the provided peers, improving their chances of staying connected to the network after being pruned.

Backoff Logic

• Goal: Prevent peers from immediately rejoining a mesh after being pruned or after unsubscribing, which could lead to instability and excessive churn.
• Implementation:
  • Introduced a backoff table that records per-peer, per-topic backoff timers.
  • When a peer is pruned or unsubscribes, a backoff entry is recorded for that peer/topic combination.
  • Any attempts to re-graft (rejoin) during the backoff period are ignored.
  • Backoff is enforced for both PRUNE (default: 60s) and unsubscribe (default: 10s) events, configurable via Gossipsub parameters.
  • Expired backoff entries are cleaned up automatically on access.

Testing

• Added comprehensive tests for the new peer exchange and backoff behaviors, including timing-sensitive scenarios for both pruning and unsubscribing.