Progress Update: Universal Connectivity DApp for py-libp2p

[sumanjeet0012]

Universal Connectivity DApp (TCP-first) Implementation Progress Update

Issue Reference: libp2p/py-libp2p#624

🚀 Current Status

Hey @seetadev, I've been working on implementing the Universal Connectivity DApp for py-libp2p as outlined in issue #624. Here's the current status of my implementation:

✅ What's Working:

1. TCP-based CLI Peer Communication is fully functional

   • ✅ Peers can successfully establish TCP connections over localhost and LAN
   • ✅ GossipSub messaging protocol working correctly with proper mesh formation
   • ✅ Message propagation between multiple peers working reliably
   • ✅ Persistent peer IDs and multiaddr generation/parsing
   • ✅ Connection logging and system event tracking

2. Core Infrastructure Complete

   • ✅ CLI interface with connection, port, and nickname options
   • ✅ Automatic port discovery with manual override capability
   • ✅ Dual logging system (console + file) with separate system event tracking
   • ✅ Graceful shutdown and connection cleanup

3. Multi-Peer Testing Validated

   • ✅ Successfully tested with up to 6+ concurrent peers
   • ✅ Mesh network formation and message propagation verified
   • ✅ Connection stability across different network scenarios

🔧 Current Implementation Structure:

Following standard libp2p patterns with py-libp2p integration:

py-peer/
├── main.py           # ✅ Complete - CLI entry point and host setup
├── chatroom.py       # ✅ Complete - GossipSub integration and message   
└── ui.py             # 🔧 In Progress - Terminal UI with async compatibility issues

Core Implementation Files:

1. main.py - Primary entry point and libp2p host management

   • CLI argument parsing and configuration
   • libp2p host creation with TCP transport
   • GossipSub setup and peer connection logic
   • System event logging and error handling
   • Integration point between CLI and UI modes

2. chatroom.py - Chat functionality and pubsub integration

   • ChatRoom class managing peer-to-peer messaging
   • GossipSub topic subscription and message handling
   • Message serialization/deserialization (JSON-based)
   • Peer discovery and connection state management
   • Real-time message propagation across mesh network

3. ui.py - Terminal user interface implementation

   • Textual-based TUI with chat interface
   • Real-time message display and peer list management
   • Input handling for user message composition
   • System status indicators and connection monitoring
   • Current Challenge: Async compatibility between trio (libp2p) and asyncio (Textual)

Key Achievement: Messages are propagating successfully across the TCP mesh network, validating core py-libp2p functionality.

🤔 Where I Need Guidance - Async Compatibility Challenge:

This is where I could use some input. The CLI version works perfectly, but I'm facing async runtime compatibility issues when implementing the Terminal UI:

Specific Problem:

• py-libp2p uses Trio as its async runtime
• Textual (our UI library) uses asyncio
• Connection Issue: Hosts started with UI cannot connect to hosts started in CLI mode due to incompatible async contexts

Current Approach:

• 🔧 Attempting to solve using anyio as a compatibility layer
• 🔧 Investigating trio-asyncio bridge patterns
• 🔧 Looking into running Textual in a separate thread with proper communication channels

Completed:

• ✅ Multi-peer TCP connectivity across localhost and LAN
• ✅ GossipSub message propagation and mesh formation
• ✅ Connection stability and graceful shutdown
• ✅ CLI interface functionality and error handling

In Progress:

• 🔧 UI integration testing with async compatibility
• 🔧 Interoperability testing with go-peer and js-peer implementations

Still Needed:

• 📋 Unit tests for core components
• 📋 Integration tests for UI components
• 📋 Cross-implementation compatibility validation

[sumanjeet0012]

Universal Connectivity DApp Implementation Progress Update

🚀 Current Status

I've been working on implementing the Universal Connectivity DApp for py-libp2p and have made significant progress on the core functionality. Here's the current status:

✅ What's Working (with applied patches):

• py-libp2p peer successfully running
• Bootstrap connectivity established
• Kademlia DHT integration is functional
• Random walk functionality working as expected
• Cross-implementation communication - py peer successfully communicates with Go peer of Universal Connectivity DApp

🔧 Key Achievement:

The py-peer implementation can now communicate and maintain interoperability with the Go implementation, which is a major milestone for cross-language libp2p compatibility.

🚫 Current Blockers

I'm currently facing several critical issues that are preventing full functionality:

1. Release Availability Issue

• Latest release of py-libp2p is not available, requiring patches to achieve current functionality
• This makes deployment and distribution challenging

2. Bootstrap Initialization Race Condition

• Problem: When using bootstrap during host creation, the peer attempts to connect to bootstrap nodes and negotiate pubsub protocols, which requires pubsub instance.
• Root Cause: Pubsub is not yet initialized at this stage (requires host to be initialized first)
• Impact: Classic chicken-and-egg problem preventing clean bootstrap process

3. Pubsub Protocol Negotiation Crashes

• Problem: When negotiation on pubsub protocol (meshsub) fails, it raises an unhandled exception
• Impact: Entire program crashes instead of graceful fallback (Keep running with raw TCP connectio)
• Severity: Critical - affects network stability

4. Kademlia DHT Initial Peer Discovery

• Problem: When node starts and connects to bootstrap nodes, the routing table remains empty
• Impact: Cannot perform random walk operations, DHT lookups fail
• Expected Behavior: Kademlia DHT should use connected bootstrap peers as initial peers for first lookup operations

🎯 Next Steps

Looking for guidance on:

1. Workarounds for the bootstrap/pubsub initialization order
2. Best practices for handling pubsub negotiation failures gracefully
3. Proper DHT initialization with bootstrap peers
4. Timeline for next py-libp2p release with required fixes.

Testing Environment:

• py-libp2p with custom patches
• py peer with go peer
• Bootstrap nodes configured
• Kademlia DHT with Routing table enabled.

[sumanjeet0012]

@acul71 Here are the specific technical patches and solutions I've implemented to address core py-libp2p functionality issues:

1. Environment Setup and Dependencies

• Action: Created a fresh virtual environment and installed the latest py-libp2p release from libp2p repository.
• Purpose: Ensuring compatibility with the most recent codebase and avoiding dependency conflicts

2. Bootstrap Race Condition Fix

• Issue: Race condition between host initialization and bootstrap process
• Patch: Manual bootstrap start method invocation after host and pubsub initialization.
• Proposed Solution: After Connection manager will be implemented, Bootstrap will correctly adds peers to the peerstore first, then the connection manager attempts connections after proper initialization of both host and pubsub.
  Result: Eliminates timing-related connection failures during startup

3. Protocol Negotiation Exception Handling

• Issue: Pubsub was throwing unhandled exceptions when protocol negotiation failed
• Original Code:

self.event_handle_peer_queue_started.set()
async for peer_id in self.peer_receive_channel:
    # Add Peer
    self.manager.run_task(self._handle_new_peer, peer_id)

• Updated Code (Patch):

self.event_handle_peer_queue_started.set()
async for peer_id in self.peer_receive_channel:
    # Add Peer
    try:
        self.manager.run_task(self._handle_new_peer, peer_id)
    except Exception as e:
        logger.info(f"Protocol negotiation failed: {e}")

• Result: Graceful handling of negotiation failures with proper logging

4. Random Walk and Routing Table Enhancement

• Issue: Random walk functionality failing due to empty routing table
• Root Cause: Initial peer discovery mechanism only added peers to peerstore and tried connecting.
• Patch: Manually filled Routing table with peers from peer store
• proposed solution: For initial Random walk we use Connected peers.
  Later on we will use peers with bootstrap tags which will be done by connection manager.

@acul71 Love to hear your thoughts on these issues.
I will be happy to create issues and apply the fixes in main py-libp2p repo.

[seetadev]

@sumanjeet0012 : Awesome work, Sumanjeet. Will discuss in detail with you soon.