Feat/add webrtc transport

[Nkovaturient]

Description

• This PR introduces a comprehensive WebRTC transport implementation for py-libp2p, enabling browser-to-browser, browser-to-server, and server-to-server real-time peer-to-peer connections

Issue #546

Updates

✅ Core Transport Implementation

• Private-to-Private WebRTC: Circuit relay-based signaling for NAT traversal
• WebRTC-Direct: UDP hole punching for direct peer connections
• Stream Multiplexing: Multiple protocols over single WebRTC data channels
• Certificate Management: Self-signed certificates with proper hash encoding
• Protocol Registration: Full multiaddr support with WebRTC, WebRTC-Direct, and certhash protocols

✅ js-libp2p Compatibility

• Protocol Codes: Exact match with js-libp2p specifications (0x0119, 0x0118, 0x01d2)
• Multiaddr Format: Compatible address formatting for cross-implementation connections
• Signaling Protocol: /libp2p/webrtc/signal/1.0.0 message format compliance
• Certificate Format: uEi prefixed base64url certificate hashes

✅ Production-Ready Features

• Async Bridge: Robust trio-asyncio integration for WebRTC operations
• Timeout Protection: Network operations complete in 3s with graceful fallback
• Error Handling: Comprehensive error states and recovery mechanisms
• Resource Cleanup: Proper connection and stream lifecycle management

Technical Architecture & Design Decisions

Why trio-asyncio Instead of Pure Trio?

Core Challenge: py-libp2p uses Trio for async operations, but aiortc (the robust Python WebRTC library) is built entirely on asyncio. This created a fundamental integration challenge that required careful architectural decisions.

Solution: I have implemented a sophisticated trio-asyncio bridge (WebRTCAsyncBridge) that provides:

• Context-managed integration: Safe async context handling across both frameworks
• Trio token capture: Cross-thread communication from asyncio callbacks to trio contexts
• Resource lifecycle management: Proper cleanup of both trio and asyncio resources
• Performance optimization: Minimal overhead bridge operations with connection pooling

Alternative Considered: Writing a pure trio WebRTC implementation would eliminate bridge complexity but would require reimplementing significant portions of WebRTC protocols (DTLS, SCTP, ICE) - a massive undertaking that would delay delivery and introduce bugs.

The trio-asyncio Bridge: Why Essential?

# aiortc operations are asyncio-native:
peer_connection = RTCPeerConnection(config)  # asyncio context required
offer = await peer_connection.createOffer()   # asyncio coroutine

# py-libp2p expects trio operations:
stream = await host.new_stream(peer_id, protocols)  # trio context
await stream.write(data)  # trio async

# Our bridge seamlessly connects both:
async with WebRTCAsyncBridge():
    offer = await bridge.create_offer(peer_connection)  # trio-safe operation

Bridge Benefits:

• Thread Safety: Handles asyncio callbacks safely in trio contexts via trio tokens
• Resource Management: Ensures proper cleanup of both asyncio and trio resources
• Context Isolation: Prevents context bleeding between async frameworks
• Performance: Minimal overhead with connection reuse and smart batching

Current Status

Main Test Suite (test_webrtc_transport.py)

• 15 comprehensive tests covering basic functionality, interoperability, and advanced features
• Network-independent operation with smart timeout handling
• Real WebRTC connection testing with SDP generation and data channel setup

Specialized Test Suites

• test_js_libp2p_interop.py: Dedicated js-libp2p compatibility validation
• test_live_signaling.py: Live signaling with circuit relay simulation
• test_network_optimized.py: Network-independent testing for CI/CD environments

Questions

1. Performance Impact: How does the trio-asyncio bridge affect performance in high-throughput scenarios with 100+ concurrent WebRTC connections? Should we prioritize a pure trio WebRTC implementation for v2?

2. Stream Multiplexing Integration: How should WebRTC streams integrate with existing yamux/mplex stream multiplexing? Should WebRTC connections be treated as muxed connections themselves, or as transport-level primitives?

3. Circuit Relay Strategy: What's the preferred integration path for circuit relay reservations with the existing relay discovery mechanism? Should WebRTC transport handle relay discovery independently or leverage existing infrastructure?

4. NAT Traversal Prioritization: When both STUN servers and direct UDP hole punching are available, what's the preferred fallback hierarchy? How do we balance connection speed vs reliability?

5. Resource Management Philosophy: Given the trio-asyncio bridge complexity, should we add connection pooling and resource limits to prevent memory leaks in long-running applications?

Next work

• Implementing IHost for utilising real network resources for addressing webrtc listeners and get_network [currently on timeout and mocks ]
• TODO: Return circuit relay addresses that can be used for WebRTC signaling in private-to-private transport.py. [Must be modular for its extension and utilisation in webrtc-direct transport as well]
• Implement and test lacunae in NAT Traversal and Circuit-Relay Implementations
• Implement pubsub-based offer/answer exchange in WebRTC-Direct transport
• Demonstrate complete end-to-end connectivity via webRTC Private-to-Private & Private-to-Public Connections
• Demonstrate direct WebRTC connection with NAT traversal
• Demonstrate P2P connection through circuit relay

Cute Animal Picture

[sukhman-sukh]

Hey @Nkovaturient, after our discussion, I have made some changes to the SDP and ICE exchange protocols for it to be interoperable with JS.
Key changes:

1. Replaced JSON with .proto serialization
2. Removed message-length bytes at start of message for now (as it was not in JS. We can add later if needed)
3. Fixed the ICE_candidate object creation, sending, and receiving handlers.
4. Fixed some linting errors.
   Please have a look at the changes.
   Now, I will move ahead with the above checkboxes.

Also, in the meantime, can you check for pyrefly typecheck and try to fix them?

[Nkovaturient]

Also, in the meantime, can you check for pyrefly typecheck and try to fix them?

Sure, gonna fix them.

[seetadev]

@Nkovaturient , @sukhman-sukh : Great work :)

Added some feedback on the tests proposed at #839 .

Wonderful progress indeed. Reviewing the PR and looking forward to successful completion of the WebRTC direct PR in the coming days. CCing @pacrob, @Winter-Soren, @AkMo3, @acul71, @guha-rahul and @lla-dane for their feedback and pointers.

Looking forward to seeing WebRTC direct in production soon :)

[sukhman-sukh]

Hey @seetadev,
I don't know why this is happening but on my local I have 2 more commits which are not getting pushed.
I tried forced push too but it is not showing on github.
Do you have any idea?

commit 177c14939fa785780bce43bff99401ab9101f6ca (HEAD)
Author: sukhman <[email protected]>
Date:   Tue Aug 12 03:28:00 2025 +0530

    Fix lint error in CI

commit bd7f93a6192b3a610162cd57fe37d695e2157c8a
Author: sukhman <[email protected]>
Date:   Mon Aug 11 06:20:19 2025 +0530

    Add stream read/write for webrtc

Add stream is the second last and Fix lint is the last commit

[sukhman-sukh]

Also, I am facing difficulty in some functions of listener which for now I have created as a placeholder as JS has something called ICEUDPMUXListener in its webrtc's base library for listening to STUN requests for webrtc while I can't find anything equivalent in python (still looking for some workarounds), in worst case we will have to implement it.
Ref: https://github.com/libp2p/js-libp2p/blob/cf9aab5c841ec08bc023b9f49083c95ad78a7a07/packages/transport-webrtc/src/private-to-public/listener.ts#L154-L155
It would be really nice if you could guide me how to go ahead with this.