Transport Integration system

[acul71]

Transport Integration System

This document describes the new transport integration system in py-libp2p, which enables dynamic transport selection and easy integration of new transport protocols.

Overview

The transport integration system provides a flexible, extensible architecture for managing different transport protocols in py-libp2p. It replaces the previous hardcoded transport selection with a dynamic registry-based approach.

Key Features

• Dynamic Transport Selection: Automatically selects the appropriate transport based on multiaddr protocols
• Transport Registry: Central registry for all transport implementations
• WebSocket Support: Full support for /ws protocol, /wss in development
• Extensible Architecture: Easy to add new transport protocols
• Backward Compatibility: Existing TCP code continues to work unchanged

Architecture

Transport Registry

The TransportRegistry class manages the mapping between protocol identifiers and transport implementations:

from libp2p.transport import get_transport_registry

registry = get_transport_registry()

# Register a new transport
registry.register_transport("quic", QUICTransport)

# Get a transport class
transport_class = registry.get_transport("tcp")

# Create a transport instance
transport = registry.create_transport("ws", upgrader)

Transport Factory

The create_transport_for_multiaddr() function automatically creates the appropriate transport based on a multiaddr:

from libp2p.transport import create_transport_for_multiaddr

# Automatically creates WebSocket transport
ws_transport = create_transport_for_multiaddr(
    Multiaddr("/ip4/127.0.0.1/tcp/8080/ws"), 
    upgrader
)

# Automatically creates TCP transport
tcp_transport = create_transport_for_multiaddr(
    Multiaddr("/ip4/127.0.0.1/tcp/8080"), 
    upgrader
)

# Note: /wss addresses are not yet supported
# wss_transport = create_transport_for_multiaddr(
#     Multiaddr("/ip4/127.0.0.1/tcp/8080/wss"), 
#     upgrader
# )

Supported Protocols

TCP Transport (/tcp)

• Protocol: /tcp
• Multiaddr Format: /ip4/127.0.0.1/tcp/8080
• Implementation: libp2p.transport.tcp.tcp.TCP
• Dependencies: None
• Status: ✅ FULLY IMPLEMENTED

WebSocket Transport (/ws)

• Protocol: /ws
• Multiaddr Format: /ip4/127.0.0.1/tcp/8080/ws
• Implementation: libp2p.transport.websocket.transport.WebsocketTransport
• Dependencies: TransportUpgrader
• Status: ✅ FULLY IMPLEMENTED

WebSocket Secure Transport (/wss)

• Protocol: /wss
• Multiaddr Format: /ip4/127.0.0.1/tcp/8080/wss
• Implementation: libp2p.transport.websocket.transport.WebsocketTransport
• Dependencies: TransportUpgrader
• Status: ❌ NOT YET IMPLEMENTED - Currently throws NotImplementedError("/wss (TLS) not yet supported")
• Note: TLS support planned for future implementation

Usage Examples

Basic Usage

from libp2p import new_host
import multiaddr

# Create host with WebSocket transport
host = new_host(listen_addrs=[
    multiaddr.Multiaddr("/ip4/127.0.0.1/tcp/8080/ws")
])

# Create host with TCP transport
host = new_host(listen_addrs=[
    multiaddr.Multiaddr("/ip4/127.0.0.1/tcp/8080")
])

Custom Transport Registration

from libp2p.transport import register_transport

class CustomTransport:
    async def dial(self, maddr):
        # Implementation
        pass
    
    def create_listener(self, handler):
        # Implementation
        pass

# Register the custom transport
register_transport("custom", CustomTransport)

# Now it can be used in multiaddrs
# /ip4/127.0.0.1/custom/8080

Direct Transport Creation

from libp2p.transport import create_transport
from libp2p.transport.upgrader import TransportUpgrader

upgrader = TransportUpgrader({}, {})

# Create specific transport types
tcp_transport = create_transport("tcp")
ws_transport = create_transport("ws", upgrader)
# wss_transport = create_transport("wss", upgrader)  # Not yet implemented

Integration with libp2p

The new transport system integrates seamlessly with existing libp2p code:

Automatic Transport Selection

When you call new_host() or new_swarm() with listen_addrs, the system automatically:

1. Parses the multiaddr to identify the transport protocol
2. Creates the appropriate transport instance
3. Falls back to TCP if no specific transport is found
4. Provides clear error messages for unsupported protocols

No Breaking Changes

Existing code continues to work unchanged:

# This still works exactly as before
from libp2p import new_host

# Automatically uses TCP transport
host = new_host()

# Automatically uses WebSocket transport
host = new_host(listen_addrs=["/ip4/127.0.0.1/tcp/8080/ws"])

Adding New Transport Protocols

To add a new transport protocol:

1. Implement the Transport Interface

from libp2p.abc import ITransport
from libp2p.custom_types import THandler

class NewTransport(ITransport):
    async def dial(self, maddr):
        # Implementation
        pass
    
    def create_listener(self, handler: THandler):
        # Implementation
        pass

2. Register the Transport

from libp2p.transport import register_transport

register_transport("new_protocol", NewTransport)

3. Use in Multiaddrs

# Now you can use /new_protocol in multiaddrs
maddr = Multiaddr("/ip4/127.0.0.1/new_protocol/8080")

Testing

The transport system includes comprehensive tests:

# Run transport registry tests
pytest tests/core/transport/test_transport_registry.py

# Run WebSocket transport tests
pytest tests/core/transport/test_websocket.py

# Run all transport tests
pytest tests/core/transport/

Demo Scripts

Transport Integration Demo

python examples/transport_integration_demo.py

This demonstrates:

• Transport registry functionality
• Dynamic transport creation
• Multiaddr-based transport selection
• Custom transport registration

WebSocket Demo

# Start WebSocket server
python examples/websocket/websocket_demo.py -p 8080

# Connect to server  
python examples/websocket/websocket_demo.py -p 8081 -d /ip4/127.0.0.1/tcp/8080/ws/p2p/...

Error Handling

The system provides clear error messages for common issues:

# Unsupported protocol
try:
    host = new_host(listen_addrs=["/ip4/127.0.0.1/udp/8080"])
except ValueError as e:
    print(e)  # "Unknown transport in listen_addrs: ['/ip4/127.0.0.1/udp/8080']. Supported protocols: ['tcp', 'ws']"

# Missing upgrader for WebSocket
try:
    transport = create_transport("ws")  # Missing upgrader
except ValueError as e:
    print(e)  # "WebSocket transport requires an upgrader"

Performance Considerations

• Lazy Loading: Transports are created only when needed
• Registry Caching: Transport registry is cached globally
• Minimal Overhead: Transport selection adds minimal runtime overhead
• Memory Efficient: Transports are created on-demand and can be garbage collected

Future Enhancements

Planned improvements include:

• TLS WebSocket Support: Implementation of /wss protocol (planned)
• QUIC Transport: Native QUIC protocol support
• Transport Metrics: Performance monitoring and metrics
• Load Balancing: Multiple transport support for the same peer
• Protocol Negotiation: Runtime protocol negotiation between peers

Troubleshooting

Common Issues

1. Transport Not Found: Ensure the transport is registered in the registry
2. Upgrader Required: WebSocket transports require a TransportUpgrader instance
3. Protocol Mismatch: Check that the multiaddr uses supported protocols
4. WSS Not Supported: /wss addresses currently throw NotImplementedError("/wss (TLS) not yet supported") - use /ws for now

Debug Mode

Enable debug logging to see transport selection:

import logging
logging.basicConfig(level=logging.DEBUG)

Getting Help

• Check the test files for usage examples
• Review the demo scripts for working implementations
• Enable debug logging to trace transport selection
• Check that all required dependencies are installed

Conclusion

The new transport integration system provides a robust, extensible foundation for py-libp2p transport protocols. It maintains backward compatibility while enabling easy integration of new transport types and automatic protocol selection based on multiaddrs.

The system is production-ready and provides a solid foundation for future transport protocol implementations.