Improve RSA Key Compatibility, Public Key Extraction from Peer IDs, and Pubsub Connection Handling

[sumanjeet0012]

Description

Problem Statement

The current implementation of py-libp2p has few limitations that affect universal connectivity interoperability:

1. RSA Key Compatibility Issue

• RSAPublicKey.from_bytes() fails when parsing certain PKIX-formatted DER keys that PyCryptodome cannot handle
• This causes interoperability issues with RSA keys generated by other libp2p implementations (Go, Rust, JavaScript)
• The error prevents establishing connections with peers using different key formats

2. Missing Public Key Extraction from Peer IDs

• No mechanism to extract public keys from peer IDs when they use identity multihash
• This is a standard libp2p feature where small keys (Ed25519, Secp256k1) are embedded directly in peer IDs
• Required for proper signature verification in pubsub and other protocols
• Currently, the implementation always expects keys to be sent separately, which is inefficient for small keys

3. Pubsub Connection Management Issues

• Premature peer removal: Peers are removed from pubsub when ANY connection closes, even if multiple connections exist
• Duplicate streams: No check to prevent opening multiple pubsub streams to the same peer
• Service crashes: Protocol negotiation failures can crash the entire pubsub service

Proposed Solution

1. Enhanced RSA Key Parsing

• Add fallback to cryptography library when PyCryptodome fails to parse a key
• Convert incompatible formats (PKIX) to PKCS1 format that PyCryptodome understands
• Maintain backward compatibility with existing keys

2. Implement Peer ID Public Key Extraction

• Add extract_public_key() method to the ID class
• Check if peer ID uses identity multihash (hash function code 0x00)
• Extract and deserialize the embedded public key if present
• Return None for peer IDs using SHA-256 hash (typically RSA keys)

3. Update Pubsub Signature Validation

• Modify signature validator to first attempt extracting the public key from peer ID
• Fall back to using msg.key field if extraction fails or returns None
• Add detailed logging for debugging key resolution
• Align with libp2p specification for pubsub message signing

4. Improve Pubsub Connection Management

• Add duplicate stream detection before opening new streams
• Check for active connections before removing peers from pubsub
• Wrap peer handling in exception handlers to prevent service crashes

Expected Benefits

✅ Better Interoperability: Can parse RSA keys from any libp2p implementation
✅ Spec Compliance: Follows libp2p spec for key embedding in peer IDs
✅ More Robust: Handles edge cases in pubsub connection management
✅ Easier Debugging: Comprehensive logging for troubleshooting connection issues
✅ Performance: Reduces message overhead for small keys (no need to send key in every message)