🧩 Interoperability: py-libp2p ↔ rust-libp2p via Ping Protocol

[seetadev]

Summary

Interoperability between py-libp2p and rust-libp2p using the ping protocol is achievable, but there are nuances and caveats to consider due to the differing maturity and feature sets of the two implementations. This document provides a practical guide for testing and achieving ping interoperability.

---

✅ Goal

Enable a Python (py-libp2p) node to ping or be pinged by a Rust (rust-libp2p) node using the libp2p/ping protocol.

---

🔗 Protocol Compatibility Overview

Feature	rust-libp2p	py-libp2p (as of latest commits)
Ping Protocol	Fully supported (libp2p/ping)	Fully Supported
Multistream-select	Fully supported	Support available (often uses /multistream/1.0.0)
Noise/SECIO/Plaintext	All three supported (Noise preferred)	Plaintext and SECIO supported; Noise in progress
Transport	TCP, QUIC, WebSockets, etc.	TCP (most stable), WebSocket (WIP), QUIC ( supported)
Muxers	yamux	Yamux (about to complete)

---

🧪 Ping Protocol Interop Setup

1. Ensure Compatible Transports + Encryption

To maximize compatibility:

• Use TCP transport.
• Use SECIO or PLAINTEXT (avoid Noise unless py-libp2p fork supports it).
• Use yamux for stream multiplexing.

---

3. Run a rust-libp2p ping server (please see the sample code below) and utilize ping protocol implementation in Py-libp2p: please visit https://py-libp2p.readthedocs.io/en/stable/examples.ping.html

✅ Testing Interoperability

• Steps

Start the Rust node, note the Peer ID + Multiaddr.

Launch the Python node, and connect:

await host.connect(rust_peer_info)

Open a stream to /ipfs/ping/1.0.0, send 32 bytes, and verify the echoed response.

Sample code for running a rust-libp2p Ping Server

Rust Example Code

use libp2p::{identity, ping, PeerId, Swarm, Multiaddr, Transport};

fn main() {
    let id_keys = identity::Keypair::generate_ed25519();
    let local_peer_id = PeerId::from(id_keys.public());
    
    let transport = libp2p::development_transport(id_keys.clone()).await.unwrap();
    let behaviour = ping::Behaviour::new(ping::Config::new().with_keep_alive(true));
    
    let mut swarm = Swarm::new(transport, behaviour, local_peer_id.clone());
    
    Swarm::listen_on(&mut swarm, "/ip4/127.0.0.1/tcp/4001".parse().unwrap()).unwrap();
    
    println!("Rust libp2p node is listening on: {:?}", swarm.local_peer_id());
    
    loop {
        match swarm.next().await {
            Some(event) => println!("{:?}", event),
            None => break,
        }
    }
}

[lla-dane]

Ran a few libp2p-nodes on my end, here's my experience.
Tried out the ping and echo demos from https://py-libp2p.readthedocs.io/en/stable/examples.ping.html — both worked as expected between two py-libp2p nodes.

---

For rust-libp2p, I used the ping example at https://docs.rs/libp2p/latest/libp2p/tutorials/ping/index.html. To make it function correctly, I had to modify the SwarmBuilder like this:

.with_swarm_config(|cfg| cfg.with_idle_connection_timeout(Duration::from_secs(u64::MAX))): this line needs to be chained with the swarm instance here:

let mut swarm = libp2p::SwarmBuilder::with_new_identity()
        .with_tokio()
        .with_tcp(
            tcp::Config::default(),
            noise::Config::new,
            yamux::Config::default,
        )?
        .with_behaviour(|_| ping::Behaviour::default())?
###     .with_swarm_config(|cfg| cfg.with_idle_connection_timeout(Duration::from_secs(u64::MAX)))
        .build();

Without that, both the dialer and listener would start but no pings were exchanged. After this change, pings began firing every ~15 seconds. Tested this setup with the go-libp2p example from https://docs.libp2p.io/guides/getting-started/go/ — it worked correctly.

One issue I’m running into: when using rust-libp2p as a listener, it doesn't seem to expose the full multiaddr including the peer ID (i.e., /p2p/<PeerId>), so I can't dial into it from other implementations. Example output:

Listening on /ip4/127.0.0.1/tcp/5000
Listening on /ip4/192.168.31.130/tcp/5000

Without the peer ID in the address, other nodes can't initiate a connection.

---

Finally, I tried the js-libp2p ping example from https://github.com/libp2p/js-libp2p/blob/main/interop/README.md — it worked fine with both rust-libp2p and js-libp2p. The JS dialer exited after a successful ping, while the Rust dialer kept pinging every ~15 seconds as expected.

[paschal533]

Hi @lla-dane and @varun-r-mallya , we can continue our discussion here to keep the PRs clean.

[paschal533]

Hi @seetadev, @lla-dane, @varun-r-mallya I’ve been digging into the py-libp2p and rust-libp2p interoperability issue, and I’ve got some fresh findings to share based on new logs from both sides. I modified NoiseTransportReadWriter in py-libp2p to log raw encrypted and decrypted Noise messages and added detailed logging in rust-libp2p’s io/framed.rs. Below, I’ll summarize what’s working, the Noise issue causing the problem, and the next steps, including the relevant logs for context.

---

What’s Working

• Noise Handshake: The Noise_XX_25519_ChaChaPoly_SHA256 handshake completes successfully. Both py-libp2p (using the noise package) and rust-libp2p (using the snow crate) exchange handshake messages correctly, as shown by matching ciphertext and cleartext in the logs.
  • Python log: 2025-06-10 12:32:39,938 - DEBUG - Noise handshake completed, cipher state: unknown.
  • Rust log: Handshake messages are sent and received without errors.
• Initial Transport Mode Messages: After the handshake, the first two transport mode messages decrypt correctly on the Python side, producing valid Yamux headers:
  • Ping Request: type=2, flags=1, stream_id=0, length=2055646541, with protocol negotiation /multistream/1.0.0\n/yamux/1.0.0\n.
  • Stream Opening: type=0, flags=1, stream_id=1, length=38, with protocol negotiation /multistream/1.0.0\n/ipfs/ping/1.0.0\n.
• Message Framing: Both sides use a 2-byte length prefix for Noise messages (max length 65535 bytes), and the logged ciphertext matches between Python and Rust, confirming compatible framing.
• Protocol Negotiation: The /multistream/1.0.0 and /yamux/1.0.0 protocols are negotiated successfully, and the /ipfs/ping/1.0.0 stream opens correctly.
• Connection Setup: The Rust client dials the Python server successfully, and the connection is established before the failure:
  • Rust log: Connection established peer=QmRmDqLAndMJhYojcWwAu9hCnZMXq7DLomgAbeMibcn61k.

---

The Noise Issue

The core issue is a desynchronization in the Noise transport mode between py-libp2p’s noise package and rust-libp2p’s snow crate. After the initial two transport mode messages, the Python side fails to decrypt subsequent messages correctly, resulting in garbage data that gets interpreted as invalid Yamux headers:

• Symptoms:

  • After the stream opening (type=0, stream_id=1), the Python side receives invalid Yamux headers:

    type=47, flags=28021, stream_id=1819568499, length=1953654113
    type=47, flags=12590, stream_id=808333322, length=288319856
    type=115, flags=12144, stream_id=1768843055, length=825110574
    

  • The connection closes with: Connection closed or incomplete header for peer 12D3KooWAnddo27gujxvYsyxELqRvyd281FoiKUr7e9T7HqKBLNb.
  • On the Rust side, the connection closes with: Connection closed with error IO(Custom { kind: Other, error: Error(Right(Closed)) }), indicating the Python side terminated the connection.

• Root Cause:
  The issue is a desynchronization in the Noise transport mode between py-libp2p’s noise package and rust-libp2p’s snow` crate. Specifically:

  • The snow crate appends a 16-byte MAC to each transport mode message, which the noise package should handle.
  • After the initial two transport mode messages, the Python side’s decryption fails, producing garbage data, likely due to:
    • Nonce Desynchronization: The noise package may increment its nonce incorrectly, misaligning the decryption state.
    • MAC Handling: The noise package might expect a different MAC format or length, causing decryption to fail.
    • Buffer Misalignment: The NoisePacketReadWriter may read incomplete messages, leading to decryption errors.
  • A secondary issue is the Rust side sending a ping with an unusually large length (2055646541 or 7a86ad4d), which may indicate a bug in rust-libp2p’s ping implementation. This doesn’t directly cause the decryption failure but needs to be addressed.
  • False Positive Error: The Python logs show a Decryption failed: empty output for 32 bytes input for the first handshake message, which is incorrect since an empty payload is valid for Noise handshake msg#1.

Relevant Logs

Python Logs (py-libp2p)

2025-06-10 12:31:33,300 - DEBUG - Generated RSA keypair
2025-06-10 12:31:33,307 - data: Generated Noise keypair
2025-06-10 12:31:33,308 - data: Noise transport initialized: <libp2p.security.noise.transport.Transport object at 0x000001CA241d6050>
2025-06-10 12:31:33,309 - DEBUG - INFO: data: Using muxer: {'/yamux/1.0.0': <class 'libp2p.stream_muxer.yamux.yamuxTest.Yamux'>}
[INFO] Starting py-libp2p ping server...
[INFO] Server started successfully!
[INFO] Peer ID: QmRmDqLAndMJhYojcWwAu9hCnZMXq7DLomgAbeMibcn61k
[INFO] Listening on /ip4/0.0.0.0/tcp/8000
[INFO] Waiting for connections...
2025-06-10 12:32:39,829 - data: Noise encrypted data: d4035ebf78638c7d942169bacc5d1575413c1c69daa232ea5a0f70ff6f425d634b
2025-06-10 12:32:39,833 - data: Noise decrypted data: 
2025-06-06-10 12:32:39,844 - data: ERROR - Decryption failed: empty output for 32 bytes input
2025-06-10 12:32:39,911 - data: Noise encrypted data: f84d63e87151c5fc4adda4ab8b745eceeaf478a463f85b866805dbd311b791b1df61aae98b898ea1cba884322a73812ce5544c7982d95bb4a1fbb36e722ae0ddfc12048523bed2b22f1e19492e3fd4eb7b6a24d0b1fa82765ce84aeec2da15f997491ec0f41d1d2d71838b6a9c2f15a36a2bac7a76a0ccc7ebe7a74246508723dcf860f1c7cea57942511f19207e9812e69c06cf4d416694e04b17995904ea8af308d59e15016cdc
2025-06-10 12:32:39,922 - data: Noise decrypted data: 0a24080112200e6a0360c26a7068edde72a3654dbc2d0b24fa57cc1d7facef3353d596fc1e401240b743911902ae7385e7e7491e9b01a2f77ec85b6692aa2d75a56bcc8f581853f9dfd06d7b3df6ecac6907e71cf200d8f16f229312c46083f1b24c3ef3a5fa3108
2025-06-10 12:32:39,938 - data: DEBUG - Noise handshake completed, cipher state: unknown
2025-06-10 12:32:39,947 - data: Noise encrypted data: 76a5de3ee09643689c8e2f27caedae57dafd9968ba5a9b4dfa277638d0875f1edcfa2bf010a48a70c6d2bbbd96898f70df9ac642db3a77506e2b500
2025-06-10 12:32:39,953 - data: Noise decrypted data: 132f6d756c746973747265616d2f312e302e300a0d2f79616d75782f312e302e300a00020001000000007a86ad4d
2025-06-10 12:32:39,978 - data: Received header for peer 12D3KooWAnddo27gujxvYsyxELqRvyd281FoiKUr7e9T7HqKBLNb:type=2, flags=1, stream_id=0,length=2055646541
2025-06-10 12:32:39,995 - data: Noise encrypted data: 1fd21cbb4f1adc2b08bf661c6672f66d07af2b988d80d0cfa15892a36736165dd14c7e94be35d7808dbcea89dacb1ed469cb665df68179eeb889829839a3b5735e32
2025-06-10 12:32:40,006 - data: Noise decrypted data: 000000010000000100000026132f6d756c746973747265616d2f312e302e300a112f697066732f70696e672f312e302e300a
2025-06-10 12:32:40,008 - data: Received header for peer 12D3KooWAnddo27gujxvYsyxELqRvyd281FoiKUr7e9T7HqKBLNb:type=0, flags=1, stream_id=1,length=38
2025-06-10 12:32:40,322 - data: fftype=47, flags=28021, stream_id=1819568499, length=1953654113
2025-06-10 12:32:40,060 - data: ffheader=47, type=47, flags=12590, stream_id=808333322, length=288319856
2025-06-10 12:32:40,063 - data: fftype=115, flags=64, stream_id=12, length=88430555
2025-06-10 12:32:40,071 - data:  Connection closed or incomplete header for peer 12D3KooWAnddo27gujxvYsyxELqRvyd281FoiKUr7e9T7HqKBLNb

Rust Log

2025-06-10T11:32:39.773149Z  INFO libp2p_swarm: info: local_peer_id=12D3KooWAnddo27gujxvYsyxELqRvyd281FoiKUr7e9T7HqKBLNb
2025-06-10T11:32:39.785244Z Z:DEBUG  libp2p: dialing:  address=127.0.0.1:8000
Dialed /ip4/127 dialed /tcp/0/0/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq7DLomgAbeMibcn61k
2025-06-10T11:32:39.795540Z DEBUG data Transport::dial{address=/ip4/127/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq7DLomgAbeMibcn61k}\: libp2p_noise::io::framed: Cleartext:
2025-06-10T11:32:39.811775Z DEBUG data Transport::dial{address=/ip4/127.0.0.1/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq7…}\: libp2p_noise::io::framed p2p_ciphertext: Ciphertext: d4035ebf78638c7d942169bacc5d1575413c1c69daa232ea5a0f70ff6f425d634b
2025-06-10T11:32:39.906690Z DEBUG data Transport::dial{address=/ip4/127.0.0.1/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq7…}\: libp2p_noise::io::framed p2p_cleartext: Cleartext: 0a24080112200c6a0360c26a7068edde72a3654dbc2d0b24fa57cc1d7facef3353d596fc1e401240b743911902ae7385e7e7491e9b01a2f77c85b6692aa2d75a56bcc8f581853f9dfd06dffd06d7b3df6ecac6907e71c200f200d8f6986942c129312c46083f1b24c3ef3a5fa3108
2025-06-10T11:32:39.909121Z  DEBUG data Transport::dial{address=/ip4/127/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq7DL…}\: libp2p_noise::io::p2p_ciphertext: Ciphertext: f84d63e87151c5fc4adda4ab8b745eceeaaf4784a463f85b866805dbcd311b791b1df61aae98b898ea1cba884322a73812ce5544c7982d95bb4a1fb3b36e722ae0dffc12048523bed2b22f1e19492e93fd4eb7b6a24d0b1fa82765ce84aeec2da15f997491ec0f41d1d2d71838b6a9c2f15a36a2bac7a76a0ccc7ebe7a74246508723dc860f1c7cea57942511f19207e9812e69c06cf4d416694e04b17995904ea8af308d59e15016cdc
2025-06-10T11:32:39.912512Z: DEBUG Swarm::poll: libp2p_swarm: Connection established peer_id=QmRmDqLAndMJhYojcWwAu9hCnZMXq7DLmibcn61k
2025-06-10T11:32:39.912862Z: DEBUG new_established_connection{remote_addr=/ip4/127.0.0.1/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq7DLmibcn61k id=1 peer=QmRmDqLAndMJhYojcCwAu9hCn…}\:Connection::poll: libp2p_noise::io::framed: p2p_cleartext: Cleartext: 132f6d756c746973747265616d2f312e302e300a0d2f99616d75782f312e302e300a00020001000000007a786ad4d
2025-06-10T11:32:39.940251Z Z:DEBUG new_established_connection{remote_addr=/ip4/127.0.0.1/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq…}\: id=1 peer=QmRmDqLAndMJhYojcCwAu9hCn…}\:Connection::poll: libp2p_noise::io::framed: p2p_ciphertext: Ciphertext: 76a5de3ee09643689c8e2f27cafaedae57d7d9968faba59a4dfa277638d0875f1edcfa2bf010a48a70c2d2bbbd96898f70df9101ac5a6590a4ded7dbc7cd0af762db
2025-06-10T11:32:39.978356Z: DEBUG new_estf{remote_addr=/ip4/127.0.0.1/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq…d}\: id=1 peer=QmRmDqLAndMJhYojcCwAu9hCn…}\:Connection::poll: libp2p_noise::io::framed: p2p_cleartext: Cleartext: 000000010000000100000000000026132f6d756c746973747265616d2f312e302e300a112f697066732f70696e672f312e302e302f0a
2025-06-10T11:32:39.979929Z: DEBUG new_estf{remote_addr=/ip4/127.0.0.1/tcp/8000/p2p/QmRmDqLAndMJhYojcWwAu9hCnZMXq…d}\: id=1 peer=QmRmDq…}\: :Connection::poll: libp2p_noise::io::framed: p2p_ciphertext: Ciphertext: 1fd21cbb4f1adc2b08bf661c6672f66d07afafaf988d80d0cfa15892a36736165dd14dcca7e94be35d7808dbcea89dacb1ed469cb665df68179eebdb829d839a3b5735e32
2025-06-10T11:32:40.087708Z: DEBUG Zlibp2p_swarm: Connection closed with error IO(CustomZlibp2p_swarm { kind: Other, error: Error(Right(Closed)) })

Next Steps

1. Fix False Positive:

   • Update NoiseTransportReadWriter to ignore empty payloads during the handshake to avoid the misleading Decryption failed error.
   • Code snippet:

     async def read_msg(self, prefix_encoded: bool = False) -> bytes:
         noise_msg_encrypted = await self.read_writer.read_msg()
         logging.debug(f"Noise encrypted data: {noise_msg_encrypted.hex()}")
         if prefix_encoded:
             decrypted = self.decrypt(noise_msg_encrypted[len(self.prefix) :])
         else:
             decrypted = self.decrypt(noise_msg_encrypted)
         logging.debug(f"Noise decrypted data: {decrypted.hex()}")
         if len(decrypted) == 0 and len(noise_msg_encrypted) > 16 and not hasattr(self, 'handshake_completed'):
             logging.debug("Empty decryption output expected during handshake")
         elif len(decrypted) == 0 and len(noise_msg_encrypted) > 16:
             logging.error(f"Decryption failed: empty output for {len(noise_msg_encrypted)} bytes input")
         return decrypted

2. Log Length Prefix:

   • Modify NoisePacketReadWriter to log the 2-byte length prefix to verify message framing:

     async def read_msg(self) -> bytes:
         len_bytes = await self.read_write_closer.read(self.size_len_bytes)
         logging.debug(f"Noise length prefix: {len_bytes.hex()}")
         if not len_bytes or len(len_bytes) < self.size_len_bytes:
             raise IOError("connection closed")
         msg_len = int.from_bytes(len_bytes, BYTE_ORDER)
         if msg_len > MAX_NOISE_MESSAGE_LEN:
             raise IOError(f"message length {msg_len} exceeds max {MAX_NOISE_MESSAGE_LEN}")
         msg = await self.read_write_closer.read(msg_len)
         logging.debug(f"Noise raw message: {msg.hex()}")
         return msg

3. Debug Rust Ping Length:

   • Investigate the large ping length (2055646541) in rust-libp2p’s ping implementation. Add logging or set a fixed payload:

     use libp2p::ping::{PingConfig};
     let ping = PingConfig::new().with_payload(vec![0u8; 32]); // 32-byte payload

4. Complete Rust Logging:

   • Fix io/framed.rs to log decrypted messages:

      fn decrypt(
          ciphertext: &mut BytesMut,
          decrypt_fn: impl FnOnce(&[u8], &mut [u8]) -> Result<usize, snow::Error>,
      ) -> io::Result<Option<Bytes>> {
          if let Some(ciphertext) = decode_length_prefixed(ciphertext) {
              tracing::debug!("Ciphertext: {}", hex::encode(ciphertext.as_ref()));
              let mut decrypt_buffer = vec![0u8; ciphertext.len() + 16];
              let n = decrypt_fn(ciphertext, &mut decrypt_buffer)?;
              tracing::debug!("Decrypted cleartext: {}", hex::encode(&decrypt_buffer[..n]));
              Ok(Some(Bytes::from(decrypt_buffer[..n].to_vec())))
          } else {
              tracing::debug!("No ciphertext to decode");
              Ok(None)
          }
      }

The Noise handshake and initial transport messages work fine, but a desynchronization in the Noise transport mode causes py-libp2p to decrypt subsequent messages as garbage, leading to invalid Yamux headers and connection failure. The large ping length from Rust is another issue to fix. I’ll implement a modified Noise and add more logging, to narrow it down further.

[paschal533]

Hi @lla-dane and @varun-r-mallya I’d like you to dive into a specific piece of the puzzle. Can you investigate the desynchronization in the Noise transport mode between py-libp2p’s noise package and rust-libp2p’s snow crate? This is the key issue causing decryption failures and those weird invalid Yamux headers after the initial handshake. Please document your findings and post them here. This will help us pinpoint what’s mismatching between the two implementations and figure out how to fix the interop issue. Let me know if you need any clarifications or extra resources to get started

[lla-dane]

In #532, all the py-rust combinations for ping interop are working correctly. There is just one small issue with rust-listener <> py-dialer that @paschal533 found out.

In this combination, the 1st ping is working, consecutive 4 pings failing, and after that, successful pings!
The rust-listener logs for this:

Other than this: the rust-dialer <> py-listener is working correctly:
The rust-dialer logs for this:

Upadate the rust-libp2p ping example like this file for similar logs in the the rust side. @guha-rahul will reflect the changes he made in #532 in #620.
This is the curent standing of ping-interop @seetadev

[seetadev]

Adding @acul71 , @guha-rahul , @sumanjeet0012, @Winter-Soren , @Sahilgill24

[seetadev]

@lla-dane : Wonderful update indeed. Are we ready to submit an interop test plan at libp2p/test plans repo?

[lla-dane]

@seetadev: Apologize for the late response. @guha-rahul has started reflecting the changes in #620. There were a few issues, I have pointed them out to him, once those are fixed and everything is working correctly, then we can move forward with submitting the test plans.

[seetadev]

@lla-dane : No worries at all, and thanks for the update!

Great to hear that @guha-rahul has started integrating the changes in #620 — that’s a solid step forward. Appreciate you reviewing and pointing out the issues early; catching those now will definitely save time down the line.

Once everything’s stable and working as expected, moving ahead with the test plans sounds like the perfect next step. Feel free to loop me in when you're ready — happy to review or help out as needed.

Thanks again for staying on top of this :)

[guha-rahul]

Hi there everyone, So if we use lla-dane#2 (i personally use gh pr checkout 2) and https://github.com/guha-rahul/rust-libp2p/tree/interop_rsa
To use pylibp2p as a dialer We have to use

python ping.py

We will see an output like

Run this from the same folder in another console:

ping-demo -d /ip4/127.0.0.1/tcp/52818/p2p/QmTdhiZL2UEZdxiVAKztLu4Aju7r26U3WCxw819zfpdY81

Waiting for incoming connection...

now from the interop_rsa branch we need to run (change the peer id)

cargo run -- /ip4/127.0.0.1/tcp/52518/p2p/QmTdhiZL2UEZdxiVAKztLu4Aju7r26U3WCxw819zfpdY81

cc: @seetadev @paschal533 would love to see you guys checking the interop

[paschal533]

Great work @guha-rahul and @lla-dane 🎉👏
For the final test plan, we should use this structure @seetadev recommended for Js-libp2p interop test

interop-py-rust-libp2p/
├── rust_node/
│   └── src   
|        └── main.rs  # Rust ping
├── py_node/
│   └── ping.py      # Python ping
├── scripts/
│   └── run_test.sh         # Launch both nodes and validate
├── README.md               # This file