Add WebSocket transport support #781

GautamBytes · 2025-07-20T09:42:36Z

Closes #585

This pull request introduces WebSocket transport support to py-libp2p, enabling interoperability with other major libp2p implementations like js-libp2p and go-libp2p that rely on /ws multiaddrs.

The implementation adds a new libp2p.transport.websocket module containing the necessary components to dial and listen on WebSocket connections, integrating seamlessly with the existing TransportUpgrader and Swarm.

Key Features

WebsocketTransport and WebsocketListener: Implements the core ITransport and IListener interfaces for handling the WebSocket connection lifecycle.
/ws Multiaddr Handling: Correctly parses /ip4/.../tcp/.../ws multiaddrs for both dialing and listening. The listener also advertises the correct public /ws address.
trio-websocket Integration: Uses trio-websocket for native, non-blocking I/O within the py-libp2p trio runtime.
Interop Testing: Includes a new integration test (test_js_ws_ping.py) that successfully establishes a connection and runs the /ipfs/ping/1.0.0 protocol with a live js-libp2p node, confirming bi-directional communication.
Upgrader Compatibility: The new transport works with the existing TransportUpgrader, allowing security protocols (e.g., noise) and stream multiplexers (e.g., mplex) to be layered on top.

Scope

This PR implements the plaintext WebSocket (/ws) transport.
Support for secure WebSockets (/wss) is deferred for future work as discussed in the issue. The transport will raise a NotImplementedError if a /wss address is provided.

Signed-off-by: GautamBytes <[email protected]>

GautamBytes · 2025-07-20T09:43:41Z

cc @seetadev

seetadev · 2025-07-20T11:14:43Z

@GautamBytes : Thanks Gautam for submitting the PR. Appreciate your initiative and efforts. Please resolve the CI/CD issues.

CCing @sumanjeet0012, @guha-rahul and @lla-dane for their feedback and pointers too. This module will be needed by a number of projects. Lets productionize it well.

Signed-off-by: GautamBytes <[email protected]>

guha-rahul · 2025-07-20T19:54:11Z

libp2p/transport/websocket/transport.py

+            raise ValueError(f"No host protocol found in {maddr}")
+
+        port = int(maddr.value_for_protocol("tcp"))
+        uri = f"ws://{host}:{port}"


are we leaving support for ipv6 for later on?

Thanks for the catch!
I'll update the dial method to properly enclose IPv6 hosts in brackets.

…libp2p into add-ws-transport

GautamBytes · 2025-07-23T16:24:17Z

@seetadev , @guha-rahul , i am trying to fix this failing workflows but facing difficulty . Its because of my test_js_ws_ping.py file not passing , error logs shows -

(venv) gautam@Gautam:/mnt/e/PY-LIBP2P/py-libp2p$ pytest -s tests/interop/test_js_ws_ping.py
======================================================================= test session starts =======================================================================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /mnt/e/PY-LIBP2P/py-libp2p/venv/bin/python3
cachedir: .pytest_cache
rootdir: /mnt/e/PY-LIBP2P/py-libp2p
configfile: pyproject.toml
plugins: anyio-1.4.0, Faker-37.4.0, trio-0.8.0, xdist-3.7.0
collected 1 item                                                                                                                                                  

tests/interop/test_js_ws_ping.py::test_ping_with_js_node FAILED

============================================================================ FAILURES =============================================================================
_____________________________________________________________________ test_ping_with_js_node ______________________________________________________________________
  + Exception Group Traceback (most recent call last):
  |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/trio/_core/_run.py", line 1070, in __aexit__
  |     raise combined_error_from_nursery
  | BaseExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
  +-+---------------- 1 ----------------
    | libp2p.exceptions.MultiError: Error 1: fail to open connection to peer 12D3KooWS771iWN9eXL7hSrpyS9WQos5wRXJjeiXz2HMDD3rfs4g
    | 
    | The above exception was the direct cause of the following exception:
    | 
    | Traceback (most recent call last):
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/tests/interop/test_js_ws_ping.py", line 99, in test_ping_with_js_node
    |     await host.connect(peer_info)
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/libp2p/host/basic_host.py", line 267, in connect
    |     await self._network.dial_peer(peer_info.peer_id)
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/libp2p/network/swarm.py", line 161, in dial_peer
    |     raise SwarmException(
    | libp2p.network.exceptions.SwarmException: unable to connect to 12D3KooWS771iWN9eXL7hSrpyS9WQos5wRXJjeiXz2HMDD3rfs4g, no addresses established a successful connection (with exceptions)
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/pytest_trio/plugin.py", line 195, in _fixture_manager
    |     yield nursery_fixture
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/pytest_trio/plugin.py", line 250, in run
    |     await self._func(**resolved_kwargs)
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/tests/interop/test_js_ws_ping.py", line 102, in test_ping_with_js_node
    |     pytest.fail(
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/_pytest/outcomes.py", line 177, in fail
    |     raise Failed(msg=reason, pytrace=pytrace)
    | Failed: Connection failed with SwarmException.
    | THE REAL ERROR IS: MultiError([SwarmException('fail to open connection to peer 12D3KooWS771iWN9eXL7hSrpyS9WQos5wRXJjeiXz2HMDD3rfs4g')])
    | 
    +------------------------------------
======================================================================== warnings summary =========================================================================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.ScalarMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================== slowest 50 durations =======================================================================
9.74s call     tests/interop/test_js_ws_ping.py::test_ping_with_js_node
4.15s setup    tests/interop/test_js_ws_ping.py::test_ping_with_js_node

(1 durations < 0.005s hidden.  Use -vv to show these durations.)
===================================================================== short test summary info =====================================================================
FAILED tests/interop/test_js_ws_ping.py::test_ping_with_js_node - Connection failed with SwarmException.

would really appreciate if someone can tell how to fix this!

seetadev · 2025-07-24T04:36:52Z

@seetadev , @guha-rahul , i am trying to fix this failing workflows but facing difficulty . Its because of my test_js_ws_ping.py file not passing , error logs shows -

(venv) gautam@Gautam:/mnt/e/PY-LIBP2P/py-libp2p$ pytest -s tests/interop/test_js_ws_ping.py
======================================================================= test session starts =======================================================================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /mnt/e/PY-LIBP2P/py-libp2p/venv/bin/python3
cachedir: .pytest_cache
rootdir: /mnt/e/PY-LIBP2P/py-libp2p
configfile: pyproject.toml
plugins: anyio-1.4.0, Faker-37.4.0, trio-0.8.0, xdist-3.7.0
collected 1 item                                                                                                                                                  

tests/interop/test_js_ws_ping.py::test_ping_with_js_node FAILED

============================================================================ FAILURES =============================================================================
_____________________________________________________________________ test_ping_with_js_node ______________________________________________________________________
  + Exception Group Traceback (most recent call last):
  |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/trio/_core/_run.py", line 1070, in __aexit__
  |     raise combined_error_from_nursery
  | BaseExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
  +-+---------------- 1 ----------------
    | libp2p.exceptions.MultiError: Error 1: fail to open connection to peer 12D3KooWS771iWN9eXL7hSrpyS9WQos5wRXJjeiXz2HMDD3rfs4g
    | 
    | The above exception was the direct cause of the following exception:
    | 
    | Traceback (most recent call last):
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/tests/interop/test_js_ws_ping.py", line 99, in test_ping_with_js_node
    |     await host.connect(peer_info)
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/libp2p/host/basic_host.py", line 267, in connect
    |     await self._network.dial_peer(peer_info.peer_id)
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/libp2p/network/swarm.py", line 161, in dial_peer
    |     raise SwarmException(
    | libp2p.network.exceptions.SwarmException: unable to connect to 12D3KooWS771iWN9eXL7hSrpyS9WQos5wRXJjeiXz2HMDD3rfs4g, no addresses established a successful connection (with exceptions)
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/pytest_trio/plugin.py", line 195, in _fixture_manager
    |     yield nursery_fixture
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/pytest_trio/plugin.py", line 250, in run
    |     await self._func(**resolved_kwargs)
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/tests/interop/test_js_ws_ping.py", line 102, in test_ping_with_js_node
    |     pytest.fail(
    |   File "/mnt/e/PY-LIBP2P/py-libp2p/venv/lib/python3.12/site-packages/_pytest/outcomes.py", line 177, in fail
    |     raise Failed(msg=reason, pytrace=pytrace)
    | Failed: Connection failed with SwarmException.
    | THE REAL ERROR IS: MultiError([SwarmException('fail to open connection to peer 12D3KooWS771iWN9eXL7hSrpyS9WQos5wRXJjeiXz2HMDD3rfs4g')])
    | 
    +------------------------------------
======================================================================== warnings summary =========================================================================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.ScalarMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================== slowest 50 durations =======================================================================
9.74s call     tests/interop/test_js_ws_ping.py::test_ping_with_js_node
4.15s setup    tests/interop/test_js_ws_ping.py::test_ping_with_js_node

(1 durations < 0.005s hidden.  Use -vv to show these durations.)
===================================================================== short test summary info =====================================================================
FAILED tests/interop/test_js_ws_ping.py::test_ping_with_js_node - Connection failed with SwarmException.

would really appreciate if someone can tell how to fix this!

@GautamBytes : Thank you so much for taking the initiative to work on fixing the failing workflows. Your effort on test_js_ws_ping.py is really appreciated. The SwarmException error you’re seeing usually means that the WebSocket connection between the Python host and the JS peer isn’t being established — this could be due to a mismatch in multiaddr protocols, peer discovery issues, or the JS node not being ready when the test starts.

@acul71, @guha-rahul , @varun-r-mallya — can you jump in to help debug this WebSocket issue with ? Maybe we can review the interop/test_js_ws_ping.py setup together and check:

If the JS node is properly launched and listening on the expected /ws multiaddr.

Whether dial_peer is using the correct transport and address.

If any recent changes in the libp2p handshake or transport modules might have caused regressions.

@GautamBytes, you might also try:

Running the JS ping test manually with verbose logs (DEBUG=libp2p* pytest -s tests/interop/test_js_ws_ping.py) to see detailed handshake failures.

Verifying that the JS node and Python host share the same set of supported protocols and transports.

Testing if basic_host.connect(peer_info) works when you manually start a JS libp2p node with WebSocket enabled.

Let’s collaborate and get this test passing soon.

acul71 · 2025-08-07T14:09:56Z

@GautamBytes @seetadev
I'm working on WebSocket transport.
I've added some additional tests, I've seen that I can do an echo test with PLAINTEXT_PROTOCOL_ID
But it fails with Noise

Work in progress.......

…Add transport_registry.py for centralized transport management - Integrate WebSocket transport with new registry - Add comprehensive test suite for transport registry - Include WebSocket examples and demos - Update transport initialization and swarm integration

acul71 · 2025-08-09T23:57:01Z

@GautamBytes can you give me access to push to your fork ?

seetadev · 2025-08-10T05:55:30Z

@GautamBytes : Hi Gautam. @acul71 is our amazing Py-libp2p developer, key maintainer to py-multiaddr, py-cid and core py-libp2p modules. Kindly give him push access so that he can collaborate with you on taking web socket to a production stage. Appreciate your efforts and initiative.

seetadev · 2025-08-10T05:57:16Z

@GautamBytes @seetadev I'm working on WebSocket transport. I've added some additional tests, I've seen that I can do an echo test with PLAINTEXT_PROTOCOL_ID But it fails with Noise

Work in progress.......

@acul71 : Great, thanks Luca. Appreciate your initiative and support.

I have asked Gautam to share access with you soon. He might be unavailable because of his engineering exams this week. Will be available soon in a day or two.

GautamBytes · 2025-08-10T06:34:37Z

@acul71 , i have invited u as collaborator , can u check the mail and confirm once?

seetadev · 2025-08-10T11:25:33Z

@acul71 : Can you confirm with @GautamBytes ? He did share an invite today.

- Fix INotifee interface compliance in WebSocket demo - Fix handler function signatures to be async (THandler compatibility) - Fix is_closed method usage with proper type checking - Fix pytest.raises multiple exception type issue - Fix line length violations (E501) across multiple files - Add debugging logging to Noise security module for troubleshooting - Update WebSocket transport examples and tests - Improve transport registry error handling

…ansport

acul71 · 2025-08-10T23:53:18Z

@seetadev @GautamBytes

Python WebSocket Implementation — Status & Discussion

Discussion Links:

Main WebSocket discussion: libp2p/py-libp2p#823
Transport registry discussion: libp2p/py-libp2p#824

📌 Overview

Our current WebSocket implementation provides a robust foundation with comprehensive support for /ws protocol, DNS addressing, IPv6, and proper libp2p integration.
The implementation is production-ready for non-TLS scenarios and demonstrates solid engineering practices.

✅ Key Strengths

Full /ws protocol implementation
DNS addressing support (IPv4, IPv6, DNS4, DNS6)
Proper libp2p integration
Noise encryption support
Working examples and tests
Comprehensive multiaddr validation

❌ Primary Gap

TLS support (/wss, /tls/ws) — Main missing feature

📂 Recent Changes

Added tests and a working example
Added debug logging (in Noise module — too much debug logging for review, but kept for now)
Implemented a transport registry to manage different transport modules

🐛 Known Issues

1. Failing Test

FAILED tests/interop/test_js_ws_ping.py::test_ping_with_js_node - Connection failed with SwarmException.

This is not a proper pytest test.
Options:
- Convert to a valid pytest test
- Move interop-tests to another directory outside tests/
.gitignore is currently Python-oriented, not suited for JavaScript files.

2. Missing `wss` Support

Can be implemented using python ssl

acul71 · 2025-08-11T17:14:53Z

@seetadev @GautamBytes
interop is expected to fail cause it is not a proper test. (Just a utility to check interop)
We have to think of another strategy for interop tests
wss is required to be specs compliant

@GautamBytes what do you think of last changes see #781 (comment)

Proposal: Implement missing wss protocol, for now, than we see what to do about interop testing structure (but we can use interop testing in our local devenvironment "at home" to see that it working )

seetadev · 2025-08-12T19:52:39Z

@acul71 : Thanks for the clarification — that makes perfect sense regarding the current interop utility not being a proper test. I agree that while it’s been useful for quick verification, a more structured interop testing approach will be important going forward, especially if we want repeatable results and alignment with spec requirements.

I’m aligned with your and Gautam's proposal to implement the missing WSS protocol first so we’re fully specs-compliant. This will give us a solid baseline for any future interoperability efforts. Once WSS is in place, we can revisit the interop testing framework and decide how to best structure it for automation while still keeping the ability to run quick “at home” checks for early feedback.

I also reviewed your latest changes in the PR— the direction looks good, and I appreciate how you’re keeping both compliance and developer workflow in mind. Let’s proceed with WSS first, then use the local interop runs to validate, and afterward we can design the formal interop testing structure without blocking progress on the transport work.

Excited to see this come together — with WSS and structured interop testing in place, py-libp2p’s transport story will be much stronger.

acul71 · 2025-08-12T22:00:52Z

I’m aligned with your and Gautam's proposal to implement the missing WSS

Waiting for @GautamBytes feedback, I can help complete this PR

GautamBytes added 2 commits July 20, 2025 09:23

added WebSocket transport support

1874183

Signed-off-by: GautamBytes <[email protected]>

small tweak

227a5c6

Signed-off-by: GautamBytes <[email protected]>

Merge branch 'main' into add-ws-transport

fa0b64d

Prevent crash in JS interop test

4fb7132

Signed-off-by: GautamBytes <[email protected]>

guha-rahul reviewed Jul 20, 2025

View reviewed changes

seetadev and others added 5 commits July 21, 2025 06:44

Merge branch 'main' into add-ws-transport

b6c3637

Merge branch 'main' into add-ws-transport

7469238

Merge branch 'main' into add-ws-transport

8a21435

Fix IPv6 host bracketing in WebSocket transport

1997777

Merge branch 'add-ws-transport' of https://github.com/GautamBytes/py-…

53a16d0

…libp2p into add-ws-transport

GautamBytes requested a review from guha-rahul July 23, 2025 08:27

seetadev added 3 commits July 26, 2025 01:56

Merge branch 'main' into add-ws-transport

65faa21

Merge branch 'main' into add-ws-transport

7a1aa54

Merge branch 'main' into add-ws-transport

651bf0f

acul71 added 2 commits August 9, 2025 20:18

Merge upstream/main into add-ws-transport

a6f8569

Merge branch 'main' into add-ws-transport

19c1f5e

Merge branch 'main' into add-ws-transport

167dfdc

acul71 added 2 commits August 11, 2025 01:25

Merge remote-tracking branch 'origin/add-ws-transport' into add-ws-tr…

a5b0db1

…ansport

seetadev added 2 commits August 12, 2025 00:24

Merge branch 'main' into add-ws-transport

9573ab5

Merge branch 'main' into add-ws-transport

f1872bb

Add WebSocket transport support #781

Are you sure you want to change the base?

Add WebSocket transport support #781

Uh oh!

Conversation

GautamBytes commented Jul 20, 2025

Key Features

Scope

Uh oh!

GautamBytes commented Jul 20, 2025

Uh oh!

seetadev commented Jul 20, 2025

Uh oh!

guha-rahul Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

GautamBytes Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

GautamBytes commented Jul 23, 2025

Uh oh!

seetadev commented Jul 24, 2025

Uh oh!

acul71 commented Aug 7, 2025

Uh oh!

acul71 commented Aug 9, 2025

Uh oh!

seetadev commented Aug 10, 2025

Uh oh!

seetadev commented Aug 10, 2025

Uh oh!

GautamBytes commented Aug 10, 2025

Uh oh!

seetadev commented Aug 10, 2025

Uh oh!

acul71 commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Python WebSocket Implementation — Status & Discussion

📌 Overview

✅ Key Strengths

❌ Primary Gap

📂 Recent Changes

🐛 Known Issues

1. Failing Test

2. Missing wss Support

Uh oh!

acul71 commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seetadev commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

acul71 commented Aug 12, 2025

Uh oh!

Uh oh!

acul71 commented Aug 10, 2025 •

edited

Loading

2. Missing `wss` Support

acul71 commented Aug 11, 2025 •

edited

Loading

seetadev commented Aug 12, 2025 •

edited

Loading