Skip to content

Commit 94d695c

Browse files
sumanjeet0012seetadevpacrob
authored
feat: Implement Random walk in py-libp2p (#822)
* Implementing random walk in py libp2p * Add documentation for Random Walk module implementation in py-libp2p * Add Random Walk example for py-libp2p Kademlia DHT * refactor: peer eviction from routing table stopped * refactored location of random walk * add nodesin routing table from peerstore * random walk working as expected * removed extra functions * Removed all manual triggers * added newsfragments * fix linting issues * refacored logs and cleaned example file * refactor: update RandomWalk and RTRefreshManager to use query function for peer discovery * docs: added Random Walk example docs * added optional argument to use random walk in kademlia DHT * enabled random walk in example file * Added tests for RandomWalk module * fixed lint issues * Update refresh interval and some more tests are added. * Removed Random Walk module documentation file * Extra parentheses have been removed from the random walk logs. Co-authored-by: Paul Robinson <[email protected]> --------- Co-authored-by: Manu Sheel Gupta <[email protected]> Co-authored-by: Paul Robinson <[email protected]>
1 parent dabb3a0 commit 94d695c

File tree

16 files changed

+1516
-3
lines changed

16 files changed

+1516
-3
lines changed

‎docs/examples.random_walk.rst

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
Random Walk Example
2+
===================
3+
4+
This example demonstrates the Random Walk module's peer discovery capabilities using real libp2p hosts and Kademlia DHT.
5+
It shows how the Random Walk module automatically discovers new peers and maintains routing table health.
6+
7+
The Random Walk implementation performs the following key operations:
8+
9+
* **Automatic Peer Discovery**: Generates random peer IDs and queries the DHT network to discover new peers
10+
* **Routing Table Maintenance**: Periodically refreshes the routing table to maintain network connectivity
11+
* **Connection Management**: Maintains optimal connections to healthy peers in the network
12+
* **Real-time Statistics**: Displays routing table size, connected peers, and peerstore statistics
13+
14+
.. code-block:: console
15+
16+
$ python -m pip install libp2p
17+
Collecting libp2p
18+
...
19+
Successfully installed libp2p-x.x.x
20+
$ cd examples/random_walk
21+
$ python random_walk.py --mode server
22+
2025-08-12 19:51:25,424 - random-walk-example - INFO - === Random Walk Example for py-libp2p ===
23+
2025-08-12 19:51:25,424 - random-walk-example - INFO - Mode: server, Port: 0 Demo interval: 30s
24+
2025-08-12 19:51:25,426 - random-walk-example - INFO - Starting server node on port 45123
25+
2025-08-12 19:51:25,426 - random-walk-example - INFO - Node peer ID: 16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef
26+
2025-08-12 19:51:25,426 - random-walk-example - INFO - Node address: /ip4/0.0.0.0/tcp/45123/p2p/16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef
27+
2025-08-12 19:51:25,427 - random-walk-example - INFO - Initial routing table size: 0
28+
2025-08-12 19:51:25,427 - random-walk-example - INFO - DHT service started in SERVER mode
29+
2025-08-12 19:51:25,430 - libp2p.discovery.random_walk.rt_refresh_manager - INFO - RT Refresh Manager started
30+
2025-08-12 19:51:55,432 - random-walk-example - INFO - --- Iteration 1 ---
31+
2025-08-12 19:51:55,432 - random-walk-example - INFO - Routing table size: 15
32+
2025-08-12 19:51:55,432 - random-walk-example - INFO - Connected peers: 8
33+
2025-08-12 19:51:55,432 - random-walk-example - INFO - Peerstore size: 42
34+
35+
You can also run the example in client mode:
36+
37+
.. code-block:: console
38+
39+
$ python random_walk.py --mode client
40+
2025-08-12 19:52:15,424 - random-walk-example - INFO - === Random Walk Example for py-libp2p ===
41+
2025-08-12 19:52:15,424 - random-walk-example - INFO - Mode: client, Port: 0 Demo interval: 30s
42+
2025-08-12 19:52:15,426 - random-walk-example - INFO - Starting client node on port 51234
43+
2025-08-12 19:52:15,426 - random-walk-example - INFO - Node peer ID: 16Uiu2HAmAbc123xyz...
44+
2025-08-12 19:52:15,427 - random-walk-example - INFO - DHT service started in CLIENT mode
45+
2025-08-12 19:52:45,432 - random-walk-example - INFO - --- Iteration 1 ---
46+
2025-08-12 19:52:45,432 - random-walk-example - INFO - Routing table size: 8
47+
2025-08-12 19:52:45,432 - random-walk-example - INFO - Connected peers: 5
48+
2025-08-12 19:52:45,432 - random-walk-example - INFO - Peerstore size: 25
49+
50+
Command Line Options
51+
--------------------
52+
53+
The example supports several command-line options:
54+
55+
.. code-block:: console
56+
57+
$ python random_walk.py --help
58+
usage: random_walk.py [-h] [--mode {server,client}] [--port PORT]
59+
[--demo-interval DEMO_INTERVAL] [--verbose]
60+
61+
Random Walk Example for py-libp2p Kademlia DHT
62+
63+
optional arguments:
64+
-h, --help show this help message and exit
65+
--mode {server,client}
66+
Node mode: server (DHT server), or client (DHT client)
67+
--port PORT Port to listen on (0 for random)
68+
--demo-interval DEMO_INTERVAL
69+
Interval between random walk demonstrations in seconds
70+
--verbose Enable verbose logging
71+
72+
Key Features Demonstrated
73+
-------------------------
74+
75+
**Automatic Random Walk Discovery**
76+
The example shows how the Random Walk module automatically:
77+
78+
* Generates random 256-bit peer IDs for discovery queries
79+
* Performs concurrent random walks to maximize peer discovery
80+
* Validates discovered peers and adds them to the routing table
81+
* Maintains routing table health through periodic refreshes
82+
83+
**Real-time Network Statistics**
84+
The example displays live statistics every 30 seconds (configurable):
85+
86+
* **Routing Table Size**: Number of peers in the Kademlia routing table
87+
* **Connected Peers**: Number of actively connected peers
88+
* **Peerstore Size**: Total number of known peers with addresses
89+
90+
**Connection Management**
91+
The example includes sophisticated connection management:
92+
93+
* Automatically maintains connections to healthy peers
94+
* Filters for compatible peers (TCP + IPv4 addresses)
95+
* Reconnects to maintain optimal network connectivity
96+
* Handles connection failures gracefully
97+
98+
**DHT Integration**
99+
Shows seamless integration between Random Walk and Kademlia DHT:
100+
101+
* RT Refresh Manager coordinates with the DHT routing table
102+
* Peer discovery feeds directly into DHT operations
103+
* Both SERVER and CLIENT modes supported
104+
* Bootstrap connectivity to public IPFS nodes
105+
106+
Understanding the Output
107+
------------------------
108+
109+
When you run the example, you'll see periodic statistics that show how the Random Walk module is working:
110+
111+
* **Initial Phase**: Routing table starts empty and quickly discovers peers
112+
* **Growth Phase**: Routing table size increases as more peers are discovered
113+
* **Maintenance Phase**: Routing table size stabilizes as the system maintains optimal peer connections
114+
115+
The Random Walk module runs automatically in the background, performing peer discovery queries every few minutes to ensure the routing table remains populated with fresh, reachable peers.
116+
117+
Configuration
118+
-------------
119+
120+
The Random Walk module can be configured through the following parameters in ``libp2p.discovery.random_walk.config``:
121+
122+
* ``RANDOM_WALK_ENABLED``: Enable/disable automatic random walks (default: True)
123+
* ``REFRESH_INTERVAL``: Time between automatic refreshes in seconds (default: 300)
124+
* ``RANDOM_WALK_CONCURRENCY``: Number of concurrent random walks (default: 3)
125+
* ``MIN_RT_REFRESH_THRESHOLD``: Minimum routing table size before triggering refresh (default: 4)
126+
127+
See Also
128+
--------
129+
130+
* :doc:`examples.kademlia` - Kademlia DHT value storage and content routing
131+
* :doc:`libp2p.discovery.random_walk` - Random Walk module API documentation

‎docs/examples.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,4 @@ Examples
1414
examples.circuit_relay
1515
examples.kademlia
1616
examples.mDNS
17+
examples.random_walk

‎docs/libp2p.discovery.random_walk.rst

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
libp2p.discovery.random_walk package
2+
====================================
3+
4+
The Random Walk module implements a peer discovery mechanism.
5+
It performs random walks through the DHT network to discover new peers and maintain routing table health through periodic refreshes.
6+
7+
Submodules
8+
----------
9+
10+
libp2p.discovery.random_walk.config module
11+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
12+
13+
.. automodule:: libp2p.discovery.random_walk.config
14+
:members:
15+
:undoc-members:
16+
:show-inheritance:
17+
18+
libp2p.discovery.random_walk.exceptions module
19+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
20+
21+
.. automodule:: libp2p.discovery.random_walk.exceptions
22+
:members:
23+
:undoc-members:
24+
:show-inheritance:
25+
26+
libp2p.discovery.random_walk.random_walk module
27+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
28+
29+
.. automodule:: libp2p.discovery.random_walk.random_walk
30+
:members:
31+
:undoc-members:
32+
:show-inheritance:
33+
34+
libp2p.discovery.random_walk.rt_refresh_manager module
35+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
36+
37+
.. automodule:: libp2p.discovery.random_walk.rt_refresh_manager
38+
:members:
39+
:undoc-members:
40+
:show-inheritance:
41+
42+
Module contents
43+
---------------
44+
45+
.. automodule:: libp2p.discovery.random_walk
46+
:members:
47+
:undoc-members:
48+
:show-inheritance:

‎docs/libp2p.discovery.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Subpackages
1010
libp2p.discovery.bootstrap
1111
libp2p.discovery.events
1212
libp2p.discovery.mdns
13+
libp2p.discovery.random_walk
1314

1415
Submodules
1516
----------

‎examples/kademlia/kademlia.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -227,7 +227,7 @@ async def run_node(
227227

228228
# Keep the node running
229229
while True:
230-
logger.debug(
230+
logger.info(
231231
"Status - Connected peers: %d,"
232232
"Peers in store: %d, Values in store: %d",
233233
len(dht.host.get_connected_peers()),

0 commit comments

Comments
 (0)