Skip to content

feat: Implement Random walk in py-libp2p #822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

sumanjeet0012
Copy link
Contributor

@sumanjeet0012 sumanjeet0012 commented Aug 10, 2025

What was wrong?

Fixes #788

Random Walk Module was not present in py-libp2p. The py-libp2p library lacked a Random Walk mechanism for its DHT implementation, which is essential for continuous peer discovery and maintaining a healthy routing table in dynamic peer-to-peer networks.

How was it fixed?

Implemented Random Walk module in py-libp2p with the following key components:

1. Core Module Implementation

  • Created libp2p/discovery/random_walk/ directory with:
    • random_walk.py: Main RandomWalkDiscovery class that performs periodic random peer lookups
    • utils.py: Random peer ID generation and lookup utilities
    • __init__.py: Module exports

2. DHT Integration

  • Added random walk functionality to DHT implementation
  • Modified DHT classes to support background peer discovery
  • Integrated with existing routing table refresh mechanisms

3. Key Features

  • Periodically generates random peer IDs and performs FIND_NODE queries
  • Refreshes routing table by discovering new peers and validating existing ones
  • Configurable options including:
    • Walk interval: How frequently random walks occur
    • Concurrency: Number of walks to run in parallel
    • Timeouts: Maximum time for each lookup
  • Prevents routing table staleness in networks with high churn
  • Improves overall DHT performance and peer discovery success rates
  • Brings py-libp2p closer to feature parity with Go-libp2p and JS-libp2p

4. Background Process Management

  • Implements non-blocking background task execution
  • Graceful shutdown and cleanup mechanisms
  • Error handling for failed lookups and network issues

5. Examples and Tests

  • Created comprehensive example demonstrating random walk functionality
  • Added integration tests to verify continuous peer discovery
  • Performance benchmarks for different configuration options

To-Do

  • Clean up commit history
  • Add or update documentation related to these changes
  • Add entry to the release notes

Cute Animal Picture

put a cute animal picture link inside the parentheses

@seetadev
Copy link
Contributor

@sumanjeet0012 : Great progress on the Random Walk module! 🎉

The approach you’ve taken makes the module clean, extensible, and in line with the existing py-libp2p architecture. I especially appreciate:

Alignment with specs – You’ve kept the implementation true to libp2p’s discovery goals, which will help ensure compatibility across implementations.

Incremental progress – You’re clearly testing and refining as you go, which shows a disciplined approach to development.

The random walk module is going to be a great step forward for py-libp2p’s peer discovery capabilities. Once it’s fully integrated, it will enhance the robustness and adaptability of the network layer — something that will benefit many projects using py-libp2p.

Keep up the momentum! 🚀 Your work here is making py-libp2p stronger and more feature-complete, and it’s exciting to see it coming together.

@sumanjeet0012 sumanjeet0012 marked this pull request as ready for review August 13, 2025 05:45
@sumanjeet0012
Copy link
Contributor Author

@seetadev The Random walk module is now working as expected.
I will shortly add some test cases and a screencast video.

This PR is now ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Random Walk Module for Continuous Peer Discovery in py-libp2p
2 participants