Skip to content

Conversation

yashksaini-coder
Copy link
Contributor

What was wrong?

There was no integration for thin waist address validation.

Issue #804

How was it fixed?

Implements Phase 1 & partial Phase 2 of issue #804 by introducing Thin Waist Address Validation utilities and updating the echo example to use them, Using the multiaddr.

  1. New utility module: libp2p/utils/address_validation.py
    • get_available_interfaces
    • get_optimal_binding_address
    • expand_wildcard_address
    • Safe fallbacks when Thin Waist functions are not present in multiaddr.utils
  2. Re-exported utilities in libp2p/utils/__init__.py
  3. Updated examples/echo/echo.py:
    • Replaced hardcoded /ip4/0.0.0.0/tcp/{port} with get_optimal_binding_address
    • Added interface discovery logging
  4. Added advanced demo: examples/advanced/network_discovery.py
  5. Added tests:
    • tests/utils/test_address_validation.py
    • tests/examples/test_echo_thin_waist.py (light integration)

Why

  • Enables dynamic discovery of bindable interfaces (IPv4 & IPv6 where available)
  • Provides a clean abstraction for examples to avoid hardcoding wildcard addresses
  • Demonstrates best practices for future example migrations
  • Maintains backward compatibility (graceful fallbacks if Thin Waist APIs not installed)

Backward Compatibility

  • If multiaddr.utils.get_thin_waist_addresses / get_network_addrs are unavailable, utilities fallback to conservative defaults.
  • Existing behavior (binding on wildcard / loopback) still achievable.

To-Do

  • Utilities implemented with safe fallbacks
  • Echo example migrated
  • Tests added & passing locally
  • No breaking public API changes
  • Clean up commit history
  • Add or update documentation related to these changes
  • Add entry to the release notes

yashksaini-coder added 3 commits August 9, 2025 01:22
…tilities

- Added `network_discover.py` to demonstrate Thin Waist address handling.
- Introduced `address_validation.py` with functions for discovering available network interfaces, expanding wildcard addresses, and determining optimal binding addresses.
- Included fallback mechanisms for environments lacking Thin Waist support.
- Replaced hardcoded listen address with `get_optimal_binding_address` for improved flexibility.
- Imported address validation utilities in `echo.py` and updated `__init__.py` to include new functions.
- Introduced `test_echo_thin_waist.py` to validate the echo example's output for Thin Waist lines.
- Added `test_address_validation.py` to cover functions for available interfaces, optimal binding addresses, and wildcard address expansion.
- Included parameterized tests and environment checks for IPv6 support.
@acul71
Copy link
Contributor

acul71 commented Aug 9, 2025

Hi @yashksaini-coder, ping me if you need help.
you can run make PR and make docs to check everything is ok

@yashksaini-coder
Copy link
Contributor Author

Hi @yashksaini-coder, ping me if you need help. you can run make PR and make docs to check everything is ok

Right now, I've created a draft PR to introduce the multiadd thin waist address. with integration to examples and tests, could you give me feedback on this approach. I've only done Phase 1 + 2 (partially) based on the issue #804

@acul71
Copy link
Contributor

acul71 commented Aug 10, 2025

@yashksaini-coder

Can you modify echo.py so that it "mimic" javascript echo, exposing all available interfaces?

  • Current echo.py
(venv) luca@r17:~/PNL_Launchpad_Curriculum/Libp2p/tmp/yashksaini-coder/py-libp2p$ python examples/echo/echo.py
I am 16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
Run this from the same folder in another console:

echo-demo -d /ip4/192.168.136.148/tcp/33387/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk

Waiting for incoming connections...
  • javascript echo
luca@r17:~/PNL_Launchpad_Curriculum/Libp2p/js-libp2p-examples/examples/js-libp2p-example-chat$ node src/listener.js 
Listener ready, listening on:
/ip4/127.0.0.1/tcp/10333/p2p/12D3KooWCS6QSMEbtt77pWeB618W824aGVxhLhPEPaCUMGMvBhuk
/ip4/192.168.136.148/tcp/10333/p2p/12D3KooWCS6QSMEbtt77pWeB618W824aGVxhLhPEPaCUMGMvBhuk
/ip4/10.149.6.41/tcp/10333/p2p/12D3KooWCS6QSMEbtt77pWeB618W824aGVxhLhPEPaCUMGMvBhuk
  • Desired python echo
echo.py
I am 16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
Listener ready, listening on:
/ip4/127.0.0.1/tcp/10333/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
/ip4/192.168.136.148/tcp/10333/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
/ip4/10.149.6.41/tcp/10333/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk

Run this from the same folder in another console:

echo-demo -d /ip4/192.168.136.148/tcp/33387/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk

Waiting for incoming connections...

@yashksaini-coder
Copy link
Contributor Author

@yashksaini-coder

Can you modify echo.py so that it "mimic" javascript echo, exposing all available interfaces?

  • Current echo.py
(venv) luca@r17:~/PNL_Launchpad_Curriculum/Libp2p/tmp/yashksaini-coder/py-libp2p$ python examples/echo/echo.py
I am 16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
Run this from the same folder in another console:

echo-demo -d /ip4/192.168.136.148/tcp/33387/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk

Waiting for incoming connections...
  • javascript echo
luca@r17:~/PNL_Launchpad_Curriculum/Libp2p/js-libp2p-examples/examples/js-libp2p-example-chat$ node src/listener.js 
Listener ready, listening on:
/ip4/127.0.0.1/tcp/10333/p2p/12D3KooWCS6QSMEbtt77pWeB618W824aGVxhLhPEPaCUMGMvBhuk
/ip4/192.168.136.148/tcp/10333/p2p/12D3KooWCS6QSMEbtt77pWeB618W824aGVxhLhPEPaCUMGMvBhuk
/ip4/10.149.6.41/tcp/10333/p2p/12D3KooWCS6QSMEbtt77pWeB618W824aGVxhLhPEPaCUMGMvBhuk
  • Desired python echo
echo.py
I am 16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
Listener ready, listening on:
/ip4/127.0.0.1/tcp/10333/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
/ip4/192.168.136.148/tcp/10333/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk
/ip4/10.149.6.41/tcp/10333/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk

Run this from the same folder in another console:

echo-demo -d /ip4/192.168.136.148/tcp/33387/p2p/16Uiu2HAmRxqPcNXm9KAUAVKyQkpXF8c8r7S6jdpGK6K963w9Puwk

Waiting for incoming connections...

Got it exposing all available interfaces. @acul71

@acul71
Copy link
Contributor

acul71 commented Aug 10, 2025

@yashksaini-coder
Great work just follow this mods:

Summary: yashksaini-coder's Thin Waist Address Validation PR

🎯 Quick Assessment

Overall: Excellent foundation, needs minor refinements for JavaScript parity.

Status: Phase 1 ✅ Complete, Phase 2 🔄 Partially Complete

What's Working Well

  • Core utilities: Robust, well-tested, proper error handling
  • Backward compatibility: Graceful fallbacks when Thin Waist unavailable
  • Test coverage: Comprehensive unit and integration tests
  • Type safety: Full type annotations throughout

🔧 Key Issues to Fix

1. JavaScript Parity (High Priority)

Problem: Echo shows only 1 address instead of multiple interfaces like JS libp2p.

Current:

/ip4/192.168.136.148/tcp/8000/p2p/16Uiu2...

Target (like JS):

/ip4/127.0.0.1/tcp/8000/p2p/16Uiu2...
/ip4/192.168.136.148/tcp/8000/p2p/16Uiu2...
/ip4/10.149.6.41/tcp/8000/p2p/16Uiu2...

Fix: Use get_available_interfaces() instead of get_optimal_binding_address()

2. Missing Loopback (High Priority)

Problem: No 127.0.0.1 in output.

Fix: Add loopback to get_available_interfaces():

addrs.append(Multiaddr(f"/ip4/127.0.0.1/{protocol}/{port}"))

3. Command Display (Medium Priority)

Problem: echo-demo -d shows 0.0.0.0 instead of concrete address.

Fix: Use first expanded address instead of host.get_addrs()[0]

🚀 Quick Wins

  1. Replace this line in echo.py:

    listen_addr = get_optimal_binding_address(port)

    With:

    listen_addrs = get_available_interfaces(port)
  2. Add loopback to address validation:

    # In get_available_interfaces()
    addrs.append(Multiaddr(f"/ip4/127.0.0.1/{protocol}/{port}"))
  3. Update display logic:

    # Show all addresses
    for addr in listen_addrs:
        print(f"{addr}/p2p/{host.get_id().to_string()}")

📋 Action Items

High Priority

  • Fix JavaScript parity (multiple interface display)
  • Add loopback interface (127.0.0.1)
  • Fix command generation (use concrete addresses)

Medium Priority

  • Improve random port handling
  • Add IPv6 loopback support
  • Enhanced error messages

🏆 Recommendation

Approve with minor revisions. The core implementation is solid - just needs these small tweaks to match JavaScript libp2p behavior.

🧪 Test Status

Unit Tests - All Passing

  • test_get_available_interfaces()
  • test_get_optimal_binding_address()
  • test_expand_wildcard_address_ipv4()
  • test_expand_wildcard_address_port_override()
  • test_expand_wildcard_address_ipv6()

Integration Test - Failing

Problem: test_echo_thin_waist.py expects log messages that don't exist in current echo.py.

Test expects:

"Selected binding address:"
"Available candidate interfaces:"

Current echo.py outputs:

"I am {peer_id}"
"echo-demo -d {address}"
"Waiting for incoming connections..."

Fix: Update integration test to match actual echo.py output format.

💡 Pro Tips

  1. Test with: python examples/echo/echo.py -p 8000
  2. Compare with: JavaScript libp2p output format
  3. Focus on: User experience consistency across implementations
  4. Remember: Loopback is crucial for local development

Bottom Line: Great work! Just need to show all interfaces like JavaScript libp2p does. 🚀

@yashksaini-coder
Copy link
Contributor Author

@acul71 got it this clears a lot, I will focus on the javascript libp2p interface showcase.

@seetadev
Copy link
Contributor

@yashksaini-coder , @acul71 : Thank you for submitting the PR. Appreciate it. Wish if CI/CD issues could be resolved.

@acul71
Copy link
Contributor

acul71 commented Aug 18, 2025

Thank you for submitting the PR. Appreciate it. Wish if CI/CD issues could be resolved.

@yashksaini-coder @seetadev
Hi @yashksaini-coder keep working on those mods we were talking, about the issues are resolved here:

Thin Waist Feature Fixes

Summary

Fixed linting, type checking, and test issues in the Thin Waist address validation feature.

Repository & Branch

Fixes Applied

1. Linting Issues (E501 - Line too long)

  • Fixed long lines in examples/advanced/network_discover.py
  • Fixed long lines in libp2p/utils/address_validation.py
  • Fixed long lines in tests/examples/test_echo_thin_waist.py

2. Type Checking Issues

  • Fixed import path: from multiaddr.protocols import P_IP4, P_IP6, P_TCP, P_P2P
  • Added proper type hints for subprocess handling
  • Removed unused variables

3. Test Improvements

  • Fixed hanging integration test by using unbuffered Python output
  • Improved multiaddr validation using py-multiaddr protocol constants
  • Enhanced test assertions with proper protocol validation

Instructions for yashksaini-coder

Option 1: Cherry-pick from fork

git remote add acul71 https://github.com/acul71/py-libp2p-fork.git
git fetch acul71
git cherry-pick acul71/feat/804-add-thin-waist-address

Option 2: Pull specific commits

git remote add acul71 https://github.com/acul71/py-libp2p-fork.git
git fetch acul71
git merge acul71/feat/804-add-thin-waist-address

Verification

All tests pass:

  • make fix (linting)
  • make typecheck (type checking)
  • make pr (full test suite including new Thin Waist test)

Files Modified

  • examples/advanced/network_discover.py
  • examples/echo/echo.py
  • libp2p/utils/address_validation.py
  • tests/examples/test_echo_thin_waist.py
  • tests/utils/test_address_validation.py

@yashksaini-coder
Copy link
Contributor Author

Thank you for submitting the PR. Appreciate it. Wish if CI/CD issues could be resolved.

@yashksaini-coder @seetadev Hi @yashksaini-coder keep working on those mods we were talking, about the issues are resolved here:

Thin Waist Feature Fixes

Summary

Fixed linting, type checking, and test issues in the Thin Waist address validation feature.

Repository & Branch

Fixes Applied

1. Linting Issues (E501 - Line too long)

  • Fixed long lines in examples/advanced/network_discover.py
  • Fixed long lines in libp2p/utils/address_validation.py
  • Fixed long lines in tests/examples/test_echo_thin_waist.py

2. Type Checking Issues

  • Fixed import path: from multiaddr.protocols import P_IP4, P_IP6, P_TCP, P_P2P
  • Added proper type hints for subprocess handling
  • Removed unused variables

3. Test Improvements

  • Fixed hanging integration test by using unbuffered Python output
  • Improved multiaddr validation using py-multiaddr protocol constants
  • Enhanced test assertions with proper protocol validation

Instructions for yashksaini-coder

Option 1: Cherry-pick from fork

git remote add acul71 https://github.com/acul71/py-libp2p-fork.git
git fetch acul71
git cherry-pick acul71/feat/804-add-thin-waist-address

Option 2: Pull specific commits

git remote add acul71 https://github.com/acul71/py-libp2p-fork.git
git fetch acul71
git merge acul71/feat/804-add-thin-waist-address

Verification

All tests pass:

  • make fix (linting)
  • make typecheck (type checking)
  • make pr (full test suite including new Thin Waist test)

Files Modified

  • examples/advanced/network_discover.py
  • examples/echo/echo.py
  • libp2p/utils/address_validation.py
  • tests/examples/test_echo_thin_waist.py
  • tests/utils/test_address_validation.py

Yes thanks for the engangement, I have done the previous requests, of modifying the echo.py for javascript IP address info, and also for IPV6 addresses too. Also thanks for giving the repo with seperate branch.

@yashksaini-coder
Copy link
Contributor Author

@seetadev can you please approve the workflow runs, I picked some commit changes of linting issues, and merge them in this PR, needs to check the workflow

@yashksaini-coder yashksaini-coder marked this pull request as ready for review August 19, 2025 15:20
@acul71
Copy link
Contributor

acul71 commented Aug 20, 2025

Hi @yashksaini-coder
You can verify the workflow locally before submitting doing:
make pr and make docs # or make linux-docs

I can see that you have a lint error just split the long comment in more lines

ruff (legacy alias)......................................................Failed
- hook id: ruff
- exit code: 1

tests/examples/test_echo_thin_waist.py:14:89: E501 Line too long (93 > 88)
   |
13 | # This test is intentionally lightweight and can be marked as 'integration'.
14 | # It ensures the echo example runs and prints the new Thin Waist lines using Trio primitives.
   |                                                                                         ^^^^^ E501
15 |
16 | current_file = Path(__file__)
   |

Found 1 error.

runned echo-demo,
but I can't see all available addrs.

(venv) luca@r17:~/PNL_Launchpad_Curriculum/Libp2p/py-libp2p$ echo-demo 
I am 16Uiu2HAmSt1ybASKw5UJzmGTbjjz9MiRNsYVVdxsEbxTfhBiY8Vp
Listener ready, listening on:
/ip4/192.168.1.75/tcp/0/p2p/16Uiu2HAmSt1ybASKw5UJzmGTbjjz9MiRNsYVVdxsEbxTfhBiY8Vp

Run this from the same folder in another console:

echo-demo -d /ip4/192.168.1.75/tcp/44969/p2p/16Uiu2HAmSt1ybASKw5UJzmGTbjjz9MiRNsYVVdxsEbxTfhBiY8Vp

Waiting for incoming connections...

@acul71
Copy link
Contributor

acul71 commented Aug 24, 2025

At present, IPv6 is not supported in py-libp2p

While the Thin Waist branch has made significant progress in IPv6 support, approximately 40 files still contain hardcoded IPv4 references that limit full IPv6 functionality. The most critical issues have been addressed in the transport layer, but additional work is needed in:

  1. Identity/Identify module for proper IPv6 address handling
  2. mDNS discovery for IPv6 address conversion
  3. Examples and tests for comprehensive IPv6 coverage

@acul71
Copy link
Contributor

acul71 commented Aug 24, 2025

@yashksaini-coder @sumanjeet0012
I thought to filter out (comment out) ipv6 from get_available_interface in libp2p/utils/address_validation.py and added a TODO to re-enable it when ipv6 support will be added to py-libp2p. Opening an issue soon.
Just cherry-pick
acul71@7a1198c
to get the fix.
Doing so this PR would be after a review ready to merge, and we can fix the ipv6 issue in py-libp2p

@acul71
Copy link
Contributor

acul71 commented Aug 24, 2025

Hi @yashksaini-coder, you should add a doc newsfragment file to complete this PR.
Something like this:


File to create: newsfragments/811.feature.rst

Add Thin Waist address validation utilities and integrate into echo example

- Add ``libp2p/utils/address_validation.py`` with dynamic interface discovery
- Implement ``get_available_interfaces()``, ``get_optimal_binding_address()``, and ``expand_wildcard_address()``
- Update echo example to use dynamic address discovery instead of hardcoded wildcard
- Add safe fallbacks for environments lacking Thin Waist support
- Temporarily disable IPv6 support due to libp2p handshake issues (TODO: re-enable when resolved)

✅ Status

  • API documentation - Already exists in libp2p/utils/__init__.py
  • Example demo - Already exists in examples/advanced/network_discover.py
  • Echo integration - Already updated to use Thin Waist utilities
  • Tests - Already exist and passing
  • Newsfragment - MISSING (required for release notes)

@yashksaini-coder
Copy link
Contributor Author

@acul71 @seetadev I've add newsfragment, now this PR is ready to merge,

Copy link
Contributor

@acul71 acul71 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you can refactor then for me is ready to be merged

Copy link
Contributor

@acul71 acul71 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done

@acul71
Copy link
Contributor

acul71 commented Aug 24, 2025

@seetadev @pacrob
This is ready to be merged.
The Phase 4 will be implemented in another PR.

PR #811 Review Summary

Status: ✅ APPROVED

PR: #811 - Feat: add Thin Waist address validation utilities and integrate into echo example


🎯 Key Achievements


🔧 Core Changes

New Module: libp2p/utils/address_validation.py

  • get_available_interfaces() - Dynamic interface discovery
  • get_optimal_binding_address() - Smart address selection
  • expand_wildcard_address() - Wildcard expansion
  • find_free_port() - Centralized port finding utility
  • Safe fallbacks for missing Thin Waist support

Enhanced Echo Example

  • Before: Hardcoded /ip4/0.0.0.0/tcp/{port}
  • After: Dynamic get_available_interfaces(port)
  • Shows all available interfaces (JS parity)
  • Note: Only 1 of 15 examples updated (Phase 4 needed for complete migration)

IPv6 Handling

  • Smart decision: Temporarily disabled IPv6 due to libp2p handshake issues
  • Clear TODO comments with future work plan
  • Prevents crashes while maintaining IPv4 functionality

✅ Strengths

  1. Robust Implementation - Comprehensive error handling
  2. Backward Compatibility - Graceful degradation
  3. Clean Code - Proper type hints, documentation
  4. Centralized Utilities - No code duplication
  5. Pragmatic Approach - Ships working feature while planning IPv6 support

🚀 Ready for Merge

  • All quality checks pass
  • No breaking changes
  • Demonstrates Thin Waist best practices
  • Provides foundation for future IPv6 support
  • Newsfragment added for release notes

Verdict: APPROVE - Excellent work! 🎉

@yashksaini-coder yashksaini-coder force-pushed the feat/804-add-thin-waist-address branch from aaffe2c to 4195ee1 Compare August 24, 2025 19:46
@yashksaini-coder yashksaini-coder force-pushed the feat/804-add-thin-waist-address branch from 4195ee1 to 6a0a7c2 Compare August 24, 2025 20:01
@seetadev
Copy link
Contributor

@acul71 : Great to hear. I just had a long discussion with @yashksaini-coder. Wonderful progress and great work.

Special thanks to you and @sumanjeet0012 for the great support and help.

Will wait for a thumbs up from @pacrob and will then merge today.

Copy link
Member

@pacrob pacrob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment that can be removed, otherwise looks good! Thanks, @yashksaini-coder!

@yashksaini-coder
Copy link
Contributor Author

Just one comment that can be removed, otherwise looks good! Thanks, @yashksaini-coder!

Done, @pacrob @seetadev

@seetadev
Copy link
Contributor

@pacrob : Thank you so much for your feedback. Appreciate it.

@yashksaini-coder : Re-running CI/CD pipeline and doing a final review. Will merge the PR soon.

Kindly share a detailed blog on how you did this PR. Since this was your first PR in py-libp2p, which is getting merged, we would like you to share your experiences from a technical perspective in detail. This would also help the new contributors to take up projects in py-libp2p and arrive at a good conclusion on them.

Keep up the good work :)

@seetadev seetadev merged commit 292bd1a into libp2p:main Aug 25, 2025
1 check was pending
@yashksaini-coder
Copy link
Contributor Author

@seetadev I already shared the details in the of developer docs, in the discussion #807 But sure I will share a blog too, on dev.to

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants