Skip to content

Conversation

AkMo3
Copy link
Collaborator

@AkMo3 AkMo3 commented Jul 13, 2025

What was wrong?

This PR aims to bring QUIC transport support

How was it fixed?

Summary of approach:

What's Working

  • Connection Establishment, Data sharing and cleanup
  • Key support for RSA, ED25519, SECP

What remaining

  • Focused unit tests (eg parsing different types of keys, handshake state etc)
  • Better integration tests
  • Documentation

To-Do

  • Clean up commit history
  • Add or update documentation related to these changes
  • Add entry to the release notes

Cute Animal Picture

Cute

@AkMo3
Copy link
Collaborator Author

AkMo3 commented Jul 13, 2025

Everyone is encouraged to review the PR and suggest changes/improvements/skill-issue etc

@seetadev seetadev mentioned this pull request Jul 14, 2025
4 tasks
@seetadev
Copy link
Contributor

@AkMo3 : Thank you so much for your excellent contribution and for opening the PR on QUIC support — it’s a significant step forward for the py-libp2p ecosystem. We truly appreciate the time, effort, and deep thought you've put into integrating aioquic as a transport — it’s not only technically impressive but also strategically important for the broader libp2p community.

As a next step, we’ll be consolidating our previous development efforts and integrating them into your PR. This will ensure all related work is streamlined, reviewed collectively, and built upon consistently. Moreover, your contribution is enabling us to extend transport interoperability with other libp2p modules — an exciting milestone. I've looped in @guha-rahul and @lla-dane to collaborate on this integration and testing effort. We’re also setting up a dedicated py-peer branch at libp2p/test-plans to track and document these interoperability tests.

I ran the CI/CD pipeline earlier and noticed one of the checks is currently failing. I’ll be sharing a detailed report and suggested fixes on a separate discussion thread so we can troubleshoot collaboratively.

In parallel, @lla-dane, @guha-rahul, and I have been actively testing aioquic peer-to-peer connectivity. As of today, we've successfully exchanged hello messages between peers using the new QUIC transport — which is a fantastic sign of progress! Big thanks to @lla-dane for validating this. I’ll be requesting him to open a detailed discussion soon at libp2p/py-libp2p#578, so others in the community can follow along and contribute.

Finally, since this is a major addition to the py-libp2p transport stack, we would be grateful if you could share your implementation journey in a discussion post — insights into the challenges you faced, your approach to using aioquic, and any relevant learnings. It would be incredibly valuable to future contributors and maintainers. Once the CI/CD issues are resolved, we’d also appreciate it if you could add more test plans and a newsfragment to round out the PR.

Thanks once again for your excellent work and for being part of this initiative. We’re excited to keep building this with you :)

@seetadev
Copy link
Contributor

@AkMo3 : I ran the full test suite and noticed that one test case failed — however, it appears unrelated to your QUIC changes. The failing test is:

FAILED tests/core/pubsub/test_gossipsub_px_and_backoff.py::test_peer_exchange

This is part of the pubsub module and unrelated to transport layers, so we will handle it separately. I’ll open a dedicated discussion to document this and explore the root cause independently.

CCing @mystical-prog, @acul71 and @sumanjeet0012 , who could help resolve this issue soon.

@seetadev seetadev mentioned this pull request Jul 14, 2025
3 tasks
@lla-dane
Copy link
Contributor

lla-dane commented Jul 14, 2025

In rust-libp2p, the QUIC multiaddress format is like this: /ip4/127.0.0.1/udp/52182/quic-v1 and here it is: /ip4/127.0.0.1/udp/47788/quic. rust-libp2p has quic-v1 instead of only quic. We should change it to quic-v1 here also.

@lla-dane
Copy link
Contributor

lla-dane commented Jul 14, 2025

Ran the ping-example with QUIC transport, the ping is exchanging successfully, but after the 1st ping is exchanged, the dialer terminates itself. But at the listener side, I think the dialer's stream is not ended, listener is sending ping-packets again and again, but not getting a response, and finally terminates with timeout.

Logs at the left side is of the listner, and the other side are the dialer's logs

Relevant logs:

image

Timeout logs at the listener side:
image

@lla-dane
Copy link
Contributor

Tried to run the ping example with py<>rust sides.

I think if no quic-config is provided with transport-opt here while building the swarm, it takes in the default QUICTransportConfig in __init__.py::new_swarm():

 if listen_addrs is None:
        transport_opt = transport_opt or {}
        quic_config: QUICTransportConfig | None = transport_opt.get('quic_config')

        if quic_config:
            transport = QUICTransport(key_pair.private_key, quic_config)
        else:
            transport = TCP()
    else:
        addr = listen_addrs[0]
        if addr.__contains__("tcp"):
            transport = TCP()
        elif addr.__contains__("quic"):
            transport_opt = transport_opt or {}
            quic_config = transport_opt.get('quic_config', QUICTransportConfig())
            transport = QUICTransport(key_pair.private_key, quic_config)
        else:
            raise ValueError(f"Unknown transport in listen_addrs: {listen_addrs}")

and in the default QUICTransportConfig, I see this:

@dataclass
class QUICTransportConfig:
    """Configuration for QUIC transport."""
	....

    # TLS settings
    verify_mode: ssl.VerifyMode = ssl.CERT_NONE
    
    ....

I think it means the transport will not run certificate verification. But I ran the py-listner<>rust-dialer on QUIC, and these are the logs:

image

I seems its failing on certificate verification. What will be the correct way to run ping examples of py<>rust with each other? @AkMo3

@seetadev
Copy link
Contributor

@AkMo3 : Hi Akash. The CI/CD issue got resolved.

Wish to share to please respond to @lla-dane 's feedback on transport interop with rust-libp2p.

Kindly also add a newsfragment.

We are heading towards a final review + merge stage :) Appreciate your great efforts and initiative.

Next, we will be trying transport interop with other libp2p modules and submitting test plans at https://github.com/libp2p/test-plans . We will start with interop with the following libp2p modules: go-libp2p, rust-libp2p, dotnet-libp2p and nim-libp2p.

signature_payload = b"libp2p-tls-handshake:" + cert_public_key_bytes

try:
public_key.verify(signature_payload, signature)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while verification , shouldnt we hash the data before verifying it?

@seetadev
Copy link
Contributor

@AkMo3 : Thank you for your continued effort. Please share feedback to the suggestions by @lla-dane and @guha-rahul.

@lla-dane and @guha-rahul : Wish if you could work on the test suite with @AkMo3 and help take QUIC to production.

CCing @pacrob for his feedback and pointers on the PR.

@AkMo3
Copy link
Collaborator Author

AkMo3 commented Jul 20, 2025

Grateful to everyone for all the suggestions and feedback around QUIC. I apologise for not being able to respond to the queries, I am going through a bit of a rough patch around my health and will quickly spin up the wheels as soon I recover.

@seetadev
Copy link
Contributor

@AkMo3 : Hi Akash. Thank you so much for your thoughtful message — and no need to apologize at all! Your work on QUIC has been absolutely wonderful and deeply appreciated by everyone in the community.

Please prioritize your health and take all the time you need to recover fully — we’ll be right here when you’re ready to jump back in. Your contributions have already made a big impact, and we're looking forward to collaborating more once you're feeling better. Sending you strength and best wishes for a smooth and speedy recovery.

Take care, and thanks again for everything you’ve done so far.

@seetadev
Copy link
Contributor

seetadev commented Jul 21, 2025

@AkMo3 , @lla-dane and @guha-rahul : Wish to share that there are some merge conflicts coming up, which need to be resolved.

Seems like due to recent merge of PR: #766 (we updated py-multiaddrs).

CCing @acul71 and @sumanjeet0012 .

acul71 added a commit to acul71/py-libp2p-fork that referenced this pull request Jul 23, 2025
@acul71
Copy link
Contributor

acul71 commented Jul 23, 2025

@AkMo3 take a look to AkMo3#2 (comment)
In pyproject.toml Please use

    # "multiaddr>=0.0.9",
    "multiaddr @ git+https://github.com/multiformats/py-multiaddr.git@db8124e2321f316d3b7d2733c7df11d6ad9c03e6",

for py-multiaddr until we can publish to PyPi

@seetadev
Copy link
Contributor

@AkMo3 : Hi Akash. Hope you are doing well. Can you give access to @lla-dane and @guha-rahul so that they can help complete the QUIC PR?

Wish to share that there are some merge conflicts that need to be resolved soon. Looking forward to a production version of QUIC in py-libp2p soon.

@AkMo3
Copy link
Collaborator Author

AkMo3 commented Aug 11, 2025

Sure, provided access to @lla-dane and @guha-rahul.

@seetadev
Copy link
Contributor

@AkMo3 , @lla-dane , @guha-rahul : Wish if the merge conflicts could be resolved. Was trying to re-run CI/Cd pipeline.

@seetadev
Copy link
Contributor

@AkMo3 : Fantastic progress, Akash. Also, appreciate the support by @lla-dane and @guha-rahul. Great work, friends.

Each of these fixes addresses an important piece of the transport layer puzzle:

ASN.1 format certificate extension: great fix — making sure we’re using the correct format here is critical for interoperability and for keeping peer identity verification standards-compliant.

Peer ID derivation on dial: skipping this when necessary, ensures we don’t run into false negatives or block valid connections while still keeping the verification step intact where it matters.

QUIC multiaddr test fixes: tests are always a good indicator of correctness, and fixing the maddr parsing/handling is essential for ensuring real-world dialing works smoothly.

Allowing accept stream to wait indefinitely: this is a thoughtful change — it improves stability by making sure we don’t cut off valid streams just because of arbitrary timeouts.

Stream opening and connection parameter changes: great to see these adjustments — setting parameters correctly up front will prevent a lot of subtle bugs down the line, especially as QUIC is sensitive to handshake and stream state management.

Bringing all of this together, the module is much closer to a production-ready state. The flow of commits shows a good balance of fixing edge cases, improving test coverage, and refining core functionality. Once checks finish running and the branch is rebased cleanly, this will be in excellent shape for deeper review and merge.

Wish to also thank @lla-dane and @guha-rahul for their wonderful initiatives too. Great progress everyone :)

@seetadev
Copy link
Contributor

@AkMo3 , @lla-dane and @guha-rahul: Thank you for making improvements and testing QUIC PR.

Re-ran the CI/CD pipeline. We do have some new merge issues that need to be resolved.

else:
addr = listen_addrs[0]
is_quic = addr.__contains__("quic") or addr.__contains__("quic-v1")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have the is_quic_multiaddr function in utils. Could we use that here instead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup

self._remote_peer_id = remote_peer_id
self._local_peer_id = local_peer_id
self.peer_id = remote_peer_id or local_peer_id
self.__is_initiator = is_initiator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the double underscore here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, there is no requirement of double underscore any longer. Refractoring.

@lla-dane lla-dane force-pushed the main branch 2 times, most recently from a335595 to 89cb8c0 Compare August 30, 2025 08:42
@lla-dane
Copy link
Contributor

Merge conflicts are fixed @seetadev

@seetadev
Copy link
Contributor

@AkMo3 and @lla-dane : Great work. We are almost there. Please also address feedback points shared by @pacrob and add a newsfragment.

Please share notes at the discussion page whenever convenient this week. Doing a final review. Great work.

@AkMo3
Copy link
Collaborator Author

AkMo3 commented Aug 31, 2025

@seetadev Thanks a lot for the constant help through the PR. I believe we are at juncture were we can start performing the final review. @pacrob Looking forward towards more of your reviews!

@lla-dane @guha-rahul While the review are going, we can parallely analyse the current code and branch coverage of tests for quic module that we can later expand to entire repo. Let me know your thoughts on this.

@seetadev
Copy link
Contributor

seetadev commented Sep 1, 2025

@AkMo3 : Thanks a lot for driving this forward and for the consistent effort you’ve put into shaping the QUIC implementation. 🙌 Great to see the PR reaching the stage where we can begin the final review cycle. I’ll make sure to go through it thoroughly and share my comments soon.

Lets now try dialing to rust-libp2p and/or go-libp2p, while the final review process is on. We'll have a separate tracking issue on transport (quic) interop efforts with other libp2p modules. @acul71 (Luca) will help us here.

Also, I agree with your suggestion to start looking at branch and code coverage in parallel. Getting clarity on the current test coverage for the QUIC module — and then expanding that discipline repo-wide — will give us both higher confidence in stability and a stronger foundation for future contributions.

@pacrob: Thank you so much for your reviews and feedback. Appreciate it. This PR is indeed ready for final review + merge :) Looking forward to merging it after careful review of the code and testing for next 4-5 days.

@lla-dane @guha-rahul, your analysis of test coverage can really help us close the loop cleanly. Also, CCing other key members of libp2p community for their thoughts and feedback at the QUIC discussion page.

This feels like a key milestone for py-libp2p, and I’m excited to see it landing this week 🚀

@seetadev
Copy link
Contributor

seetadev commented Sep 1, 2025

@AkMo3 and @lla-dane : Wish to share that we are having some merge conflicts.

Updated the branch and merged this PR: please visit #876

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants