replace async_service with anyio_service

[Khwahish29]

What was wrong?

The codebase previously relied on a custom async_service implementation, which was no longer actively maintained and introduced unnecessary complexity. It made lifecycle management fragile and lacked support for modern async patterns and structured concurrency.
Issue #524

How was it fixed?

The legacy async_service implementation was fully replaced with a new design built using anyio, a well-supported, high-level async framework. This transition modernizes the service infrastructure and significantly improves reliability, maintainability, and task supervision.

Cute Animal Picture

[pacrob]

You don't need to edit the release notes directly. Create a newsfragment for each section you want to update with the original issue, e.g. 524.feature.rst, 524.docs.rst, etc. Newsfragments are then compiled into release notes with each release.

[Khwahish29]

I was facing the same issue as mentioned in #595 , while tinkering to solve it I accidentally committed the unnecessary changes, will revert them back, sorry for the confusion caused due to this

[mystical-prog]

hey @seetadev
I am assisting @Khwahish29 on this PR as it involves changes majorly around Gossipsub implementation, so I thought it would be a good idea to collaborate on this one.

[Khwahish29]

hey @seetadev I am assisting @Khwahish29 on this PR as it involves changes majorly around Gossipsub implementation, so I thought it would be a good idea to collaborate on this one.

Thanks for jumping in, @mystical-prog - really appreciate your support on this! Since the PR touches quite a bit on Gossipsub, it’s great to have your expertise here. I’ll continue driving this forward and loop you in wherever needed for the protocol-specific bits. Looking forward to refining this together! 🚀

[seetadev]

hey @seetadev I am assisting @Khwahish29 on this PR as it involves changes majorly around Gossipsub implementation, so I thought it would be a good idea to collaborate on this one.

@mystical-prog : Awesome :) great to hear.

Thank you so much for the initiative. This is indeed a major PR. Please open a discussion page and add developer documentation side by side, if convenient.

[mystical-prog]

I will add all the necessary documentation and references so that everyone remains in loop, looking to close this ASAP

[paschal533]

Hi @mystical-prog, How is the progress going? I will be fixing the failing tests in this PR from today and will keep you posted

[mystical-prog]

hey @paschal533
I believe @Khwahish29 has done a wonderful job in getting the basic structure ready, all that is left now is to fix CI and make sure that tests are passing !

[seetadev]

hey @paschal533 I believe @Khwahish29 has done a wonderful job in getting the basic structure ready, all that is left now is to fix CI and make sure that tests are passing !

@paschal533 : Completely agree with @mystical-prog. Wish to share that the key code commits are almost complete. Vraj and Khwahish have arrived at a good conclusion on the feature enhancement.

Wish to recommend you to bootstrap interop tests with JS-libp2p with focus on ping protocol. I'll be opening the issue and assigning it to you and Luca. On the same note, @mystical-prog authored ping protocol in cohort-1 and will be able to help arrive at a good conclusion on the issue.

@mystical-prog : Thank you for the update. Appreciate the initiative and efforts by you and @Khwahish29. I'll add you to the thread on interop tests between py-libp2p and js-libp2p with focus on ping protocol.

[Khwahish29]

hey @seetadev

we have identified an issue with anyio integration, due to a peer p2pclient's dependency of anyio being strictly less than 2.0.0, we are stuck with outdated anyio version, can you please guide on how to arrive at a solution which uses the latest anyio version and resolve p2pclient's dependency, we are currently blocked on this.

[seetadev]

hey @seetadev

we have identified an issue with anyio integration, due to a peer p2pclient's dependency of anyio being strictly less than 2.0.0, we are stuck with outdated anyio version, can you please guide on how to arrive at a solution which uses the latest anyio version and resolve p2pclient's dependency, we are currently blocked on this.

@Khwahish29 : Hi Khwahish. Thank you for sharing the issue.

We are facing similar kind of issues in pymultiaddrs too. Did share pointers and feedback with you.

I'll investigate possible resolutions and discuss with you and @mystical-prog soon.

Please check if a newer version of p2pclient exists that has relaxed or updated its anyio dependency (i.e., supports anyio>=2.0.0 or higher). If such a version exists, upgrading p2pclient directly should resolve the conflict.

If no such version exists, we may want to reach out to the maintainers or check open issues/PRs to see if there is any ongoing work in this direction.

We might want to update the package locally (we just did that for multiformats/pymultiaddrs) in case we are not getting the requisite support. In case of pymultiaddrs, the multiformats have agreed to let Py-libp2p dev team be the external maintainers of the repo.

@pacrob : Wish to have your thoughts, feedback and pointers on the above approach.

[pacrob]

It looks like a similar issue to pymultiaddrs, but not as complicated. The library - https://github.com/mhchia/py-libp2p-daemon-bindings - is in the personal repository of one of the original py-libp2p authors.

In order to move forward quickly, we can fork the repository and make changes to it, then update the dependency in py-libp2p to pull from that new fork instead of pypi. Since it is only a test dependency, I do not have any immediate security concerns.

We can then either:

• Contact @mhchia and ask for permission to take over the pypi project
• Publish our own fork (with credit) under a different name on pypi

It may also be worth investigating importing the code directly in to the py-libp2p testing utils (tests/utils). If the code is small enough, this might be simpler than maintaining a new repo.

[seetadev]

It looks like a similar issue to pymultiaddrs, but not as complicated. The library - https://github.com/mhchia/py-libp2p-daemon-bindings - is in the personal repository of one of the original py-libp2p authors.

In order to move forward quickly, we can fork the repository and make changes to it, then update the dependency in py-libp2p to pull from that new fork instead of pypi. Since it is only a test dependency, I do not have any immediate security concerns.

We can then either:

• Contact @mhchia and ask for permission to take over the pypi project
• Publish our own fork (with credit) under a different name on pypi

It may also be worth investigating importing the code directly in to the py-libp2p testing utils (tests/utils). If the code is small enough, this might be simpler than maintaining a new repo.

@pacrob : Thank you so much for your feedback, guidance and pointers. Appreciate your great support.

Absolutely agree with your assessment—this does feel similar to the situation with pymultiaddrs, but definitely more contained, especially given that it's scoped to test utilities. I appreciate you outlining multiple pragmatic paths forward.

Forking the original repository to unblock our current development sounds like a great immediate step. I’d be happy to collaborate with @mystical-prog , @Khwahish29 and @acul71 to help set that up and update the dependency in py-libp2p to point to the new fork. This gives us flexibility without delaying improvements or CI pipelines.

I also like that you've laid out longer-term options—particularly contacting @mhchia for PyPI publishing rights. That would keep things clean and centralized, but having a backup plan like publishing under a new name (with attribution) ensures we’re not blocked either way. I’ll look into the repository size and dependency usage to see if inlining the relevant code into tests/utils is feasible—it could indeed simplify maintenance if the logic is self-contained enough.

Thank you once again. We will go ahead with the fork + temporary dependency patch for now, and we can parallel-track the PyPI ownership or consolidation discussion. I can draft a short summary and ping @mhchia to check if they’re open to transferring ownership or collaborating on a shared solution.

Thanks again for helping move this forward efficiently—really appreciate your clear direction.

[manny-uncharted]

@seetadev has there been any progress on the fork of https://github.com/mhchia/py-libp2p-daemon-bindings any way to help unblock this issue.

[seetadev]

@seetadev has there been any progress on the fork of https://github.com/mhchia/py-libp2p-daemon-bindings any way to help unblock this issue.

@manny-uncharted : Thank you for your kind feedback.

Yes, please point to this repo at this juncture: please visit https://github.com/seetadev/py-libp2p-daemon-bindings . We have included all the changes as needed.

Please proceed with resolving the merge conflicts and arrive at a good conclusion on this important PR. Kindly discuss with @mystical-prog and @Khwahish29 once before moving ahead. You can collaborate with them on this PR.

[manny-uncharted]

Thanks I'd do just that @seetadev

[mhchia]

Hey @pacrob @seetadev, sorry I didn't notice I was tagged until recently. Can you provide a TL;DR that I can do to help you guys move forward?

[seetadev]

Hey @pacrob @seetadev, sorry I didn't notice I was tagged until recently. Can you provide a TL;DR that I can do to help you guys move forward?

Hey @mhchia 👋

Great to hear from you—and absolutely no worries at all about the delay. You've been an incredible part of the py-libp2p journey, and we're grateful for the strong foundation you helped build, especially with your work on py-libp2p-daemon-bindings.

Here's a quick TL;DR of where we’re at and how you might be able to help:

• We’re currently working on shifting py-libp2p to an anyio-based implementation to improve concurrency handling and modernize the async stack.
• As part of that, we needed to update the daemon bindings to match the new async patterns and improve compatibility across different Python environments.
• We have been updating the repo here: please visit https://github.com/seetadev/py-libp2p-daemon-bindings — it's actively being maintained as we modernize the stack.
• Your insight into the design choices and structure—especially around how message encoding/decoding, stream handling, and protobuf framing were handled—would be super valuable as we validate the updated bindings and ensure full interoperability.

If you have some time, it would be amazing if you could:

• Review the changes made in our current fork (even a high-level look would help).
• Share any pointers or feedback we should watch out for as we align the bindings with the latest daemon specs and py-libp2p core updates.
• Let us know if there are patterns or utilities you originally considered but didn’t add—we’d love to pick up where you left off if anything was left on the table.

Really appreciate you checking in—your past work continues to be a pillar for the project, and we’d love your feedback and thoughts as we take the next steps forward. Thanks again and excited to sync more.

[seetadev]

@mhchia : Wish if you could share the best way to reach out to you. Wish to share some progress on py-multiaddr and also send a request memo for including me (@seetadev ), @pacrob, @acul71, @dhuseby & @arcinston as contributors to PyPI package of py-multiaddr at https://pypi.org/project/multiaddr/

Over the past three months, we’ve taken up active maintenance of py-multiaddr, and have been updating the repository here: https://github.com/multiformats/py-multiaddr.

py-multiaddr is now central to multiple ongoing improvements in py-libp2p:

It's key to our upcoming QUIC support, which you can check out here: QUIC PR #763 and the related discussion #578.

It’s also being used in our revamped Bootstrap module, detailed in Bootstrap PR #711 and discussion #751.

And currently, py-multiaddr is referenced via github in PR #766 as we align dependencies and streamline peer discovery flows.

Wish if we could update and use https://pypi.org/project/multiaddr/. Also, update the package to include our latest PRs which have been merged in py-multiaddr (please visit https://github.com/multiformats/py-multiaddr/pulls?q=is%3Apr+is%3Aclosed ).

Thanks again for your contributions in py-libp2p. Your work has enabled a lot of what we’re building now—and we're excited to carry it forward with care and momentum. Appreciate your support and looking forward to hearing from you.