New canopen-asyncio or migrate asyncio into canopen?

[sveinse]

I'm maintaining the canopen asyncio port https://github.com/sveinse/canopen-asyncio. The port is gaining in popularity and I need to decide what we should do next and where to put the efforts. Two main options:

a) Import the asyncio port to the main library, or
b) Make the asyncio into a separate, indepenent package

The port is functional, but not complete. More precisely, not at the same level of maturity as canopen. Tests are missing and some functions cia profiles (like 402), LSS, etc are incomplete.

The main question is: How mature do we need asyncio to be to be able to merge it into canopen? Continue on #272 and PR #359. The canopen project is not very big and we aren't that many maintainers, so I think I'm not mistaking if I'm saying this: I suspect it will take a while to get asyncio implemented.

I'd like to see the canopen asyncio published on pypi rather soon. People are using it and the current git reference to the asyncio fork is very tedious. The fork use the same package namespace "canopen", which makes it unsuited for a publish as-is. If option (b) involves renaming the source and package name to something unique, and it will be separate from the canopen package. Going down that route will require more efforts to keep the two projects in sync later. PRs won't work.

My intent with this post is to gauge how much interest there is to do the work of getting asyncio into canopen. Because if there isn't really any capacity to so, it might be best to split this out into a separate project.

[acolomb]

I'm definitely willing to integrate it. Just in recent times I haven't found the time to invest into this project. We have a little progress in that it now lives in a separate GitHub organization, which we now have control over without needing to wait for @christiansandberg. That is good.

There were some open concerns in those two linked tickets, regarding the API design. I'll need to catch up what the status was, as it's been some time now. But those are definitely possible to resolve, may just need some further discussion / summarizing options.

Anyway, an incomplete async port is still better than none at all, even when we are going to merge it into this upstream repo. Lots of features in this library are "incomplete" and best-effort, as much as someone cared enough to implement. So async won't stick out that much, and we will get targeted requests and maybe contributions for missing bits that people actually need.

My bigger concern is that we were in the process of adding more type hints and as part of that, improve test coverage. A big typing blocker was resolved with #525, more should be handled incrementally. We've had two attempts I remember where a big "this fixes all typing and includes unrelated fixes / changes" PR was effectively stopped. I'd still like to continue with gradually implementing the typing stuff, even though there will be an extended period where even newly added type hints will cause some warnings to trigger.

However, I don't see an immediate requirement to tackle the typing stuff before turning to async. It's just a large chunk that we had been working on during 2024, which kind of distracted me from the async PR (though not forgotten). We can definitely shift focus back to the async feature work and postpone typing-related discussions in the meantime.

Some fixes have accumulated though since v2.3.0, so it would make sense to release a v2.4.0 under the new organization first. Then the async port would lead up to a v3.0 release, which gives us some degrees of freedom regarding API breakage (which should still be avoided as much as possible though). Would that plan work out for you @sveinse?

I'll try to devote more time to the project again in the coming weeks / months. It's always a good motivation if there's someone else trying to get things done here, as it accelerates review and allows fruitful, timely discussions. Happy to hear whether you'll be able to also keep up your involvement during that time frame?

[sveinse]

Thank you @acolomb , overall I think this looks very good. Let's proceed with getting async into canopen!

There were some open concerns in those two linked tickets, regarding the API design. I'll need to catch up what the status was, as it's been some time now. But those are definitely possible to resolve, may just need some further discussion / summarizing options.

These issues and PRs haven't been updated in a while. I have a task to update the status before you or any delve into those issues and PRs I think. I'll give notice when its ready.

My bigger concern is that we were in the process of adding more type hints and as part of that, improve test coverage. A big typing blocker was resolved with #525, more should be handled incrementally. We've had two attempts I remember where a big "this fixes all typing and includes unrelated fixes / changes" PR was effectively stopped. I'd still like to continue with gradually implementing the typing stuff, even though there will be an extended period where even newly added type hints will cause some warnings to trigger.

My plan is to incrementally implement asyncio PRs along the same lines as the typing fixes were to keep the cognitive load with them as small as possible.

However, I don't see an immediate requirement to tackle the typing stuff before turning to async. It's just a large chunk that we had been working on during 2024, which kind of distracted me from the async PR (though not forgotten). We can definitely shift focus back to the async feature work and postpone typing-related discussions in the meantime.

I agree. The typing stuff were much needed, so I'm thankful this is in place. It should not be a show-stopper for any of the upcoming asyncio work and any typing improvements can proceed in parallel on the side as needed.

Some fixes have accumulated though since v2.3.0, so it would make sense to release a v2.4.0 under the new organization first. Then the async port would lead up to a v3.0 release, which gives us some degrees of freedom regarding API breakage (which should still be avoided as much as possible though). Would that plan work out for you @sveinse?

Yes, this is a good plan I think. Getting a v2.4.0 out to pave the way for a v3.0 where we can (and will) break stuff is really wise.

I'll try to devote more time to the project again in the coming weeks / months. It's always a good motivation if there's someone else trying to get things done here, as it accelerates review and allows fruitful, timely discussions. Happy to hear whether you'll be able to also keep up your involvement during that time frame?

I'm absolutely motivated to get the asyncio port implemented (and anything else that might pop up). My day job is a tech manager, so my schedule is quite busy. I can't promise 24/7-like support and there will be periods of inactivity. But I'll try as best I can to be involved. I do read every canopen issue.

[juliusz-t]

My bigger concern is that we were in the process of adding more type hints and as part of that, improve test coverage. A big typing blocker was resolved with #525, more should be handled incrementally. We've had two attempts I remember where a big "this fixes all typing and includes unrelated fixes / changes" PR was effectively stopped. I'd still like to continue with gradually implementing the typing stuff, even though there will be an extended period where even newly added type hints will cause some warnings to trigger.

(...) It's always a good motivation if there's someone else trying to get things done here, as it accelerates review and allows fruitful, timely discussions. Happy to hear whether you'll be able to also keep up your involvement during that time frame?

I'd be happy to help prepare the next library releases. I've just posted issue #570 related to the code quality. I can look into backporting code quality changes from upstream to the asyncio fork to make the merge easier in the future. I just need guidance on what is a priority at this moment.

[acolomb]

As far as I'm concerned, what you call "code quality" is not the biggest priority, as it doesn't improve what the code does, only how it looks. I'm all for style consistency, though it need not be completely automated. But as long as we have PRs pending for functional code issues, those should be worked on with higher priority than prettifying the code.

[sveinse]

I got the answer for my question and I think we have a plan in place for asyncio migration. I don't see any point in keeping this issue open. #272 and #359 will be kept open for that purpose. Closing the issue.