Release Cadence

[jamesonuk]

Just wanted to start a discussion about release cadence as there have been quite a few updates recently 😄 All a balance act of fixing things quickly (which is great 👍) vs spamming people with update notifications that don't affect them. Fully appreciate people want the latest definitions and doing scraping means they are constantly changing but I am not sure getting what seems to be an update notification every other day is appropriate. Just not sure if there is a better way to balance this

In an ideal world the definitions and code would be separate and you can sync them separately but given the definitions are code I think downloading arbitrary code is not really a goer.

Just wondering whether you could make use of whatever HACS calls beta releases? Push changes quickly and allow people to signup for beta releases but have a stable release which is updated in slower time?

[robbrad]

@jamesonuk fair and recognised ask

Instead I think we will move to staging released to a PR that groups other PRs. This will reduce the release cycles.

I’m not a fan of using beta release here as usually the release is not beta.

Part of the challenge is the project has doesn’t have a central development team it’s distributed to who ever contributes.

[jamesonuk]

Totally get it. Ideally it would all be CI/CD pipelines and updates would trigger a new release (Frigate card switched a little while ago but the difference it that being a front end card it just one click to update where as HA puts a limitation that integrations need a restart). Having all that setup reduces the amount of overhead you need to put in 😀

It is good that updates are flowing quickly and I don't think there is really a clean way of getting updates out quickly without sending lots of updates to every user.

Was thinking more of stable vs edge releases but think HACS only has something like pre-release but agreed here they are really full releases. Managing it in groups works just as well just more work for you.

Just looking in HACs now and I see this (I think it is a bit free with it's definition of last week but 10 is possibly a bit much

[robbrad]

Wouldn’t it be nice to turn off notifications for a particular integration!

[jamesonuk]

Wouldn’t it be nice to turn off notifications for a particular integration!

There is a whole world of improvements that could be made to third party integrations that could be sorted by having something like HACS built into HA. Again I understand why Nabu Casa don't want to (more accurately can't) provide a store without opening themselves up to all sorts of complaints from manufacturers and litigation (especially given a lot of what is on HACS is probably reverse engineering APIs and various other hacks and not even mentioning the opportunity to inject all sorts of nasty stuff into HA itself).

HACS is currently the best (only) option and I can't see that changing anytime soon but would be nice. You can skip updates so perhaps I might actually also put a submission into HACS to snooze updates for a set period.

In all honesty I probably wouldn't even notice an update a week and is all a bit moot but just wanted to raise it

[robbrad]

What I’ll do is look to protect master branch to incoming releases only not just any PR

the issue on release for the custom component is it pulls a pip library for each release it’s 121 relationship

[jamesonuk]

Given that the council files are just python files, why can't they they just update dynamically? Or load them from GitHub as they run?

many reasons. First one is loading them dynamically creates a cloud dependency, but that isn't the end of the world (you need internet connection to scrape the providers anyway)

Big one is security. Having a facility to download code directly is extremely dangerous (Just think of all the things you could do with a handful of lines of Python). HACS itself is a bit of a security nightmare but at some level you are probably pulling a GitHub release where changed have been reviewed by a person which add a vague semblance of security.

There are also various issues with the actual setup (this is why you need to restart HA for integration changes to be picked up).

Ultimately because the files are executable code not metadata definitions (necessarily so because of how this works) it is bad idea to allow things to be updated on the fly.

In fact perhaps that is something @robbrad you might want to consider... The HA best practice is that the integration and the logic should be separate with the logic available from Pypi (https://developers.home-assistant.io/docs/development_checklist). If you were to split out providers a a Python package and the integration as a separate one you can update one without the other (ie. update the provider package regularly and tie specific versions to the integration releases)

[robbrad]

It is a python package https://pypi.org/project/uk_bin_collection/

[jamesonuk]

It is a python package https://pypi.org/project/uk_bin_collection/

It is but when you download the integration you are also downloading the source of that package not the package itself.
Was thinking more about splitting what is essentially in the custom_components directory and what is in uk_bin_collection / uk_bin_collection_api_server directories into separate repositories. You build the pypi package from the one with the providers and just add that as a requirement in the integration

[robbrad]

No it doesn’t work like that. When you download a version of the custom component via HACs it has a requirement inside of it for the python package version. Then when you start HA the pip package is installed. The CC is just a wrapper. Now the list of councils and properties are dynamically loaded when you load the CC.

This is how most HA integrations work - the other way people use is code directly in the CC.

We opted for trunk based deployments due to the many many contributors and issues with long lived branches.

https://martinfowler.com/articles/branching-patterns.html#Trunk-basedDevelopment

Also a good read

https://trunkbaseddevelopment.com/

Moving to a stacked PR would move us out of truck based deployment mechanisms 😥

[jamesonuk]

No it doesn’t work like that.

It can. Take a look at
https://github.com/mattrayner/pod-point-home-assistant-component
https://github.com/JurajNyiri/HomeAssistant-Tapo-Control
https://github.com/make-all/tuya-local
or even Alexa Media Player but they do things in odd ways and it isn't probably the best thing to look at.....

All have the actual package doing the work in a separate repository to the integration decoupling the integration code from the doing code. In reality it is no different to any other dependency

This would allow trunk based development to continue fine (just mainly on the provider package). You then just bump the version in the requirements of the integration code to build a release

[robbrad]

@jamesonuk thanks

I took a look at podpoint and can’t see how it’s different to how we are doing it.

in the manifest is a ref to a pip lib like we do

are you saying the actual python and pip publishing is in a different repository all together?

My question would be how does that solve the issue of bumping the version of the custom component? That still needs to happen to pull the latest pip library- as far as I know ? Unless HA just gets latest every time from pip?

Sorry to plough on just keen to learn if there is something obvious I’m missing

[jamesonuk]

I took a look at podpoint and can’t see how it’s different to how we are doing it.

Obviously a million and one ways to skin a cat and you need to do what works for you but yes for Podpoint you have
https://github.com/mattrayner/podpointclient which is a separate repository and published as a package to Pypi. This can happen independently of the actual integration code which is in https://github.com/mattrayner/pod-point-home-assistant-component

putting the councils in a separate repository would mean you could take PRs directly against that and release as often as you want. If people do want to use the code without HA they can also use that package without having to get the custom component stuff too. You could also use a github pipeline to create and deploy this on each PR.

In the integration repository it just has a specific version of the client podpointclient==1.6.1 so when you wanted to release a new version you would need to bump that to whatever version you want to release.

The downside of doing it this way is the release notes are no longer automatic as the commits are on the other repository but not sure how that works (this is how Zigbee2MQTT works with almost daily updates to the underlying converters but monthly releases and the release notes include links to the converters repository https://github.com/Koenkk/zigbee2mqtt/releases)

[robbrad]

Got it. This was my understanding too