Future Development of Homebridge Hue

[ebaauw]

UPDATE NOV 2024
UPDATE DEC 2023
UPDATE AUG 2023

Homebridge Hue is my oldest plugin. I started working on it in February 2016, and published the first version on GitHub in October 2016. I added support for deCONZ in the summer of 2017, including web socket notifications. In February 2020, I added HTTPS support for the gen-2 Hue bridge, followed by API v2 push notifications in December 2021.
Over that period, the number of different devices supported by Homebridge Hue has increased dramatically, fuelled by the consumerisation of Zigbee devices. Starting with (then) Philips Hue, OSRAM Lightify, and IKEA Trådfri, followed by Xiaomi, and many OEM devices, and even devices from Lidl and Aldi.
The code base has become increasingly complex over the years, and is due for a refactor. Homebridge Hue still uses Homebridge's traditional static platform accessory model (see #4), where most recent plugins use the dynamic platform accessory model, which offers many benefits. The Hue and deCONZ APIs have been diverging, and their v2 APIs are completely different, making it increasingly hard to combine Hue bridge and deCONZ gateway support in a single plugin.

Homebridge Hue is also my most used plugin. I don't collect any statistics, but the NPM registry mentions up to 8,000 weekly downloads, and GitHub Insights lists over 3,400 unique visitors.
Homebridge Hue is also, by far, my largest plugin; I'm exposing over 140 accessories from my deCONZ gateway. I would take me half a day to reconfigure HomeKit if those accessories would be removed from HomeKit and re-exposed.
Because of this, I've been hesitant to refactor Homebridge Hue. While I can technically manage to expose my accessories using a dynamic platform plugin, without HomeKit losing them, this is tricky and requires a deep understanding of the internal working of Homebridge and HAP-NodeJS. For me, a big-bang update would be out of the question, as I think for most users.

Going forward, I will create a new plugin for deCONZ, homebridge-deconz, and a new plugin for Hue APIv2, homebridge-hue2, while keeping the current Homebridge Hue plugin. These three plugins can be run in parallel, using different child bridges, connecting to the same Hue bridge(s) and/or deCONZ gateway(s). This allows you to setup a new HomeKit home for testing. By connecting the child bridges for all plugins to the same Home, white- and blacklisting can be used to move accessories over to the new plugins one-by-one. This will also allow me to release work-in-progress versions of the new plugins, not yet supporting all device types.
Eventually the current Homebridge Hue plugin will drop support for the gen-2 Hue bridge and the deCONZ gateway. After that, I might release a new version with the dynamic platform accessory model. I don't think there are enough users left using the gen-1 Hue bridge or HA-Bridge or Tasmote to justify the effort needed to provide a gradual migration path.

[HARDIK-TSH1392]

Hey, I am new to open source and I am finding it really difficult as to how to start doing anything. I just came across your project, could you guide/explain as to what should I do because I am interested in contributing to open source, so taking your project as a start can you help me contribute?

[ebaauw]

@HARDIK-TSH1392 better start with something far less technically complex.

[sschuste]

Wow. That's a lot of work you have to do. So much code and I'm only using it for two devices. And I don't have enough programming skills to contribute anything essential. If you're ever in Frankfurt am Main, your dinner's on me. But maybe you have some better ideas how we can support you.

[Olepir]

@ebaauw thank you for your job

my whole smart home is based on you plugin

can’t wait for your next project

[vvdheuvel]

Hi Erik, wise decision! 🙏🏻 I think this will take out some complexity and better focus on both platforms. When do you suspect it will stable for usage? Will there be a possibility to migrate?

[ebaauw]

Homebridge deCONZ should be pretty stable by now.

I cannot opaquely move accessories from Homebridge Hue to Homebridge deCONZ or Homebridge Hue2, without HomeKit noticing. In theory this would only be possible if all plugins use the same Homebridge installation (without child bridges). As Homebridge deCONZ and Homebridge Hue2 generate the same accessory UUID as Homebridge Hue, HomeKit wouldn't care which plugin handles the accessory (and doesn't even know), as long as it's exposed by the same HomeKit bridge (HAP server). However, only one plugin can expose that accessory, or you would see the dreaded "cannot add another accessory with the same UUID" message.

Homebridge Hue still uses the (ancient) static platform accessory model, where all accessories need to be declared to Homebridge, before it starts the HAP server. Homebridge deCONZ and Homebridge Hue2 use the (newer) dynamic platform accessory model, where a plugin can add (and remove) accessories at runtime, after Homebridge has started the HAP server. So I would need to restart Homebridge for Homebridge Hue to no longer expose an accessory, but Homebridge deCONZ and Homebridge Hue2, can only re-create that accessory, after the HAP server has started. This would leave a small window where the accessory isn't exposed, causing HomeKit to remove the accessory, and to treat the re-created accessory as a new one. I did have a hack in homebridge-lib, to delay the startup of Homebridge from a dynamic platform plugin, but that's too complex to share and no longer works with child bridges. I discussed with the team to include this feature in Homebridge, but it was deemed too dangerous, as too many users wouldn't understand why Homebridge wouldn't be visible in HomeKit, even though it is running.

To make a long story short: migration consists of removing the accessories by Homebridge Hue from HomeKit, and re-creating them by Homebridge deCONZ or Homebridge Hue2, losing any associations to HomeKit rooms, groups, scenes, automations. This sucks, but then, so does life. To mitigate the impact, I support manual migration per accessory, so you don't have to handle all the re-configuration at once. And it also allows me to release Homebridge deCONZ and Hue2 without full support for all devices.

Make sure to run Homebridge deCONZ and/or Homebridge Hue2 on a separate HAP server than Homebridge Hue. I strongly recommend to run the new plugins in their own child bridge. I would pair it to a new test home at first, to familiarise yourself with it, and to check which accessories you feel are production ready. Then un-expose all accessories, remove the child bridge from the test home, and pair it to your production home.

In the production home, select an accessory to expose. In HomeKit, you'll now see two versions of the same accessory, allowing you to add the new accessory to the same room, groups, scenes, and automations, while still being able to see how the old accessory is configured. Homebridge deCONZ now creates a migration resource link, containing all the resources it exposes. Homebridge Hue2 will do the same. Homebridge Hue can be configured to take into account these resource links (through the new homebridgeDeconz and homebridgeHue2 config.json keys). When you're satisfied with the config for the new accessory, restart Homebridge Hue, and the old accessory is removed.

Note that Eve stores the history under the accessory UUID, so the old and new accessories share the history, causing double entries. For accessories with Eve history, you want to make sure to load the history from the old accessory, before exposing the new one, and to remove the old accessory shortly after exposing the new accessory.

[jan666]

Oh, that sounds really horrible.

I guess its safe to just use the old homebridge-hue plugin forever? Re-doing everyhing in the Home App will be a real pain

[ebaauw]

Yes, that is horrible. That's why I had been delaying the refactor of Homebridge Hue for so long, up to the point where it's no longer sustainable to maintain it.

I cannot prevent that pain, unfortunately, but I won't force you when to take it, nor to take all of it at once. I will stop adding new devices and features in Homebridge Hue pretty soon, after both Homebridge deCONZ and Homebridge Hue2 are ready for mainstream production. I will eventually release a version 1.0 of Homebridge Hue that no longer supports the deCONZ gateway nor the gen-2 Hue bridge, but not within the next six months. Of course, you can continue to run (and even re-install) the 0.x version of Homebridge Hue.

[Ar-eh]

Does using an app like "Controller for HomeKit", with the ability to backup a HomeKit configuration and restore from that backup, address the migration difficulties? I've found that when migrating between Homebridge Hue and the HomeKit integration in HomeAssistant, the backup does not work perfectly due to differences in how accessories are exposed. But perhaps it would be smoother in migrating from Homebridge Hue to HomeBridge deCONZ?