Weird behavior - occasionally, an area gets stuck and *always* turns overhead lights on even when no states are assigned

[redxdev]

So I've had some really odd behavior with one specific magic area that I setup - the overhead lights like to turn on during occupancy regardless of any other states, even if zero states (not even occupancy) are assigned to the overhead lights. This has happened twice so far, and a complete restart of HA has fixed it both times.

It doesn't matter if the area is in sleep, accented, extended, and it doesn't matter if zero states are assigned to the overhead lights - they will turn on no matter what as long as light control is enabled and this bug is occurring.

My best guess right now is that reconfiguring certain options in an area can result in the integration operating on some stale data. The only thing I have to back this up is that the first time this problem happened I was seeing some warning logs about magic areas trying to turn the sleep lights in that area on/off despite no lights being assigned (the warning being that the sleep light group didn't exist). So clearly there was something going wrong, though I can't say for sure this warning and the overhead lights always turning on are related.

I'd make this an issue but I don't have any kind of repro or debug logging yet, as the issue is inconsistent and it goes away when restarting HA. Going to enable debug logs prematurely to see if I can capture any more info the next time it happens, but in the meantime I figured I'd make a thread in case the solution is obvious.

[tbrasser]

I think I also see "this" behaviour.

[jseidl]

Interesting. Keep me updated if you manage to get more information on this. I had an issue in the past that setting some options to blank would cause MA to use the default values but (I don't remember quite how) it was fixed.

I have a pending task of switching from hass.data into the new config_flow.runtime_data stuff, maybe that fixes? (hard speculation). Anyway, keep me posted!

[redxdev]

Ok, so I've found a reliable repro for me at least.

Area has some number of motion sensors setup, and overhead lights are configured for states occupied and extended. I also have an input_boolean I use to trigger sleep mode.

From a clean reboot of HA:

1. Add all motion/occupancy/presence sensors into the "Exclude entities from being analyzed" list, save. Occupancy for the area should clear.
2. Remove those sensors from the exclude list so that they get considered again, save. Occupancy for the area should get triggered and turn on the overhead lights.
3. Enable sleep mode. Observe that the overhead lights don't turn off.
4. Reboot HA entirely (reloading the integration is not enough), observe that sleep mode works properly again.

Working on getting some logs for this, lots of spam from areas I wasn't testing so I may disable those first.

[redxdev]

Or... not. This is super inconsistent, now I can't get it to bug out.

[redxdev]

So I still don't know what's up with the "old" data that was showing up in logs (magic areas trying to control groups that no longer exist doesn't seem like something wrong with my setup specifically), but I do have a number of factors that probably contributed to lights acting wonky:

1. I am using zigbee groups to control the majority of my lights
2. I have multiple sets of groups that can cover the same lights to avoid issuing a ton of commands across multiple groups (my kitchen, for example, is actually made up of 3 groups of lights that sometimes I want to control independently and sometimes I want to control together)
3. adaptive lighting, combined with the above, can result in some lights always turning on depending on how the groups were added to it. In particular, if any single light in a group that adaptive lighting is enabled for is turned on by itself, adaptive lighting may see this and then try to turn the whole group on (since it only sees the group turning on, not the individual light).

My mistake was adding some of the overlapping groups to adaptive lighting, which resulted in a mess where turning some groups on would turn other groups on. Anyway, things seem solved now that I reworked groups to avoid this issue.

Honestly, zigbee's group system is just such a mess - individual control results in "popcorning" and unreliable delivery, group control results in flooding the network if you need to send more than 1-2 commands. Anyway, aside from that one phantom group in the logs (which doesn't seem to have come back) I'd say this was user error, though I'll blame zigbee's architecture in part :)

[jseidl]

I never used Zigbee groups myself but in plain HA, having lights member of multiple groups can be funky if you have automations on those.

For adaptive lighting, you can set the "All Lights" group on it as the single entity as it will always turn on when one of the lights in the area turns on (triggering adaptive lighting) and when Adaptive Lighting adapts them, the Magic Light Group routes the call only to active lights.

[redxdev]

Yeah unfortunately having lights as part of multiple groups is all but a requirement for my setup - I have a single area with 3 sets of lights (4 ceiling lights, another single ceiling light, and two hanging lights). Those are all individually controllable via switches on the wall (which are also zigbee switches, and thus must be grouped with the lights they control for proper operation and binding), but I also generally want to control them all as a single unit for automation with, say, magic areas.

Sending commands to 3 zigbee groups at the same time seems to be a no-go - my understanding is that group commands are sent to every device on the network because devices manage their own group membership, but this results in hitting limits on how many commands can be sent. So I need additional groups in order to control that area's lights as a single unit to avoid that limit (3 commands sent to every device to handle 3 groups vs 1 command sent to every device to handle 1 group).

Really sucks that zigbee's architecture seems to require this. Not sure if zwave or matter/thread have the same kind of problem as I don't have big groups of devices of those types (or any, in the case of thread).