Lay the foundation for an AI assistant that helps maintainers manage their home

[nielsrowinbik]

Problem statement

Our "Assist" already lets users control their home through natural language. Things like turning off lights, adjusting heating, and so on are all possible through Assist. But Assist offers no help with managing the home: creating automations, setting up dashboards, diagnosing problems, or understanding what's going on. This kind of assistance is exactly where an AI assistant could add significant value for home "maintainers", and the community has already validated the demand through numerous custom integrations that fill this gap. We want to start offering this natively, beginning with the foundation that future capabilities can be built on top of.

This directly serves OHF's goal of making Home Assistant meaningfully more approachable: an AI assistant that helps maintainers set things up and troubleshoot reduces the expertise required to get the most out of HA. It also supports Nabu Casa's goal of growing HA Cloud adoption, as a native AI assistant is a natural driver for Cloud subscriptions.

Scope & Boundaries

In scope

• Adding the ability to handle management-oriented requests from home maintainers (either as an Assist extension or as something separate, TBD)
• The foundational architecture: how the assistant accesses home configuration, understands context, and takes or proposes management actions
• A first meaningful capability built on this foundation to validate the approach (TBD)

Not in scope

• Expanding control capabilities of the existing Assist (that work continues separately)
• Non-administrator access to management features
• Any specific advanced capabilities beyond the initial validating use case. Those are follow-on opportunities

Foreseen solution

A persistent, context-aware AI assistant surfaced throughout the HA interface. Always accessible, always aware of where in the UI it's being invoked and what the maintainer is working on. Similar in pattern to how Gemini is embedded across Google's products. The assistant understands the current context (e.g. which automation you're editing, which dashboard you're viewing) and offers relevant help without requiring the user to re-explain their situation. (A few quick explorations of what this might look like can be found in this Figma file.)

As a first concrete use case and migration target: exposing the Home Assistant documentation through this AI assistant.

Community signals

• Multiple custom integrations exist that add AI-assisted home management to Home Assistant

Risks & open questions

• We will need an AI back-end. Users can bring their own, and those will be fine, but is our own offering through Home Assistant cloud good enough for those that want a great out of the box experience?
• How do we handle actions that are destructive or hard to reverse? Does the assistant propose and confirm, or act directly?
• Privacy implications of giving an AI assistant access to full home configuration
• How does this interact with the existing Assist pipeline architecture?

Appetite

Large — 2 cycles. Deliberately scoped to foundation and a single validating capability, but needs lots of groundwork.

Execution issues

No response

Decision log

Date	Decision	Outcome

[felinusfish]

From the get-go, I think that basically all of this, where an AI has direct control of the home's critical automation infrastructure, there needs to be permission scoping. Admin or User doesn't cut it, because of how potentially destructive the AI's actions could be.

There also needs to be a degree of hand-holding no matter what kind of user utilises it, that could perhaps be bypassable via developer options for those who are willing to take on the risk of allowing the AI commit actions without confirmation.

While I don't think AI is scary and evil, there are definitely concerns that arise; if there were an Agent mode which could push multiple changes at a time, the risks associated are larger in magnitude. It could be great, but it could also drive people far, far away from both Nabu Casa Cloud and Home Assistant.

[nielsrowinbik]

Good points @felinusfish, and something we'll definitely take on board as we continue shaping this. Probably, because introducing permission scoping is a huge undertaking, we will start by always letting the user confirm whatever changes AI is proposing to make. There's some explorations of what this might look like in the Figma file linked in the original post.