Add chat log subscription endpoint #155287
Conversation
balloob
requested review from
a team,
arturpragacz and
synesthesiam
as code owners
|
Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
|
Hey there @home-assistant/core, @synesthesiam, @arturpragacz, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
There was a problem hiding this comment.
Pull Request Overview
This PR adds WebSocket subscription endpoints for chat log monitoring to improve debugging of LLM interactions in voice assistant sessions. The implementation allows real-time observation of conversation data exchanged between LLMs and Home Assistant.
Key changes:
- Added two WebSocket subscription endpoints for chat log access: individual conversation and index (all conversations)
- Implemented event notification system for chat log lifecycle (created, updated, deleted, content_added)
- Added
as_dict()serialization methods and timestamp tracking to content models
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
homeassistant/components/conversation/http.py |
Implements WebSocket subscription handlers for chat logs |
homeassistant/components/conversation/chat_log.py |
Adds event subscription infrastructure and serialization methods |
homeassistant/components/conversation/const.py |
Defines chat log event type enumeration |
homeassistant/components/homeassistant/const.py |
Fixes typo in exposed entities key name |
tests/components/conversation/test_http.py |
Tests WebSocket subscription endpoints and event flows |
tests/components/conversation/test_chat_log.py |
Tests chat log subscription and event notification system |
|
|
||
| role: Literal["system"] = field(init=False, default="system") | ||
| content: str | ||
| created: datetime = field(init=False, default_factory=utcnow) |
There was a problem hiding this comment.
The SystemContent.as_dict() method excludes the created field while other content classes include it. This inconsistency means system content timestamps won't be available to subscribers, which could impact debugging scenarios where system prompt timing matters. Consider including created in the serialized output for consistency with other content types.
|
|
||
| with ( | ||
| async_get_chat_session(hass, subscribed_conversation) as session, | ||
| async_get_chat_log(hass, session) as chat_log, |
There was a problem hiding this comment.
The context manager retrieves the chat log but doesn't pass conversation_input, which differs from the pattern used elsewhere. This could lead to inconsistent behavior if async_get_chat_log requires this parameter for proper initialization. Review whether conversation_input should be passed here for consistency with other usages.
| async_get_chat_log(hass, session) as chat_log, | |
| async_get_chat_log( | |
| hass, | |
| session, | |
| ConversationInput( | |
| text="", | |
| context=None, | |
| conversation_id=subscribed_conversation, | |
| agent_id=None, | |
| language=None, | |
| device_id=None, | |
| user_id=None, | |
| ), | |
| ) as chat_log, |
| del connection.subscriptions[msg["id"]] | ||
|
|
||
| unsubscribe = async_subscribe_chat_logs(hass, forward_events) | ||
| connection.subscriptions[msg["id"]] = unsubscribe | ||
| connection.send_result(msg["id"]) |
There was a problem hiding this comment.
The subscription is removed using msg["id"] but was added using msg_id. While these refer to the same value, using inconsistent variable names reduces code clarity. Use msg_id consistently throughout the function.
| del connection.subscriptions[msg["id"]] | |
| unsubscribe = async_subscribe_chat_logs(hass, forward_events) | |
| connection.subscriptions[msg["id"]] = unsubscribe | |
| connection.send_result(msg["id"]) | |
| del connection.subscriptions[msg_id] | |
| unsubscribe = async_subscribe_chat_logs(hass, forward_events) | |
| connection.subscriptions[msg_id] = unsubscribe | |
| connection.send_result(msg_id) |
|
I'm not a fan of the fact that this adds yet another mechanism to report events related to voice. |
|
@arturpragacz I hear you, however, chat log is not voice specific. It's the way we track the history of an interaction with an AI from either conversation or AI task. If we were to add these events into the Assist Pipeline websocket endpoint, we would now need to also make that one aware of AI task. Would you have another suggestion? |
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Can we not have a way to register a callback in the chat log, which can then be used in the higher layer. So the assist pipeline would use it to provide those events through its websocket and the AI task could do what is the best for it. I don't think we have any debug interface for AI tasks at this point anyway, unless I'm missing something. |
Breaking change
Proposed change
To improve debugging of LLMs, adding a set of endpoints to allow voice assistant debug sessions to look inside the chat log data to see what was sent between LLM and HA.
Frontend: home-assistant/frontend#27678
Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all.To help with the load of incoming pull requests: