Structured Output & New Timeline API

v1.6.0 Release Notes

This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output, a new Timeline API, and fixes various bugs.

⚠️ Breaking Changes

To avoid confusion with LLM Vision Memory:

• The remember action is now called create_event
• The remember parameter has been renamed to store_in_timeline.

Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.

If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.

Contributors

A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, @TheRealFalseReality, and @dcgrove!

Integration

✨ Features

• Structured Output: Valid JSON output can now be forced using the structure parameter. See the docs for more details. (by @rmaher001)
• Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
• Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the create_event action. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)

🔧 Improvements & Bug Fixes

• Create Events: The create_events action now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch)
• Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
• Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551, #553) (by @valentinfrlch)
• Special Characters: Improved handling of special characters in event titles (by @radzio)
• Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
• Unit Test Coverage: Added unit tests. (by @Smiley73)
• Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
• Fixed an issue that caused validation to fail for gpt-5 models when using Azure (#494) (by @valentinfrlch)
• Fixed an issue that caused newer claude models to fail silently (#562) (by @dcgrove)
• Pinned boto3 to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)

🌐 Languages

• Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
• Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)

Blueprint

If you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.

🔧 Improvements & Bug Fixes

• Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
• Fixes the “camera is undefined” error. (by @TheRealFalseReality)

Structured Output & New Timeline API

v1.6.0 Release Candidate 1 Release Notes

This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output and a new Timeline API.

⚠️ Breaking Changes

To avoid confusion with LLM Vision Memory:

• The remember action is now called create_event
• The remember parameter has been renamed to store_in_timeline.

Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.

If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.

Contributors

A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, @TheRealFalseReality, and @dcgrove!

Integration

✨ Features

• Structured Output: Valid JSON output can now be forced using the structure parameter. See the docs for more details. (by @rmaher001)
• Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
• Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the create_event action. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)

🔧 Improvements & Bug Fixes

• Create Events: The create_events action now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch)
• Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
• Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551, #553) (by @valentinfrlch)
• Special Characters: Improved handling of special characters in event titles (by @radzio)
• Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
• Unit Test Coverage: Added unit tests. (by @Smiley73)
• Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
• Fixed an issue that caused validation to fail for gpt-5 models when using Azure (#494) (by @valentinfrlch)
• Fixed an issue that caused newer claude models to fail silently (#562) (by @dcgrove)
• Pinned boto3 to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)

🌐 Languages

• Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
• Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)

Blueprint

If you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.

🔧 Improvements & Bug Fixes

• Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
• Fixes the “camera is undefined” error. (by @TheRealFalseReality)

Structured Output & New Timeline API

v1.6.0 Beta 1 Release Notes

This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output and a new Timeline API.

⚠️ Breaking Changes

To avoid confusion with LLM Vision Memory:

• The remember action is now called create_event
• The remember parameter has been renamed to store_in_timeline.

Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.

If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.

Contributors

A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, @TheRealFalseReality, and @dcgrove!

Integration

✨ Features

• Structured Output: Valid JSON output can now be forced using the structure parameter. See the docs for more details. (by @rmaher001)
• Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
• Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the create_event action. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)

🔧 Improvements & Bug Fixes

• Create Events: The create_events action now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch)
• Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
• Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551) (by @valentinfrlch)
• Special Characters: Improved handling of special characters in event titles (by @radzio)
• Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
• Unit Test Coverage: Added unit tests. (by @Smiley73)
• Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
• Fixed an issue that caused validation to fail for gpt-5 models when using Azure (#494) (by @valentinfrlch)
• Fixed an issue that caused newer claude models to fail silently (#562) (by @dcgrove)
• Pinned boto3 to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)

🌐 Languages

• Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
• Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)

Blueprint

If you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.

🔧 Improvements & Bug Fixes

• Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
• Fixes the “camera is undefined” error. (by @TheRealFalseReality)

Structured Output & New Timeline API

v1.6.0 Alpha 3 Release Notes

This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output and a new Timeline API.

⚠️ Breaking Changes

To avoid confusion with LLM Vision Memory:

• The remember action is now called create_event
• The remember parameter has been renamed to store_in_timeline.

Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.

If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.

Contributors

A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, and @TheRealFalseReality!

Integration

✨ Features

• Structured Output: Valid JSON output can now be forced using the structure parameter. See the docs for more details. (by @rmaher001)
• Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
• Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the create_event action. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)

🔧 Improvements & Bug Fixes

• Create Events: The create_events action now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch)
• Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
• Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551) (by @valentinfrlch)
• Special Characters: Improved handling of special characters in event titles (by @radzio)
• Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
• Unit Test Coverage: Added unit tests. (by @Smiley73)
• Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
• Pinned boto3 to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)

🌐 Languages

• Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
• Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)

Blueprint

For this alpha, please import the blueprint using this url:

https://github.com/valentinfrlch/ha-llmvision/blob/v1.6.0-beta/blueprints/event_summary_beta.yaml

If you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.

🔧 Improvements & Bug Fixes

• Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
• Fixes the “camera is undefined” error. (by @TheRealFalseReality)

Structured Output & New Timeline API

v1.6.0 Alpha 2 Release Notes

This release adds support for structured output and a new Timeline API.

⚠️ Breaking Changes

To avoid confusion with LLM Vision Memory:

• The remember action is now called create_event
• The remember parameter has been renamed to store_in_timeline.

Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version. For this alpha version please use the following url to import the v1.6.0-alpha blueprint:

https://github.com/valentinfrlch/ha-llmvision/blob/v1.6.0-beta/blueprints/event_summary_beta.yaml

If you use LLM Vision Cards, please update to v1.6.0 as well. The cards now use the new Timeline API, and older versions are no longer supported.

Contributors

A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, and @boian-ivanov!

Integration

✨ Features

• Structured Output: Valid JSON output can now be forced using the structure parameter. See the docs for more details. (by @rmaher001)
• Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. This is used by the LLM Vision Cards, but you may use the API for your automations as well. See the API docs for more details. (by @valentinfrlch)
• Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the create_event action. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)

🔧 Improvements & Bug Fixes

• Create Events: The create_events action now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch)
• Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551) (by @valentinfrlch)
• Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
• Unit Test Coverage: Added unit tests. (by @Smiley73)

🌐 Languages

• Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
• Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)

Structured Output & New Timeline API

v1.6 Alpha 1 Release Notes

⚠️ Please read the breaking changes before updating!

This update adds support for structured output and a new Timeline API.

⚠️ Breaking Changes

• The remember action is now called create_event to avoid confusion with LLM Vision Memory.

Contributors

A huge thank you to our contributors @rmaher001, and @boian-ivanov!

Integration

✨ Features

• Structured Output: Valid JSON output can now be forced using the structure parameter. See the docs for details. (by @rmaher001)
• Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. This is used by the LLM Vision Cards, but you may use the API for your automations as well. See the docs for details. (by @valentinfrlch)

🔧 Improvements & Bug Fixes

• Create Events: The create_events action now supports setting the category for an event manually. (#536) (by @valentinfrlch)

🌐 Languages

• Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)

Bug Fixes & Improvements

v1.5.2 Release Notes

⚠️ Please read the breaking changes before updating!

This update includes bug fixes and performance improvements.

Contributors

A huge thank you to our contributors @TheRealFalseReality, @bekdan1988, @alemuro, and everyone who helped test the pre-release versions!

⚠️ Breaking Changes

In our previous release (1.5.1), we moved snapshots to /config/media/llmvision/snapshots. This change caused issues for some users who were using other integrations that depend on the /media directory. As a result, the snapshots directory is moving again to /media/llmvision/snapshots.

What this means for you:

• Existing snapshots will automatically migrate to the new location when you restart after upgrading to 1.5.2.
• Remove the media folder configuration that you may have added to your configuration.yaml

  homeassistant:
    media_dirs:
      llmvision: /config/media/llmvision # Remove this line
      media: /media # Remove this line

• If you have any custom automations that use snapshots, update your template to:
  {{ response.key_frame.replace('/media','/media/local') }}
• If you use the Timeline Card, make sure to update it as well!

I apologize for any inconvenience this causes!

Integration

🌐 Languages

• Support for Hungarian language: Added 🇭🇺 Hungarian translations. (by @bekdan1988)
• Support for Catalan language: Added Catalan translations. (by @alemuro)

🔧 Improvements & Fixes

• Improved keyframe selection: Keyframe should now show the detected object more reliably. (by @valentinfrlch)
• Timeline events not deleted: Fixed a bug that caused events not being deleted properly. (by @valentinfrlch)
• Anthropic fix: Fixed an issue that caused validation to fail for the Anthropic provider. (#475) (by @valentinfrlch)
• Reduced image pipeline latency: Frames are now stored directly in memory instead of in a temporary directory. This reduces disk wear and reduces latency caused by I/O operations. It also fixes a race condition when running multiple actions in parallel. (#505, #510) (by @valentinfrlch)
• Moved Timeline snapshots: Snapshots are now stored securely behind Home Assistant authentication in /media. LLM Vision will attempt to migrate your previously captured snapshots to the new folder. (by @valentinfrlch)
• System Prompts: System prompts are now injected as actual system messages for every provider, which should improve their effectiveness. Also try out the updated default system and title prompts. (by @valentinfrlch)
• max_tokens now optional: max_tokens is now an optional parameter. Newer models use tokens for reasoning, which could result in empty responses if max_tokens was set too low. (by @valentinfrlch)

Blueprint

Make sure to update the blueprint by re-importing it!

A huge thank you to @TheRealFalseReality for maintaining the blueprint!

🔧 Improvements & Fixes

• New media folder: Updated all key_frame paths to match new media path. (by @TheRealFalseReality)

New Providers, Bug Fixes & Improvements

1.5.1 Release Notes

⚠️ Please read the breaking changes before updating!

This update adds native support for OpenRouter and Azure, fixes an issue with Google Gemini, adds support for GPT-5 models and adds Dutch and Polish translations.

Contributors

A huge thank you to our contributors @TheRealFalseReality, @NikNikovsky, @Minionguyjpro, @meceonzo—and of course everyone who helped test and provided feedback for the beta versions!

⚠️ Breaking Changes

• Timeline snapshots have moved from /www to /media. add the following to your configuration.yaml:

  # Add llmvision /media folder
  homeassistant:
    media_dirs:
      media: /media
      llmvision: /config/media/llmvision

  💡 Make sure to create the /media and the /llmvision inside it before restarting!
  💡 Update LLM Vision Card to the latest version! Otherwise snapshots won't show correctly.

• max_tokens is now displayed as a box type number selector rather than a slider. This change accommodates newer models that use thinking parameters counting toward max_tokens. Unlike the limited slider, the box allows for inputting much larger numbers.

  💡If you notice empty responses, consider increasing max_tokens.

Integration

✨ Features

• New Provider: LLM Vision now supports OpenRouter natively. (@valentinfrlch)
• New Provider: LLM Vision now supports Azure natively (#64, #103, #144). (@valentinfrlch)
• Support for Polish language: Added 🇵🇱 Polish translations. Thank you @NikNikovsky!
• Support for Dutch language: Added 🇳🇱 Dutch translations. Thank you @Minionguyjpro!

🔧 Improvements & Fixes

• Google Gemini fix: Fixed an infinite retry loop in the Google Gemini provider and corrected an issue where the fallback provider wasn't being used properly. (#398, #262), (@valentinfrlch)
• Support for GPT-5 models: Remove temperature and top_p from request when a gpt-5 model is used (#437), (@valentinfrlch)
• Moved Timeline snapshots: Snapshots are now stored securely behind Home Assistant authentication in /media. LLM Vision will attempt to migrate previously captured snapshots to the new folder.
• Remove max_tokens limit: max_tokens is now displayed as a box type number selector rather than a slider. This change allows for larger token limits, as thinking tokens count toward the max_tokens limit. (@valentinfrlch)
• Gemini key logging: Fixed a security issue that logged the Google API key in clear text (#334), (@valentinfrlch)
• Missing import: Add aiofile package to imports (#404), (@valentinfrlch)
• Sanitize IP address: Ignore any protocol input for the IP address field for Ollama and Open WebUI providers. Thank you @meceonzo!
• Fallback provider: Fixed a retry logic bug that caused an exception when no fallback provider was configured. (#461)
• Title generation: Fixed an issue that caused an exception when generate_title was not set. (#463)

Blueprint

Update the blueprint by re-importing it from the Blueprint settings page in Home Assistant.

https://github.com/valentinfrlch/ha-llmvision/blob/main/blueprints/event_summary.yaml

A huge thank you to @TheRealFalseReality for maintaining the blueprint!

✨ Features

• See snapshot quick action: Notification now shows button to preview the snapshot in a browser.

🔧 Improvements & Fixes

• Multiple Cameras: Fixed an issue that displayed the wrong camera (#403)
• Live Preview notification (iOS only): Preview mode can now be customized before and after analysis.
• Time format: Remove leading zero from 12-hour format.
• New media folder: Changed snapshot location from /www to /media/llmvision/snapshots (#457)

Improved Setup and Provider Settings

v1.5.0 Release Notes

This update improves the setup process, adds a fallback provider option, support for urls and other path types in video analyzer, introduces advanced settings for Ollama, and fixes an issue with Google Gemini.

Contributors

A huge thank you to our contributors @t3hk0d3, @TheRealFalseReality, @ond000, @natedev, @sb-develop—and of course everyone who helped test and provided feedback for the beta versions!

Integration

⚠️ Breaking Changes

• Timeline and Memory have been merged: The new Settings Config Entry unifies Timeline, Prompt and Memory (Beta) Settings as well as a new Fallback Provider option. If the Timeline provider still exists after this update, you may safely delete it. You may need to change the entity id of timeline entity (now associated with the Settings config entry).
• Temperature has moved: The temperature settings has moved to the provider settings. It can now longer be set individually for each call and is instead set for each provider. You can create multiple instances of the same provider if you need different temperature settings.

✨ Features

• New setup and reconfigure process: It is now easier to configure new providers as well as change settings for existing providers. The setup flow also allows for more provider specific settings. (@valentinfrlch)
• New Fallback provider option: You can now choose a fallback provider if the call fails with the selected provider. The request will be sent to the default model of the selected fallback provider. (@valentinfrlch)
• Support for additional paths: video_analyzer now supports URLs, media-source:// and relative /api/frigate paths. Thank you @t3hk0d3!
• New advanced settings for Ollama: You can now change the Context window length and “Keep alive” settings for Ollama. (#212)(@valentinfrlch)
• Added Top P: You can now control Top P (nucleus sampling) for all providers. This allows for more precise control over the randomness of generated responses. (@valentinfrlch)
• Support for Czech language: Added 🇨🇿 Czech translations. Thank you @ond000!

🔧 Improvements & Fixes

• Improved Image Pipeline efficiency: The image pipeline can now process multiple files or streams at once. Thank you @t3hk0d3!
• Google Gemini fix: Implemented exponential backoff retry logic for Google. Thank you @natedev! (#262)
• Update OpenAI Provider: Rename max_tokens to max_output_tokens for compatibility with newer o-series models. (#383)
• Custom OpenAI Provider: Use specified model for validation (#366)

Blueprint

• Fixed File Path default selector
• Fixed Custom Notification Channel default selector
• Added Time to notification title with the option of 12/24 hour format
• Fixed group/channel components with multiple camera entities

Thank you @TheRealFalseReality!

Improved Blueprint & Bug fixes

1.4.3 Release Notes

Warning

You may need to reconfigure certain fields of the blueprint after updating to v1.4.3.

Tip

The blueprint requires a separate update. You can update it by re-importing
(Settings → Automations & Scenes → Blueprints → three-dot menu → Re-import blueprint)

Blueprint

@TheRealFalseReality has completely rewritten the blueprint and enhanced it with lots of additional features:

• Added collapsible sections to the blueprint to make it easier to set up. Thank you @TheRealFalseReality!
• Added notification settings. Thank you @TheRealFalseReality!
  • notify_condition A list of conditions that need to be met to send the notification. The blueprint can still run and save events to the timeline but skips sending the notification. (Different from run_conditions which prevent the blueprint from running all-together).
  • delay_notification A separate cooldown for notifications (blueprint can still run, but doesn’t send a notification again. Separate from cooldown).
  • notification_sticky (Android only) make the notification stay active until dismissed.
  • notification_color (Android only) Customize the color of the notification
  • notification_icon (Android only) Customize the icon of the notification
  • notification_channel (Android only) Custom notification channel for custom notification sounds and vibration patterns. Allows overriding “Do Not Disturb”.
  • tts_notifications (Android only) Use text-to-speech to speak the notification.
• Added additional_actions (Experimental) Run custom actions after the blueprint.

Integration

• Added 🇸🇰 Slovak translation. Thank you @jose1711!
• Updated Groq default model (previous model is deprecated) (#311)
• Fixed a bug in the timeline that deleted events when retention_time was set to 0. (#309)