Skip to content

Commit 3b1ff63

Browse files
Merge pull request #396 from valentinfrlch/v1.5.0-beta
v1.5.0 Changes
2 parents d0adc2a + c07df7d commit 3b1ff63

28 files changed

+4585
-1361
lines changed

README.md

Lines changed: 26 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
<p align="center">
2-
<img src="https://github.com/user-attachments/assets/bebd92b8-765e-4d63-bb3d-47e1bb8b51ad" width=500px>
2+
<picture>
3+
<source media="(prefers-color-scheme: dark)" srcset="./logos/dark_logo@2x.png">
4+
<img alt="LLM Vision Logo" src="./logos/logo@2x.png" width="512">
5+
</picture>
36
</p>
47
<p align=center>
58
<img src=https://img.shields.io/badge/HACS-Default-orange.svg>
69
<img src="https://img.shields.io/maintenance/yes/2025.svg">
7-
<img src=https://img.shields.io/badge/version-1.4.3-blue>
10+
<img src=https://img.shields.io/badge/version-1.5.0-blue>
811
<img alt="Issues" src="https://img.shields.io/github/issues/valentinfrlch/ha-llmvision?color=0088ff">
912
<img alt="Static Badge" src="https://img.shields.io/badge/support-buymeacoffee?logo=buymeacoffee&logoColor=black&color=%23FFDD00&link=https%3A%2F%2Fbuymeacoffee.com%2Fvalentinfrlch">
1013
<p align=center style="font-weight:bold">
@@ -15,9 +18,9 @@
1518
<p align="center">
1619
<a href="#features">🌟 Features </a>
1720
·
18-
<a href="#resources">📖 Resources</a>
21+
<a href="#quick-start-guide">⬇️ Quick Start Guide</a>
1922
·
20-
<a href="#installation">⬇️ Installation</a>
23+
<a href="#resources">📖 Resources</a>
2124
·
2225
<a href="#how-to-report-a-bug-or-request-a-feature">🪲 How to report Bugs</a>
2326
·
@@ -33,21 +36,35 @@
3336
</p>
3437

3538
## Features
36-
- Compatible with OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, Groq, [LocalAI](https://github.com/mudler/LocalAI), [Ollama](https://ollama.com/), [Open WebUI](https://github.com/open-webui/open-webui) and providers with OpenAI compatible enpoints.
39+
- Compatible with OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, Groq, [Ollama](https://ollama.com/), [Open WebUI](https://github.com/open-webui/open-webui), [LocalAI](https://github.com/mudler/LocalAI) and providers with OpenAI compatible enpoints.
3740
- Analyzes images and video files, live camera feeds and Frigate events
38-
- Can remembers people, pets and objects
39-
- Maintains a timeline of camera events, so you can display them on your dashboard as well as ask about them later
40-
- Seamlessly updates sensors based on image input
41+
- Remembers people, pets and objects
42+
- Maintains a timeline of camera events, so you can display them on your dashboard and ask Assist about them
43+
- Seamlessly updates sensors based on data extracted from camera streams, images or videos
4144

4245
<br>
4346

4447
See the [website](https://llmvision.org) for the latest features as well as examples.
45-
<img width="1122" alt="bento-board" src="https://github.com/user-attachments/assets/a87091a4-314e-42e3-b068-51e8d6a97f1b" />
48+
<img width="1027" alt="Bentoboard" src="https://github.com/user-attachments/assets/68af19b0-7000-4e5f-a3d3-c180ec2905f2" />
49+
4650

4751

4852

4953
<br>
5054

55+
## Quick Start Guide
56+
>[!TIP]
57+
>LLM Vision is available in the default HACS repository. You can install it directly through HACS or click the button below to open it there.
58+
59+
[![Open a repository inside the Home Assistant Community Store.](https://my.home-assistant.io/badges/hacs_repository.svg)](https://my.home-assistant.io/redirect/hacs_repository/?owner=valentinfrlch&repository=ha-llmvision&category=Integration)
60+
1. **Install `LLM Vision`** from HACS
61+
2. **Restart** Home Assistant
62+
3. **Search for `LLM Vision`** in Home Assistant Settings/Devices & services
63+
4. **Press submit** to continue setup with default settings
64+
5. **Press 'Add Entry'** to add your first AI Provider
65+
66+
Detailed setup instructions and documentation is available here: [LLM Vision Documentation](https://llm-vision.gitbook.io/getting-started/setup/providers)
67+
5168
## Blueprint
5269
With the easy to use blueprint, you'll get camera event notifications intelligently summarized by AI. LLM Vision can also store events in a timeline, so you can see what happened on your dashboard.
5370
<br>
@@ -67,18 +84,6 @@ Check the docs for detailed instructions on how to set up LLM Vision and each of
6784
For technical questions see the discussions tab.
6885

6986

70-
## Installation
71-
>[!TIP]
72-
>LLM Vision is available in the default HACS repository. You can install it directly through HACS or click the button below to open it there.
73-
74-
[![Open a repository inside the Home Assistant Community Store.](https://my.home-assistant.io/badges/hacs_repository.svg)](https://my.home-assistant.io/redirect/hacs_repository/?owner=valentinfrlch&repository=ha-llmvision&category=Integration)
75-
1. Install `LLM Vision` from HACS
76-
2. Search for `LLM Vision` in Home Assistant Settings/Devices & services
77-
3. Select your provider
78-
4. Follow the instructions to add your AI providers.
79-
80-
Continue with setup here: https://llm-vision.gitbook.io/getting-started/setup/providers
81-
8287
## How to report a bug or request a feature
8388
> [!IMPORTANT]
8489
> **Bugs:** If you encounter any bugs and have followed the instructions carefully, file a bug report. Please check open issues first and include debug logs in your report. Debugging can be enabled on the integration's settings page.

blueprints/event_summary.yaml

Lines changed: 60 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
1+
# Fix Group and Channel to First Camera Entity, user edits - DONE
2+
# Add time to notificaion title or message, with user choose between 12/24h - DONE
3+
# Take snapshot of triggered camera entity if using multiple cameras (? help needed)
4+
# Snooze blueprint or notifications
15
blueprint:
2-
name: AI Event Summary (v1.4.3)
6+
name: AI Event Summary (v1.5.0)
37
author: valentinfrlch
8+
homeassistant:
9+
min_version: 2024.10.0
410
description: >
511
AI-powered summaries for security camera events.
612
Sends a notification with a preview to your phone that is updated dynamically when the AI summary is available.
@@ -75,9 +81,9 @@ blueprint:
7581
collapsed: true
7682
input:
7783
remember:
78-
name: Remember
84+
name: Store in Timeline
7985
description: Stores this event in the Timeline so you can ask about it. If important is enabled, only events classified as Normal or Critical will be saved.
80-
default: false
86+
default: true
8187
selector:
8288
boolean:
8389
use_memory:
@@ -120,16 +126,7 @@ blueprint:
120126
selector:
121127
number:
122128
min: 1
123-
max: 100
124-
temperature:
125-
name: Temperature
126-
description: Randomness. Lower is more accurate, higher is more creative.
127-
default: 0.1
128-
selector:
129-
number:
130-
min: 0.1
131-
max: 1.0
132-
step: 0.1
129+
max: 300
133130
# Notificaion Section
134131
notification_section:
135132
name: Notification Settings
@@ -171,26 +168,39 @@ blueprint:
171168
- Consolidated
172169
preview_mode:
173170
name: Preview Mode
174-
description: "Choose between a live preview or a snapshot of the event. \n\n **Important:** Live Preview is only supported on iOS."
171+
description: "Choose between a live preview or a snapshot of the event. The Snapshot of the last camera, if using multiple Camera Entities, will be used but the notification will be udated with the camera view that triggered the automation. \n\n **Important:** Live Preview is only supported on iOS."
175172
default: Snapshot
176173
selector:
177174
select:
178175
options:
179176
- Snapshot
180177
- Live Preview
178+
notification_time:
179+
name: Notification Time
180+
description: "Add time to Notification Title and choose between 12-hour or 24-hour time format for the notification title."
181+
default: ''
182+
selector:
183+
select:
184+
options:
185+
- label: No Time Added
186+
value: ''
187+
- label: 12 Hour
188+
value: 'at {{ now().strftime("%I:%M %p") }}'
189+
- label: 24 Hour
190+
value: 'at {{ now().strftime("%H:%M") }}'
181191
file_path:
182192
name: File Path - To send snapshot in Dynamic Mode with Snapshot Preview Mode
183-
description: "The file path to store the most current snapshot for the FIRST camera if using multiple camera entities.\n\nDefaults
184-
to `/media/local/snapshots/{{ camera_file_path }}/last_motion.jpg` that references **/media/snapshots/<CAMERA_NAME>/last_motion.jpg** \n\n Try `/local/snapshots/{{ camera_file_path }}/last_motion.jpg` that references **/config/www/snapshots/{{ camera_file_path }}/last_motion.jpg** if you are having issues ***Note that this is unsecured and exposes the images to the web.***
185-
\n\nYOU MUST ADD `/local` with the `/media` folder so -> `/media/local` or replace `/config/www` with `/local`."
186-
default: Media Folder - Secured
193+
description: "The file path to store the most current snapshot for the FIRST camera if using multiple camera entities. You can specifiy a custom location as well.\n\nDefaults
194+
to `Media Folder - Secured` that references **/media/snapshots/<CAMERA_NAME>/last_motion.jpg** \n\n Try the `Public Folder - Unsecured` option that references **/config/www/snapshots/{{ camera_file_path }}/last_motion.jpg** if you are having issues ***Note that this is unsecured and exposes the images to the web.***"
195+
default: '/media/snapshots/{{ camera_file_path }}/last_motion.jpg'
187196
selector:
188197
select:
189198
options:
190199
- label: Media Folder - Secured
191-
value: /media/local/snapshots/{{ camera_file_path }}/last_motion.jpg
200+
value: '/media/snapshots/{{ camera_file_path }}/last_motion.jpg'
201+
192202
- label: Public Folder - Unsecured
193-
value: /config/www/snapshots/{{ camera_file_path }}/last_motion.jpg
203+
value: '/config/www/snapshots/{{ camera_file_path }}/last_motion.jpg'
194204
custom_value: true
195205
tap_navigate:
196206
name: Tap Navigate
@@ -290,19 +300,20 @@ blueprint:
290300
notification_channel:
291301
name: Custom Notification Channel or Alarm Mode - Android Only
292302
description: Create a new channel for notifications to allow custom notification
293-
sounds, vibration patterns, and override Do Not Disturb mode. Configured
303+
sounds, vibration patterns, and override Do Not Disturb mode. Camera Name uses first Camera Entity if using multiple cameras. Configured
294304
directly on the Android device -> Home Assistant App Setting -> Notifications.
295-
Use `Alarm` to use the device's loud alarm notification sound.
305+
Use `Alarm` to use the device's loud alarm notification sound. You can also type in your own channel name.
296306
[Learn More](https://companion.home-assistant.io/docs/notifications/notification-commands/#volume-level)
297-
(default = Camera Name == {{ camera }} Snapshot).
307+
(default = Camera Name == {{ camera_entity_snapshot }} Snapshot).
298308
default: '{{ camera }} Snapshot'
299309
selector:
300310
select:
301311
options:
302312
- label: Camera Name
303-
value: '{{ camera }} snapshot'
313+
value: '{{ camera }} Snapshot'
304314
- label: Alarm
305315
value: alarm_stream
316+
custom_value: true
306317
# TTS Notification
307318
tts_notification:
308319
name: Use Text-to-Speech (TTS) on your device to speak the notification. Android Only
@@ -312,8 +323,8 @@ blueprint:
312323
boolean:
313324
tts_volume:
314325
name: TTS Volume
315-
description: Set TTS volume to device's current alarm volume or Max volume. Android Only
316-
default: 'alarm_stream'
326+
description: Set TTS volume to device's current alarm volume or Max volume. ALARM VOLUME WILL IGNORE DO NOT DISTURB! Android Only
327+
default: 'notification_stream'
317328
selector:
318329
select:
319330
options:
@@ -416,6 +427,8 @@ variables:
416427
notify: !input notify
417428
condition_notify: !input condition_notify
418429
delay_notification: !input delay_notification
430+
# camera_snapshot: !input camera_snapshot
431+
# camera_name: '{{ states[camera_snapshot].name.replace(" ", "_") }}'
419432
device_name_map: >
420433
{% set ns = namespace(device_names=[]) %}
421434
{% for device_id in notify_devices %}
@@ -433,20 +446,21 @@ variables:
433446
{% else %}
434447
{{ trigger.entity_id }}
435448
{% endif %}
436-
camera_entity_snapshot: >
437-
{{ camera_entities_list[0] }}
438449
tag: >
439450
{{ camera_entity + int(as_timestamp(now()))|string }}
440-
group: >
441-
{{ camera_entity }}
442451
label: Motion detected
443452
camera: >
444-
{{ camera_entity.replace("camera.", "").replace("_", " ")|capitalize }}
453+
{{ camera_entities_list[0].replace("camera.", "").replace("_", " ") | title }}
445454
importance_prompt: >
446455
Classify the security event based on this image. Choose from the following options: "passive" for unimportant events, "time-sensitive" for notable but non-critical events such as a person at the front door, and "critical" only for potential burglaries or highly suspicious activity. Respond with one of these options exactly, without additional explanation.
456+
camera_entity_snapshot: >
457+
{{ camera_entities_list[0] }}
447458
camera_file_path: >
448459
{{ camera_entity_snapshot.replace("camera.", "")}}
449460
file_path: !input file_path
461+
snapshot_access_file_path: '{{ file_path | replace(''/config/www'',
462+
''/local'') | replace(''/media'', ''/media/local'') }}'
463+
notification_time: !input notification_time
450464

451465
max_exceeded: silent
452466
mode: single
@@ -481,7 +495,6 @@ action:
481495
include_filename: true
482496
target_width: 1280
483497
max_tokens: 3
484-
temperature: 0.1
485498
response_variable: importance
486499

487500
# Cancel automation if event not deemed important
@@ -516,26 +529,25 @@ action:
516529
value_template: "{{ preview_mode=='Snapshot' }}"
517530
sequence:
518531

519-
# Save Camera Snapshot to file to send to Android devices for Dynamic Mode, perhaps make optional?
532+
# Save Camera Snapshot to file to send to Android devices for Dynamic Mode
520533
- service: camera.snapshot
521-
target:
522-
entity_id: !input camera_entities
534+
entity_id: !input camera_entities
523535
data:
524536
filename: !input file_path
525537

526-
- alias: "Send instant notification to notify devices"
538+
- alias: "Send instant notification snapshot to notify devices"
527539
repeat:
528540
for_each: "{{device_name_map}}"
529541
sequence:
530542
- action: "notify.{{ repeat.item }}"
531543
data:
532-
title: "{{ label }}"
544+
title: "{{ label }} {{ notification_time }}"
533545
message: "{{camera}} has detected activity."
534546
data:
535547
url: !input tap_navigate #iOS
536548
clickAction: !input tap_navigate #Android
537549
tag: "{{tag}}"
538-
group: "{{group}}"
550+
group: !input notification_channel
539551
alert_once: true
540552
# Priority options
541553
ttl: 0
@@ -545,9 +557,11 @@ action:
545557
color: !input notification_color
546558
notification_icon: !input notification_icon
547559
sticky: !input notification_sticky
548-
# Send a snapshot of the event to Android devices
549-
image: "{{ file_path }}"
560+
image: "{{ snapshot_access_file_path }}"
550561
# iOS only
562+
attachment:
563+
url: '{{ snapshot_access_file_path }}'
564+
content_type: JPEG
551565
push:
552566
interruption-level: "{{importance.response_text|lower if importance is defined else 'active'}}"
553567
sound:
@@ -566,13 +580,13 @@ action:
566580
sequence:
567581
- action: "notify.{{ repeat.item }}"
568582
data:
569-
title: "{{ label }}"
583+
title: "{{ label }} {{ notification_time }}"
570584
message: "{{camera}} has detected activity."
571585
data:
572586
entity_id: "{{camera_entity}}"
573587
url: !input tap_navigate
574588
tag: "{{tag}}"
575-
group: "{{group}}"
589+
group: !input notification_channel
576590
alert_once: true
577591
push:
578592
interruption-level: "{{importance.response_text|lower if importance is defined else 'active'}}"
@@ -600,7 +614,6 @@ action:
600614
max_frames: !input max_frames
601615
target_width: !input target_width
602616
max_tokens: !input max_tokens
603-
temperature: !input temperature
604617
response_variable: response
605618

606619
- alias: "Update label with title"
@@ -626,14 +639,14 @@ action:
626639
sequence:
627640
- action: "notify.{{ repeat.item }}"
628641
data:
629-
title: "{{ label }}"
642+
title: "{{ label }} {{ notification_time }}"
630643
message: "{{response.response_text}}"
631644
data:
632645
image: "{{response.key_frame.replace('/config/www/','/local/') }}"
633646
url: !input tap_navigate #iOS
634647
clickAction: !input tap_navigate #Android
635648
tag: "{{tag}}"
636-
group: "{{group}}"
649+
group: !input notification_channel
637650
alert_once: true
638651
# Priority options
639652
ttl: 0
@@ -675,4 +688,5 @@ action:
675688
- choose: []
676689
default: !input additional_actions
677690

678-
- delay: !input cooldown
691+
# - delay: '00:{{cooldown|int}}:00'
692+
- delay: !input cooldown

0 commit comments

Comments
 (0)