[Beta Support]: MQTT Snapshot, No Alert or Event in UI

[rogerquake]

Describe the problem you are having

I had previously tried the yolov9 Frigate+ model a few months ago, however due to too many false positives, I switched back to yolonas. With the release of the updated model this year, I uploaded a bunch of new images and generated a custom Yolov9 to try out on Frigate 17 Beta 2.

What I'm experiencing is that a false positive is occurring and the snapshot is being sent over MQTT and appearing inside the Home Assistant integration. However the event is not appearing anywhere in the Frigate UI so I could submit it as a false positive.

Since this happened a few days ago, I thought it was a fluke. To try and trigger a false positive again, I lowered by thresholds way down to around 50. As you can see from the screenshots, there was no event listed when the false positive occurred at 4:34PM.

Beta Version

0.17.0-93016c6

Issue Category

Object Detection / Detectors

Frigate config file

host: 192.168.4.X
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: username
  password: password

logger:
  logs:
    frigate.record: debug
    frigate.go2rtc: debug

detectors:
  # Required: name of the detector
  onnx:
    type: onnx

go2rtc:
  streams:
    driveway-west:
      - rtsp://viewer:password@192.168.4.X:554/Streaming/Channels/101
    driveway-west-mobile:
      - rtsp://viewer:password@192.168.4.X:554/Streaming/Channels/102
    driveway-west-detect:
      - rtsp://viewer:password@192.168.4.X:554/Streaming/Channels/103


ffmpeg:
  hwaccel_args: preset-nvidia-h264

genai:
  provider: ollama
  base_url: http://192.168.4.x:7869
  model: blaifa/InternVL3_5:4b
  provider_options:  # other Ollama client options can be defined
    keep_alive: -1
    options:
      num_ctx: 4192

audio_transcription:
  # Optional: Enable live and speech event audio transcription (default: shown below)
  device: GPU
  # Optional: Set the model size used for live transcription. (default: shown below)
  model_size: large
  # Optional: Set the language used for transcription translation. (default: shown below)
  # List of language codes: https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L10
  language: en

face_recognition:
  enabled: true

  model_size: large
review:
  # Optional: alerts configuration
  alerts:
    # Optional: enables alerts for the camera (default: shown below)
    enabled: true
    # Optional: labels that qualify as an alert (default: shown below)
    labels:
      - car
      - person
    # Time to cutoff alerts after no alert-causing activity has occurred (default: shown below)
    cutoff_time: 40
    # Optional: required zones for an object to be marked as an alert (default: none)
    # NOTE: when settings required zones globally, this zone must exist on all cameras
    #       or the config will be considered invalid. In that case the required_zones
    #       should be configured at the camera level.

  # Optional: detections configuration
  detections:
    # Optional: enables detections for the camera (default: shown below)
    enabled: true
    # Optional: labels that qualify as a detection (default: all labels that are tracked / listened to)
    labels:
      - car
      - person
    # Time to cutoff detections after no detection-causing activity has occurred (default: shown below)
    cutoff_time: 30
    # Optional: required zones for an object to be marked as a detection (default: none)
    # NOTE: when settings required zones globally, this zone must exist on all cameras
    #       or the config will be considered invalid. In that case the required_zones
    #       should be configured at the camera level.
  # Optional: GenAI Review Summary Configuration
  genai:
    # Optional: Enable the GenAI review summary feature (default: shown below)
    enabled: true
    # Optional: Enable GenAI review summaries for alerts (default: shown below)
    alerts: true
    # Optional: Enable GenAI review summaries for detections (default: shown below)
    detections: true


detect:
  # Optional: width of the frame for the input with the detect role (default: shown below)
  width: 1920
  # Optional: height of the frame for the input with the detect role (default: shown below)
  height: 1080
  # Optional: desired fps for your camera for the input with the detect role (default: shown below)
  # NOTE: Recommended value of 5. Ideally, try and reduce your FPS on the camera.
  fps: 6
  # Optional: enables detection for the camera (default: True)
  # This value can be set via MQTT and will be updated in startup based on retained value
  enabled: true
  # Optional: Number of frames without a detection before frigate considers an object to be gone. (default: 5x the frame rate)
  max_disappeared: 25

  # Optional: Configuration for stationary object tracking
  stationary:
    # Optional: Frequency for running detection on stationary objects (default: shown below)
    # When set to 0, object detection will never be run on stationary objects. If set to 10,
    # it will be run on every 10th frame.
    interval: 10
    # Optional: Number of frames without a position change for an object to be considered
    # stationary (default: 10x the frame rate or 10s)
    threshold: 50
    # Optional: Define a maximum number of frames for tracking a stationary object (default: not set, track forever)
    # This can help with false positives for objects that should only be stationary for a limited amount of time.
    # It can also be used to disable stationary object tracking. For example, you may want to set a value for person,
    # but leave car at the default.
    max_frames:
      # Optional: Default for all object types (default: not set, track forever)
      # Optional: Object specific values
      objects:
        person: 1000

  # Optional: Milliseconds to offset detect annotations by (default: shown below).
  annotation_offset: 0

# Optional: Global object filters for all cameras.
# NOTE: can be overridden at the camera level
objects:

  genai:
    enabled: true
  track:
    - person
    - dog
  # Optional: filters to reduce false positives for specific object types
  filters:
    person:
      # Optional: minimum width*height of the bounding box for the detected object (default: 0)
      #min_area: 1000
      # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
      #max_area: 400000
      # Optional: minimum score for the object to initiate tracking (default: shown below)
      min_score: 0.50
      # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
      threshold: 0.55
    dog:
      # Optional: minimum width*height of the bounding box for the detected object (default: 0)
      #min_area: 1000
      # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
      #max_area: 400000
      # Optional: minimum score for the object to initiate tracking (default: shown below)
      min_score: 0.50
      # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
      #threshold: 0.90
motion:
  # Optional: The threshold passed to cv2.threshold to determine if a pixel is different enough to be counted as motion. (default: shown below)
  # Increasing this value will make motion detection less sensitive and decreasing it will make motion detection more sensitive.
  # The value should be between 1 and 255.
  threshold: 15
  # Optional: Minimum size in pixels in the resized motion image that counts as motion (default: 30)
  # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
  # make motion detection more sensitive to smaller moving objects.
  # As a rule of thumb:
  #  - 15 - high sensitivity
  #  - 30 - medium sensitivity
  #  - 50 - low sensitivity
  contour_area: 30
  # Optional: Alpha value passed to cv2.accumulateWeighted when averaging the motion delta across multiple frames (default: shown below)
  # Higher values mean the current frame impacts the delta a lot, and a single raindrop may register as motion.
  # Too low and a fast moving person wont be detected as motion.
  delta_alpha: 0.2
  # Optional: Alpha value passed to cv2.accumulateWeighted when averaging frames to determine the background (default: shown below)
  # Higher values mean the current frame impacts the average a lot, and a new object will be averaged into the background faster.
  # Low values will cause things like moving shadows to be detected as motion for longer.
  # https://www.geeksforgeeks.org/background-subtraction-in-an-image-using-concept-of-running-average/
  frame_alpha: 0.2
  # Optional: Height of the resized motion frame  (default: 50)
  # This operates as an efficient blur alternative. Higher values will result in more granular motion detection at the expense
  # of higher CPU usage. Lower values result in less CPU, but small changes may not register as motion.
  frame_height: 100

record:
  # Optional: Enable recording (default: shown below)
  enabled: true
  sync_recordings: true
  # Optional: Retention settings for recording
  alerts:
    retain:
      days: 18
    pre_capture: 20
    post_capture: 20
  detections:
    retain:
      days: 18
    pre_capture: 20
    post_capture: 20
  continuous:
    days: 3
  motion:
    days: 3
snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  # This value can be set via MQTT and will be updated in startup based on retained value
  enabled: true
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: false
  # Optional: draw bounding box on the snapshots (default: shown below)
  bounding_box: false
  # Optional: crop the snapshot (default: shown below)
  crop: true
  # Optional: height to resize the snapshot to (default: original size)
  #height: 175
  # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
  required_zones: []
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 18
    # Optional: Per object retention days
    objects:
      person: 18

live:
  # Optional: Set the height of the live stream. (default: 720)
  # This must be less than or equal to the height of the detect stream. Lower resolutions
  # reduce bandwidth required for viewing the live stream. Width is computed to match known aspect ratio.
  height: 720
  # Optional: Set the encode quality of the live stream (default: shown below)
  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
  quality: 8

cameras:

driveway-west:
    ffmpeg:
      input_args: preset-rtsp-restream
      output_args:
        record: preset-record-generic-audio-copy
      # -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac
      inputs:
        - path: rtsp://127.0.0.1:8554/driveway-west-detect
          roles:
            - detect

        - path: rtsp://127.0.0.1:8554/driveway-west
          roles:
            - record

    detect:
      width: 1920
      height: 1080
      fps: 6


    best_image_timeout: 3
    objects:
      track:
        - person
        - car
        - motorcycle
        - dog


      mask: 
        0.284,0.221,0.763,0.222,0.821,0.368,1,0.433,1,0,0,0,0,0.158,0,0.356,0.274,0.284
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: true
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: false
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: false
      # Optional: crop the snapshot (default: shown below)
      crop: true
      # Optional: height to resize the snapshot to (default: shown below)
      height: 1080
      # Optional: jpeg encode quality (default: shown below)
      quality: 100

    motion:
      mask: 
        0,0.345,0.267,0.277,0.285,0.186,0.759,0.193,0.831,0.374,1,0.437,1,0.215,1,0,0,0,0,0.121
      threshold: 20
      contour_area: 30
      improve_contrast: true
    zones:
      Driveway:
        coordinates: 
          0,1,1,1,1,0.443,0.821,0.374,0.757,0.188,0.361,0.177,0.283,0.174,0.269,0.26,0.141,0.286,0,0.314
        loitering_time: 0
        inertia: 3
    review:
      alerts: {}
      detections: {}
  front-door:
    ffmpeg:
      input_args: preset-rtsp-restream
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/front-door-detect
          roles:
            - detect

        - path: rtsp://127.0.0.1:8554/front-door
          roles:
            - record

    detect:
      width: 1920
      height: 1080
      fps: 6

    objects:
      track:
        - person
        - package

      mask: 0,0,0,0.37,0.273,0.331,0.573,0.322,0.588,0,0.302,0

    best_image_timeout: 3

    motion:
      mask: 0.166,0.149,0.161,0,0,0,0,0.204,0.084,0.178,0.096,0.175,0.109,0.169




    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: true
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: false
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: false
      # Optional: crop the snapshot (default: shown below)
      crop: true
      # Optional: height to resize the snapshot to (default: shown below)
      height: 1080
      # Optional: jpeg encode quality (default: shown below)
      quality: 100
      required_zones: []
    review:
      alerts: {}
      detections: {}

Relevant Frigate log output

No logs

Relevant go2rtc log output (if applicable)

No response

Install method

Home Assistant Add-on

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:0.17.0-beta2-tensorrt
    shm_size: "40G" # update for your cameras based on calculation above
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all # or specify a specific GPU ID
              capabilities: [gpu]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility,video
      - FRIGATE_RTSP_PASSWORD=password
      - PLUS_API_KEY=KEY
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/roger/docker-compose/frigate:/config
      - /frigate_recordings:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 28000000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
      - "1984:1984/tcp" # WebRTC over

Operating system

Home Assistant OS

CPU / GPU / Hardware

No response

Screenshots

Steps to reproduce

No response

Any other information that may be helpful

No response

[hawkeye217]

See the FAQ in the official Frigate docs about the Home Assistant sensors and MQTT snapshots.

[rogerquake]

Are there plans to build sensors into the integration that use events? I remember having to create mqtt based sensors when I first started to use frigate, but thought those days were behind me. Just seems like a lot of extra work for a common requirement.

[hawkeye217]

Not for events because filtering requirements are different for everyone, but it already exists for reviews because you can determine what is an alert or a detection directly through Frigate. See blakeblackshear/frigate-hass-integration#1023