[Beta Support]: v0.18 GenAI chat bug - An error occurred while processing your request

[InstigatorX]

Describe the problem you are having

Running latest commit (dc27d4a) of v0.18 and getting "Gemini returned an error during chat_with_tools_stream: 'async for' requires an object with aiter method, got generator" when using the new chat and selecting "show recent events".

Apologies if this is too early to report issues.

Beta Version

0.18.0-dc27d4a

Issue Category

Other

Frigate config file

version: 0.18-0

mqtt:
  enabled: false

detectors:
  ov_0:
    type: openvino
    device: GPU

motion:
  enabled: false

semantic_search:
  enabled: true
  reindex: false
  model_size: large

snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  enabled: true
  # Optional: save a clean PNG copy of the snapshot image (default: shown below)
  clean_copy: true
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: false
  # Optional: draw bounding box on the snapshots (default: shown below)
  bounding_box: true
  # Optional: crop the snapshot (default: shown below)
  crop: false
  # Optional: height to resize the snapshot to (default: original size)
  #height: 175
  # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
  #required_zones: []
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 30
    # Optional: Per object retention days
    objects:
      person: 60
      car: 60
  # Optional: quality of the encoded jpeg, 0-100 (default: shown below)
  quality: 90

record:
  enabled: true
  #sync_recordings: true
  alerts:
    # Optional: Number of seconds before the alert to include (default: shown below)
    pre_capture: 10
    # Optional: Number of seconds after the alert to include (default: shown below)
    post_capture: 10
    # Optional: Retention settings for recordings of alerts
    retain:
      days: 5
      mode: all
  detections:
    # Optional: Number of seconds before the detection to include (default: shown below)
    pre_capture: 10
    # Optional: Number of seconds after the detection to include (default: shown below)
    post_capture: 10
    # Optional: Retention settings for recordings of detections
    retain:
      days: 5
      mode: motion
  continuous:
    days: 10
  motion:
    days: 10

detect:
  enabled: false
  #height: 360
  #width: 640
  #fps: 1

model:
  path: plus://xxx #yolov9s fine tune

genai:
  default:
    provider: gemini
    api_key: '{FRIGATE_GENAI_API_KEY}'
    model: gemini-3-flash-preview
    roles:
      - embeddings
      - vision
      - tools

review:
  genai:
    # Optional: Enable the GenAI review summary feature (default: shown below)
    enabled: true
    # Optional: Enable GenAI review summaries for alerts (default: shown below)
    alerts: true
    # Optional: Enable GenAI review summaries for detections (default: shown below)
    detections: true
    # Optional: Additional concerns that the GenAI should make note of (default: None)
    # additional_concerns:
      # - Animals in the garden
    # Optional: Preferred response language (default: English)
    preferred_language: English
    image_source: preview  # preview or recordings
  alerts:
    labels:
      - person
      - car
      - amazon
      - fedex
      - ups
      - usps
      - dhl
  detections:
    labels:
      - person
      - car
      - amazon
      - fedex
      - ups
      - usps
      - dhl
      - package
      - bear

objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person
    - face
    - license_plate
    - dog
    - cat
    - car
    - amazon
    - fedex
    - ups
    - usps
    - dhl
    - package
    - squirrel
    - rabbit
    - bird
    - deer
    - bicycle
    - motorcycle
    - bear
    - waste_bin
    - robot_lawnmower
  filters:
    # --- Standard Tracked Objects ---
    person:
      min_score: 0.7
      threshold: 0.85
    dog:
      min_score: 0.7
      threshold: 0.85
    cat:
      min_score: 0.7
      threshold: 0.85
    car:
      min_score: 0.7
      threshold: 0.85
    bicycle:
      min_score: 0.7
      threshold: 0.85
    motorcycle:
      min_score: 0.7
      threshold: 0.85
    package:
      min_score: 0.7
      threshold: 0.85
    waste_bin:
      min_score: 0.7
      threshold: 0.8
    robot_lawnmower:
      min_score: 0.75
      threshold: 0.85
    squirrel:
      min_score: 0.7
      threshold: 0.8
    rabbit:
      min_score: 0.7
      threshold: 0.8
    bird:
      min_score: 0.75
      threshold: 0.9
    deer:
      min_score: 0.7
      threshold: 0.85
    bear:
      min_score: 0.7
      threshold: 0.9
    # --- Attribute Labels (No Threshold) ---
    face:
      min_score: 0.8
    license_plate:
      min_score: 0.75
    amazon:
      min_score: 0.8
    fedex:
      min_score: 0.8
    ups:
      min_score: 0.8
    usps:
      min_score: 0.8
    dhl:
      min_score: 0.8

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264
  global_args: -hide_banner -loglevel warning
  input_args: preset-rtsp-restream
  output_args:
    record: preset-record-generic-audio-copy

cameras:
  gate:
    enabled: true
    detect:
      enabled: true
    motion:
      enabled: true
      mask:
        motion_mask_1:
          friendly_name: Motion Mask 1
          enabled: true
          coordinates: 
            0,0,0,0.501,0.319,0.614,0.475,0.638,0.466,0.486,1,0.485,1,0
    objects:
      genai:
        enabled: false
        use_snapshot: true
      track:
        - person
        - face
        - license_plate
        - dog
        - cat
        - car
        - amazon
        - fedex
        - ups
        - usps
        - dhl
        - package
        - squirrel
        - rabbit
        - bird
        - deer
        - bicycle
        - motorcycle
        - bear
        - waste_bin
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/gate_ext
          roles:
            - record
  lower_garage:
    enabled: true
    friendly_name: Lower Garage
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/lower_garage_ext
          roles:
            - record
  upper_side:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/upper_side_ext
          roles:
            - record
  upper_garage:
    enabled: true
    friendly_name: Upper Garage
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/upper_garage_ext
          roles:
            - record
  front_side:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/front_side_ext
          roles:
            - record
  front_door:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/front_door_ext
          roles:
            - record
  back_left:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/back_left_ext
          roles:
            - record
  back_right:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/back_right_ext
          roles:
            - record
  back_middle:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/back_middle_ext
          roles:
            - record
  foyer:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/foyer_ext
          roles:
            - record
  downstairs:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/downstairs_ext
          roles:
            - record
  sherry_garage:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/sherry_garage_ext
          roles:
            - record
  steve_garage:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/steve_garage_ext
          roles:
            - record
  laundry:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/laundry_ext
          roles:
            - record

logger:
  # Optional: Default log verbosity (default: shown below)
  default: info
  # Optional: Component specific logger overrides
  #logs:
  #  frigate.api.auth: debug
  #  frigate.event: debug
  #  frigate.embeddings.maintainer: debug
  #  frigate.data_processing.common.face.model: debug
  #  frigate.data_processing.real_time.face: debug

auth:
  enabled: false   # turn OFF Frigate’s own login system
  trusted_proxies:
    - 192.168.86.241

proxy:
  header_map:
    user: X-Forwarded-User
    role: X-Forwarded-Groups

tls:
  enabled: false   # since NPM is doing TLS termination

Relevant Frigate log output

Tag
frigate.genai.gemini
Message
Gemini returned an error during chat_with_tools_stream: 'async for' requires an object with __aiter__ method, got generator

Relevant go2rtc log output (if applicable)

No response

Install method

Proxmox via Docker

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    stop_grace_period: 30s # allow enough time to shut down the various services
    #image: ghcr.io/blakeblackshear/frigate:ef5608a-amd64
    image: ghcr.io/blakeblackshear/frigate:dc27d4a-amd64
    network_mode: host
    env_file:
      - ./frigate.env
    cap_add:
      - CAP_PERFMON
    shm_size: "1536mb" # update for your cameras based on calculation above
    devices:
      #- /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
      - /dev/dri:/dev/dri # For intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/frigate/custom-ffmpeg:/usr/lib/btbn-ffmpeg:ro
      - /etc/localtime:/etc/localtime:ro
      - /etc/frigate:/config
      - /mnt/frigate/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000

Operating system

Ubuntu

CPU / GPU / Hardware

No response

Screenshots

No response

Steps to reproduce

Updated v0.17 build to v0.18 latest commit, runs fine, but Chat gives the error noted.

Any other information that may be helpful

No response

[NickM-27]

Thanks I'll add it to my ongoing fix for genai stuff