[Detector Support]: with YOLOv9 on EdgeTPU, objects are not creating review items even when confirmed with debug screen

[raintonr]

Describe the problem you are having

I wanted to try the YOLOv9 model for Coral EdgeTPU as the documentation states it can be more accurate so followed the installation instructions here:

https://docs.frigate.video/configuration/object_detectors#yolov9

• Download the model & labelmap files and place in config path
• Modify Frigate config to use these by adding this:

model:
  model_type: yolo-generic
  width: 320
  height: 320
  path: /config/model_cache/yolov9-s-relu6-best_320_int8_edgetpu.tflite
  labelmap_path: /config/labels-coco17.txt

The labels in the labelmap are compatible with those in the Mobiledet model so expected no other changes to be necessary.

This seems to have broken review functionality though. Ie. I got practically zero Alerts or Detections on the review screen. Oddly I did manage to get one detection during testing but cannot figure out what was different about that.

Looking at a camera on the debug screen, objects of interest (eg. people) are clearly recognised and score well above the threshold necessary to trigger and Alert but none was created.

Version

0.17.0-f0d69f7

Frigate config file

version: 0.17-0
logger:
  default: info
  logs:
    frigate.event: info

mqtt:
  host: 10.1.1.30
  stats_interval: 300

detectors:
  coral1:
    type: edgetpu
    device: pci:0

model:
  model_type: yolo-generic
  width: 320
  height: 320
  path: /config/model_cache/yolov9-s-relu6-best_320_int8_edgetpu.tflite
  labelmap_path: /config/labels-coco17.txt

# Objects we are interested in
objects:
  track: [bicycle, bus, car, cat, cow, dog, horse, motorcycle, person, sheep, truck]
  filters:
    # Default filters to ignore super tiny stuff
    bicycle:
      min_area: 1000
    bus:
      min_area: 1000
    car:
      min_area: 1000
    cat:
      min_area: 1000
    cow:
      min_area: 1000
    dog:
      min_area: 1000
    horse:
      min_area: 1000
    motorcycle:
      min_area: 1000
    person:
      min_area: 1000
    sheep:
      min_area: 1000
    truck:
      min_area: 1000

# Review all objects of interest
review:
  alerts:
    labels:
      - bicycle
      - bus
      - car
      - cat
      - cow
      - dog
      - horse
      - motorcycle
      - person
      - sheep
      - truck

motion:
  improve_contrast: true
  threshold: 45

birdseye:
  enabled: false

live:
  height: 480

timestamp_style:
  format: '%Y/%m/%d %H:%M:%S'
  thickness: 2
  effect: shadow

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264

snapshots:
  enabled: true
  timestamp: true
  retain:
    default: 365

record:
  enabled: true
  # retain everything for 12 hours
  alerts:
    retain:
      days: 30
  detections:
    retain:
      days: 30
  continuous:
    days: 0.5
  motion:
    days: 0.5
cameras:
  porch_intercom:
    # Object must be considered in one of the defined zones in order to honour zone filters
    mqtt:
      enabled: false
      required_zones: [terrace_north_near, terrace_north_far, terrace_north_west,
        parking_side_near, parking_side_far, driveway_top, lawn_west_near, 
            garage_north_shed]
    review:
      alerts:
        required_zones:
          - terrace_north_near
          - terrace_north_far
          - terrace_north_west
          - parking_side_near
          - parking_side_far
          - driveway_top
          - lawn_west_near
          - garage_north_shed
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.60:554/Streaming/Channels/2
          roles:
            - detect
        - path: rtsp://user:pass@10.1.3.60:554/Streaming/Channels/1
          roles:
            - record
    detect:
      width: 704
      height: 576
    motion:
      mask:
        # Timestamp mm:ss
        - 0.841,0.781,0.896,0.781,0.896,0.799,0.841,0.799
        - 1,0,1,0.59,0.621,0.565,0.471,0.477,0.312,0.492,0.31,0.551,0.185,0.553,0,0.513,0,0
    zones:
      terrace_north_near:
        coordinates: 
          0.892,0.735,1,0.726,1,1,0.211,1,0.162,0.922,0.318,0.873,0.477,0.8,0.488,0.774,0.549,0.731
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        filters:
          person:
            min_area: 5000
        inertia: 3
        loitering_time: 0
      terrace_north_far:
        coordinates: 
          1,0.723,1,0.593,0.7,0.573,0.695,0.686,0.679,0.727,0.906,0.735
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        filters:
          person:
            min_area: 5000
        inertia: 3
        loitering_time: 0
      terrace_north_west:
        coordinates: 
          0.207,1,0,1,0,0.75,0.048,0.749,0.06,0.802,0.085,0.863,0.157,0.924
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        filters:
          person:
            min_area: 5000
        inertia: 3
        loitering_time: 0
      parking_side_near:
        coordinates: 
          0.402,0.736,0.471,0.798,0.364,0.858,0.159,0.92,0.088,0.862,0.062,0.799,0.055,0.762
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        filters:
          person:
            min_area: 5000
        inertia: 3
        loitering_time: 0
      parking_side_far:
        coordinates: 0.398,0.733,0.308,0.633,0.136,0.619,0.11,0.656,0.057,0.756
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        filters:
          person:
            min_area: 5000
        inertia: 3
        loitering_time: 0
      driveway_top:
        coordinates: 0.312,0.592,0.21,0.591,0.165,0.6,0.139,0.618,0.31,0.629
        inertia: 3
        loitering_time: 0
      lawn_west_near:
        coordinates: 
          0.205,0.589,0.054,0.561,0,0.562,0,0.745,0.058,0.748,0.102,0.658,0.151,0.6
        inertia: 3
        loitering_time: 0
      garage_north_shed:
        coordinates: 
          0.616,0.676,0.626,0.566,0.699,0.573,0.692,0.69,0.676,0.727,0.55,0.727
        loitering_time: 0
        # Don't track vehicles
        objects:
          - person
          - dog
          - cow
          - cat
          - horse
          - sheep
        inertia: 3
    objects:
      filters:
        car:
          mask: 
            0.035,0.657,0.348,0.675,0.401,0.733,0.473,0.793,0.363,0.864,0.158,0.918,0.081,0.86,0.037,0.74
  gate_intercom:
    # Object must be considered in one of the defined zones in order to honour zone filters
    mqtt:
      enabled: false
      required_zones: [gate_outside_far, gate_outside_near, gate_inside, 
            lawn_west]
    review:
      alerts:
        required_zones:
          - gate_outside_far
          - gate_outside_near
          - gate_inside
          - lawn_by_gate
      detections:
        required_zones: road
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.61:554/Streaming/Channels/2
          roles:
            - detect
        - path: rtsp://user:pass@10.1.3.61:554/Streaming/Channels/1
          roles:
            - record
    detect:
      width: 704
      height: 576
    motion:
      mask: 0,0,1,0,1,1,0.743,1,0.77,0.705,0.499,0.6,0.501,0.505,0,0.327
    zones:
      road:
        coordinates: 0.421,0.665,0.755,0.86,0.767,0.707,0.504,0.6,0.503,0.667
        inertia: 3
        loitering_time: 0
      gate_outside_far:
        coordinates: 0.413,0.667,0.748,0.862,0.721,1,0.325,0.665
        inertia: 3
        loitering_time: 0
      gate_outside_near:
        coordinates: 0.251,0.667,0.088,1,0.715,1,0.317,0.664
        inertia: 3
        loitering_time: 0
      gate_inside:
        coordinates: 0.081,1,0,1,0,0.618,0.251,0.613,0.25,0.659
        inertia: 3
        loitering_time: 0
      lawn_by_gate:
        coordinates: 
          0.034,0.613,0.038,0.341,0.497,0.507,0.495,0.557,0.251,0.465,0.251,0.609
        inertia: 3
        loitering_time: 0
    objects:
      mask: 0.736,0.997,0.76,0.793,0.762,0.605,0.784,0.165,0.767,0,1,0,1,1
  csw1:
    mqtt:
      enabled: false
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.62:554/Streaming/Channels/2
          roles:
            - detect
        - path: rtsp://user:pass@10.1.3.62:554/Streaming/Channels/1
          roles:
            - record
    detect:
      width: 640
      height: 480
    motion:
      mask:
        # Timestamp mm:ss
        - 0.912,0.921,0.973,0.921,0.973,0.942,0.912,0.942
        - 1,0,1,0.277,0.894,0.174,0.696,0.046,0.599,0.023,0.43,0.06,0.379,0.065,0.332,0.013,0.13,0.095,0.145,0.305,0.143,0.533,0.153,1,0,1,0,0
    zones:
      field:
        coordinates: 
          0.609,0,0.633,0,0.853,0.145,1,0.279,1,0.471,0.835,0.266,0.655,0.083,0.599,0.053
        inertia: 3
        loitering_time: 0
      lawn_south_near:
        coordinates: 0.6,0.133,1,0.683,1,1,0.155,1,0.148,0.523,0.148,0.311
        inertia: 3
        loitering_time: 0
      lawn_south_far:
        coordinates: 
          0.6,0.13,1,0.676,1,0.475,0.842,0.283,0.641,0.078,0.6,0.058,0.141,0.222,0.147,0.309
        inertia: 3
        loitering_time: 0
    objects:
      filters:
        # Stop erroneous large sheep!
        sheep:
          max_area: 90000
    review:
      alerts:
        required_zones:
          - field
          - lawn_south_far
          - lawn_south_near
  csw2:
    mqtt:
      enabled: false
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.63:554/Streaming/Channels/2
          roles:
            - detect
        - path: rtsp://user:pass@10.1.3.63:554/Streaming/Channels/1
          roles:
            - record
    detect:
      width: 640
      height: 480
    motion:
      mask:
        # Timestamp mm:ss
        - 0.912,0.921,0.973,0.921,0.973,0.942,0.912,0.942
        - 0.198,0.133,0.497,0.078,0.876,0.103,1,0.161,1,0,0,0,0,0.214
    zones:
      field:
        coordinates: 
          0.704,0.188,0.915,0.212,1,0.23,1,0.164,0.879,0.106,0.499,0.08,0.194,0.137,0,0.22,0,0.34,0.218,0.255,0.441,0.214
        inertia: 3
        loitering_time: 0
      lawn_south_near:
        coordinates: 0.464,0.285,0.206,0.346,0,0.445,0,1,1,1,1,0.861
        inertia: 3
        loitering_time: 0
      lawn_south_far:
        coordinates: 
          0.463,0.282,0.204,0.339,0,0.437,0,0.34,0.221,0.26,0.417,0.223
        inertia: 3
        loitering_time: 0
      lawn_west_near:
        coordinates: 0.511,0.336,0.733,0.287,1,0.29,1,0.856
        inertia: 3
        loitering_time: 0
      lawn_west_far:
        coordinates: 
          0.509,0.336,0.735,0.283,1,0.29,1,0.232,0.705,0.194,0.417,0.22
        inertia: 3
        loitering_time: 0
    review:
      alerts:
        required_zones:
          - field
          - lawn_south_far
          - lawn_south_near
          - lawn_west_far
          - lawn_west_near
  csw3:
    mqtt:
      enabled: false
    review:
      alerts:
        required_zones:
          - driveway_top
          - lawn_west_near
          - lawn_west_far
          - parking_side_far
          - terrace_west
          - field
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.66:554/Streaming/Channels/2
          roles:
            - detect
        - path: rtsp://user:pass@10.1.3.66:554/Streaming/Channels/1
          roles:
            - record
    detect:
      width: 640
      height: 480
    motion:
      mask:
        # Timestamp mm:ss
        - 0.912,0.921,0.973,0.921,0.973,0.942,0.912,0.942
        - 0.113,0.12,0.238,0,0,0,0,0.288
    zones:
      driveway_top:
        coordinates: 0.76,0.039,0.59,0.014,0.661,0.08,0.844,0.097,0.85,0.07
        inertia: 3
        loitering_time: 0
      lawn_west_near:
        coordinates: 
          0.099,1,0,1,0,0.652,0.292,0.038,0.336,0.024,0.467,0.007,0.587,0.015,0.769,0.194,0.82,0.22,0.738,0.415,0.579,0.325
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        inertia: 3
        loitering_time: 0
      lawn_west_far:
        coordinates: 0,0.404,0,0.648,0.287,0.041,0.208,0.094,0.11,0.184
        inertia: 3
        loitering_time: 0
      parking_side_far:
        coordinates: 0.679,0.084,0.678,0.179,0.887,0.264,0.9,0.107
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        inertia: 3
        loitering_time: 0
      terrace_west:
        coordinates: 1,0.282,0.822,0.22,0.74,0.416,0.582,0.326,0.104,1,1,1
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        inertia: 3
        loitering_time: 0
      field:
        coordinates: 0,0.296,0.099,0.145,0.102,0.193,0,0.4
        loitering_time: 0
    objects:
      filters:
        car: {}
  cgw:
    mqtt:
      enabled: false
    review:
      alerts:
        required_zones:
          - driveway_top
          - gate_inside
          - lawn_by_road
          - lawn_west_near
          - parking_side_far
          - lawn_west_far
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.65:554/Streaming/Channels/2
          roles:
            - detect
        - path: rtsp://user:pass@10.1.3.65:554/Streaming/Channels/1
          roles:
            - record
    detect:
      width: 640
      height: 480
    motion:
      mask:
        # Timestamp mm:ss
        - 0.912,0.921,0.973,0.921,0.973,0.942,0.912,0.942
        - 1,0,0,0,0,0.488,0.218,0.203,0.532,0.057,0.72,0.016,0.874,0.046,1,0.152
    zones:
      driveway_top:
        coordinates: 
          0.132,1,0.831,1,0.867,0.507,0.855,0.255,0.442,0.285,0.324,0.373,0.147,0.626,0.244,0.806
        inertia: 3
        loitering_time: 0
      gate_inside:
        coordinates: 
          0.727,0.031,0.716,0.021,0.65,0.049,0.55,0.09,0.592,0.087,0.605,0.11,0.581,0.16,0.526,0.218,0.443,0.28,0.86,0.249,0.663,0.061
        inertia: 3
        loitering_time: 0
      lawn_by_road:
        coordinates: 
          1,1,1,0.178,0.872,0.064,0.737,0.03,0.669,0.06,0.858,0.237,0.878,0.507,0.842,1
        inertia: 3
        loitering_time: 0
      lawn_west_near:
        coordinates: 
          0,0.591,0,0.764,0.102,0.664,0.321,0.369,0.442,0.278,0.373,0.173,0.159,0.381
        inertia: 3
        loitering_time: 0
      parking_side_far:
        coordinates: 0,0.769,0,1,0.128,1,0.238,0.805,0.142,0.62,0.065,0.703
        # Don't track vehicles
        objects:
          - cat
          - cow
          - dog
          - horse
          - person
          - sheep
        inertia: 3
        loitering_time: 0
      lawn_west_far:
        coordinates: 
          0.443,0.277,0.523,0.213,0.576,0.157,0.599,0.11,0.592,0.092,0.535,0.093,0.465,0.135,0.375,0.173
        inertia: 3
        loitering_time: 0
    objects:
      mask: 0.791,0.997,0.831,0.682,0.963,0.745,1,0.828,1,1
  cse:
    mqtt:
      enabled: false
    review:
      alerts:
        required_zones:
          - lawn_east
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.67:554/Streaming/Channels/2
          roles:
            - detect
        - path: rtsp://user:pass@10.1.3.67:554/Streaming/Channels/1
          roles:
            - record
    detect:
      width: 640
      height: 480
    motion:
      mask:
        # Timestamp mm:ss
        - 0.912,0.921,0.973,0.921,0.973,0.942,0.912,0.942
        - 1,0,0,0,0,1,0.31,1,0.145,0.309,0.135,0.108,0.571,0.079,1,0.261
    zones:
      lawn_east:
        coordinates: 0.361,1,1,1,1,0.34,0.576,0.151,0.143,0.171,0.153,0.291
    # Don't track vehicles
        inertia: 3
        loitering_time: 0
    objects:
      track: [cat, cow, dog, horse, person, sheep]
  lounge:
    enabled: false
    detect:
      enabled: false
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@10.1.3.50:554/av_stream/ch0
detect:
  enabled: true

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:0.17.0
    # shm_size is now calculated in the UI
    shm_size: "216mb"
    devices:
      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      # After kernal update module on host needs rebuilding.
      # See https://github.com/robertzaage/gasket-driver
      # dkms build -m gasket -v 0.0.git.236.387cfd9c -q
      # dkms install -m gasket -v 0.0.git.236.387cfd9c -q
      # dkms status
      # Should show:
      # gasket/0.0.git.236.387cfd9c, 6.11.3-200.fc40.x86_64, x86_64: installed
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/frigate/config:/config
      - /home/frigate/media:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 500000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds

Relevant Frigate log output

Could not find anything relevant in logs.

Install method

Docker Compose

Object Detector

Coral

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

Based on the configuration and symptoms you've described, the issue appears to be related to how the YOLOv9 EdgeTPU model handles the limited labelmap. The YOLOv9 model for Google Coral uses a 17-label subset of COCO classes, which may affect how Frigate processes detections(1).

Several factors could be preventing review items from being created:

Labelmap Compatibility Issue

The YOLOv9 EdgeTPU model uses a reduced labelmap with only 17 object classes due to hardware limitations of the Coral(1). Your configuration tracks objects like bicycle, bus, truck, etc., but these may not all be present in the 17-label subset. Verify that all objects in your objects.track list exist in /config/labels-coco17.txt(1).

Required Zones Configuration

Your cameras have required_zones configured under review.alerts. Objects must enter these zones to generate review items. If detections are occurring outside these zones (even with high confidence scores on the debug screen), they won't create alerts(2).

Zone-Specific Object Filters

Many of your zones have custom objects lists that exclude certain tracked objects. For example, several zones only track non-vehicle objects like person, dog, cat, etc., while excluding vehicles(2). Detections outside the zone's object list won't trigger alerts for that zone.

Threshold and Scoring Differences

Community feedback indicates that YOLOv9 EdgeTPU models may have different scoring characteristics compared to MobileNet(3). You may need to adjust your thresholds. The default threshold is 0.7 for most objects, but the YOLOv9 model might require different values(2).

Recommended Troubleshooting Steps

1. Verify your labelmap contains all tracked objects
2. Temporarily remove required_zones from one camera's review configuration to test if detections create review items
3. Check if detected objects on the debug screen are actually entering your defined zones
4. Lower the detection threshold for specific objects to see if scoring differences are the issue
5. Review the exact labels in your labels-coco17.txt file

The one detection that worked likely met all criteria: correct label, entered a required zone, passed threshold, and matched zone object filters.

📚 Sources:

1. Supported Hardware
2. Full Reference Config
3. Yolov9 model - Coral vs OpenVINO

---

Have another question? Just tag @inkeep.

[raintonr]

Labelmap Compatibility Issue

I have confirmed the labels (eg. person) are compatible.

Required Zones Configuration

On the debug screen I confirmed the object of interest was indeed within one of the required zones. By "within" I mean that bottom centre of the bounding box was inside the zone.

Zone-Specific Object Filters

Noted, but person for example is of interest everywhere and still did not trigger a review item.

Threshold and Scoring Differences

Noted, and I did try temporarily reducing the threshold with no change.

Recommended Troubleshooting Steps

I will try some of these steps and report back.

[NickM-27]

The debug view will show what is going on, we'd need to see that to be able to understand what may be going on.

[raintonr]

I can see in the debug view is that there is an object, of the correct class, in the correct zone and with the correct area to trigger an alert for example... but no alert is created. I don't know why this is.

Should I be looking in the logs or someplace else I have missed?

[NickM-27]

If you can share the image that would make it easier for us to answer. But first guess is that the score is too low to be a true positive.

You also need to make sure that recordings are actively enabled.

[raintonr]

The score is good and recordings work with the default model so that must be all correctly configured.

I will take a screen grab of an example.

[raintonr]

Well... this is very odd. All I did was add the YOLO model section back into my config file and it now works just like the default one (only better I hope - only time will tell).

I cannot explain why when I first tried this it wasn't triggering anything to review, but I must have accidentally done something dumb elsewhere in the config to break that as things are now OK.

I will therefore close this issue. Sorry for wasting time.