Audio detection labels selection

[skylineagle]

Hey, i have a frigate instance where i'm interested in getting alerts on specific audio events like speech yell etc.

Last day i had an alert for a "speech" audio event, i opened it and it was a bee or a fly on the mic.
I'm wondering rather it was detected wit higher score as a fly but was filtered out due to the label list i have configured?
What is the ideal setup for the label list in my case?

[inkeep-ai-assistant[bot]]

The audio detection model runs inference on all sounds regardless of your configured listen labels(1). The listen configuration only filters which detected labels create audio events - it doesn't affect what the model detects or scores(1).

When the bee/fly sound was detected as "speech," this means the model scored "speech" highest for that audio, not that a "fly" or "insect" label was filtered out(1). The model detects everything in one go, and then the list is checked for what should be monitored(1). There is no performance difference based on the number of labels you configure(1).

For your use case focused on human activity alerts, your current setup with labels like speech and yell is appropriate(2). By default, Frigate listens for bark, fire_alarm, scream, speech, and yell(2).

To reduce false positives like the bee/fly incident, you can adjust the threshold for specific labels(2)(3):

audio:
  enabled: True
  listen:
    - speech
    - yell
  filters:
    speech:
      threshold: 0.8

(2), (3)

The threshold value determines the minimum score required to trigger an audio event, with higher values being more restrictive(2)(3). You might also want to adjust min_volume to filter out quieter sounds that are less likely to be human speech(2)(3).

The included audio model has over 500 different types of audio that can be detected(2). You can view the complete list in the audio-labelmap.txt file(2).

📚 Sources:

1. Audio detection question: how to trigger on any noise?
2. Audio Detectors
3. Full Reference Config

---

Have another question? Just tag @inkeep.

[skylineagle]

@inkeep that's what i did, yet sore ch also passed the 0.8 suggested threshold.
is there a way to investigate the specific event model scores for other label to have a better understanding on the threshold to fit my needs?

[inkeep-ai-assistant[bot]]

I don't have specific information in the available sources about how to view detailed model scores for all labels for a specific audio event. The documentation doesn't describe a debug mode or logging option that would expose the raw confidence scores for all detected audio labels during a particular event.

Based on the available sources, the audio detection configuration allows you to set thresholds per label(1), but there isn't documented functionality to review what scores other labels received for a specific event to help you calibrate your thresholds.

If you'd like more detailed debugging capabilities for audio events, you might want to check the Frigate logs or consider reaching out to the community. Visit GitHub for support.

Would you like a general answer based on my knowledge instead?

📚 Sources:

1. Full Reference Config

---

Have another question? Just tag @inkeep.