You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Bird classification identifies known birds using a quantized Tensorflow model. When a known bird is recognized, its common name will be added as a `sub_label`. This information is included in the UI, filters, as well as in notifications.
7
+
8
+
## Minimum System Requirements
9
+
10
+
Bird classification runs a lightweight tflite model on the CPU, there are no significantly different system requirements than running Frigate itself.
11
+
12
+
## Model
13
+
14
+
The classification model used is the MobileNet INat Bird Classification, [available identifiers can be found here.](https://raw.githubusercontent.com/google-coral/test_data/master/inat_bird_labels.txt)
15
+
16
+
## Configuration
17
+
18
+
Bird classification is disabled by default, it must be enabled in your config file before it can be used. Bird classification is a global configuration setting.
19
+
20
+
```yaml
21
+
classification:
22
+
bird:
23
+
enabled: true
24
+
```
25
+
26
+
## Advanced Configuration
27
+
28
+
Fine-tune bird classification with these optional parameters:
29
+
30
+
- `threshold`: Classification confidence score required to set the sub label on the object.
Copy file name to clipboardExpand all lines: docs/docs/configuration/face_recognition.md
+35-19Lines changed: 35 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,21 +7,26 @@ Face recognition identifies known individuals by matching detected faces with pr
7
7
8
8
## Model Requirements
9
9
10
-
Frigate has support for CV2 Local Binary Pattern Face Recognizer to recognize faces, which runs locally. A lightweight face landmark detection model is also used to align faces before running them through the face recognizer.
10
+
### Face Detection
11
11
12
-
Users running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient.
12
+
When running a Frigate+ model (or any custom model that natively detects faces) should ensure that `face` is added to the [list of objects to track](../plus/#available-label-types) either globally or for a specific camera. This will allow face detection to run at the same time as object detection and be more efficient.
13
13
14
-
Users without a model that detects faces can still run face recognition. Frigate uses a lightweight DNN face detection model that runs on the CPU. In this case, you should _not_ define `face` in your list of objects to track.
14
+
When running a default COCO model or another model that does not include `face` as a detectable label, face detection will run via CV2 using a lightweight DNN model that runs on the CPU. In this case, you should _not_ define `face` in your list of objects to track.
15
15
16
-
:::note
16
+
### Face Recognition
17
17
18
-
Frigate needs to first detect a `face` before it can recognize a face.
18
+
Frigate has support for two face recognition model types:
19
19
20
-
:::
20
+
-**small**: Frigate will run a FaceNet embedding model to recognize faces, which runs locally on the CPU. This model is optimized for efficiency and is not as accurate.
21
+
-**large**: Frigate will run a large ArcFace embedding model that is optimized for accuracy. It is only recommended to be run when an integrated or dedicated GPU is available.
22
+
23
+
In both cases, a lightweight face landmark detection model is also used to align faces before running recognition.
21
24
22
25
## Minimum System Requirements
23
26
24
-
Face recognition is lightweight and runs on the CPU, there are no significantly different system requirements than running Frigate itself.
27
+
The `small` model is optimized for efficiency and runs on the CPU, most CPUs should run the model efficiently.
28
+
29
+
The `large` model is optimized for accuracy, an integrated or discrete GPU is highly recommended.
25
30
26
31
## Configuration
27
32
@@ -47,12 +52,15 @@ Fine-tune face recognition with these optional parameters:
47
52
48
53
### Recognition
49
54
55
+
- `model_size`: Which model size to use, options are `small` or `large`
56
+
- `unknown_score`: Min score to mark a person as a potential match, matches at or below this will be marked as unknown.
57
+
- Default: `0.8`.
50
58
- `recognition_threshold`: Recognition confidence score required to add the face to the object as a sub label.
51
59
- Default: `0.9`.
52
60
- `blur_confidence_filter`: Enables a filter that calculates how blurry the face is and adjusts the confidence based on this.
53
61
- Default: `True`.
54
62
55
-
## Dataset
63
+
## Creating a Robust Training Set
56
64
57
65
The number of images needed for a sufficient training set for face recognition varies depending on several factors:
58
66
@@ -61,11 +69,9 @@ The number of images needed for a sufficient training set for face recognition v
61
69
62
70
However, here are some general guidelines:
63
71
64
-
- Minimum: For basic face recognition tasks, a minimum of 10-20 images per person is often recommended.
65
-
- Recommended: For more robust and accurate systems, 30-50 images per person is a good starting point.
66
-
- Ideal: For optimal performance, especially in challenging conditions, 100 or more images per person can be beneficial.
67
-
68
-
## Creating a Robust Training Set
72
+
- Minimum: For basic face recognition tasks, a minimum of 5-10 images per person is often recommended.
73
+
- Recommended: For more robust and accurate systems, 20-30 images per person is a good starting point.
74
+
- Ideal: For optimal performance, especially in challenging conditions, 50-100 images per person can be beneficial.
69
75
70
76
The accuracy of face recognition is heavily dependent on the quality of data given to it for training. It is recommended to build the face training library in phases.
71
77
@@ -76,8 +82,9 @@ When choosing images to include in the face training set it is recommended to al
76
82
- If it is difficult to make out details in a persons face it will not be helpful in training.
77
83
- Avoid images with extreme under/over-exposure.
78
84
- Avoid blurry / pixelated images.
79
-
- Be careful when uploading images of people when they are wearing clothing that covers a lot of their face as this may confuse the model.
80
-
- Do not upload too many similar images at the same time, it is recommended to train no more than 4-6 similar images for each person to avoid overfitting.
85
+
- Avoid training on infrared (gray-scale). The models are trained on color images and will be able to extract features from gray-scale images.
86
+
- Using images of people wearing hats / sunglasses may confuse the model.
87
+
- Do not upload too many similar images at the same time, it is recommended to train no more than 4-6 similar images for each person to avoid over-fitting.
81
88
82
89
:::
83
90
@@ -87,7 +94,7 @@ When first enabling face recognition it is important to build a foundation of st
87
94
88
95
Then it is recommended to use the `Face Library` tab in Frigate to select and train images for each person as they are detected. When building a strong foundation it is strongly recommended to only train on images that are straight-on. Ignore images from cameras that recognize faces from an angle.
89
96
90
-
Aim to strike a balance between the quality of images while also having a range of conditions (day / night, different weather conditions, different times of day, etc.) in order to have diversity in the images used for each person and not have overfitting.
97
+
Aim to strike a balance between the quality of images while also having a range of conditions (day / night, different weather conditions, different times of day, etc.) in order to have diversity in the images used for each person and not have over-fitting.
91
98
92
99
Once a person starts to be consistently recognized correctly on images that are straight-on, it is time to move on to the next step.
93
100
@@ -97,13 +104,22 @@ Once straight-on images are performing well, start choosing slightly off-angle i
97
104
98
105
## FAQ
99
106
100
-
### Why is every face tagged as a known face and not unknown?
107
+
### Why can't I bulk upload photos?
101
108
102
-
Any recognized face with a score >= `min_score` will show in the `Train` tab along with the recognition score. A low scoring face is effectively the same as `unknown`, but includes more information. This does not mean the recognition is not working well, and is part of the importance of choosing the correct `recognition_threshold`.
109
+
It is important to methodically add photos to the library, bulk importing photos (especially from a general photo library) will lead to over-fitting in that particular scenario and hurt recognition performance.
110
+
111
+
### Why can't I bulk reprocess faces?
112
+
113
+
Face embedding models work by breaking apart faces into different features. This means that when reprocessing an image, only images from a similar angle will have its score affected.
103
114
104
115
### Why do unknown people score similarly to known people?
105
116
106
-
This can happen for a few different reasons, but this is usually an indicator that the training set needs to be improved. This is often related to overfitting:
117
+
This can happen for a few different reasons, but this is usually an indicator that the training set needs to be improved. This is often related to over-fitting:
118
+
107
119
- If you train with only a few images per person, especially if those images are very similar, the recognition model becomes overly specialized to those specific images.
108
120
- When you provide images with different poses, lighting, and expressions, the algorithm extracts features that are consistent across those variations.
109
121
- By training on a diverse set of images, the algorithm becomes less sensitive to minor variations and noise in the input image.
122
+
123
+
### I see scores above the threshold in the train tab, but a sub label wasn't assigned?
124
+
125
+
The Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results.
0 commit comments