F1AdaptiveThreshold and F1 score

[yelloji]

In Anomalib v1.2, the F1AdaptiveThreshold metric always defaults to a threshold of 0.5, and the model fails to make correct detections, resulting in an F1 score of 0.0. The threshold is not adapting as expected. Please help me
to solve this issue and suggest a solution.

[wawatt]

same

[samet-akcay]

Thank you for reporting this issue. To help us better understand and address the problem, could you please provide more information, something like;

1. Dataset details:

   • What dataset are you using?
   • What is the train/validation/test split ratio?
   • How many normal and anomalous samples are in your validation set?

2. Model configuration:

   • Which model architecture are you using?
   • What are your model's hyperparameters?
   • Are you using any specific data augmentation techniques?

3. Training details:

   • What is the distribution of your anomaly scores before thresholding?
   • Are you seeing any warnings in the logs about the validation set not containing anomalous samples?
   • What is the range of your anomaly scores (min/max values)?

The F1AdaptiveThreshold in Anomalib v1.2 is designed to:

1. Compute precision-recall curves across multiple thresholds
2. Calculate F1 scores at each threshold point
3. Select the threshold that yields the maximum F1 score

If you're getting a threshold of 0.5 and an F1 score of 0.0, this could be due to many reasons:

• The validation set might not contain any anomalous samples
• The anomaly scores might not be properly normalized
• The model might not be learning meaningful features
• There might be a mismatch between the anomaly score ranges and the threshold computation

With the additional information, we can better diagnose the root cause and provide a more targeted solution.