Problems with anomaly detection when the entire object is missing

[tominator95]

I was wondering why algorithms like PatchCore or PaDiM are struggling when the entire object in the image is missing.

For example:
I train a PatchCore model on the MVTec transistor dataset where the anomalies are normally bent pins. The model is working quite good. Why are these algorithms not able to detect the anomaly if the whole transistor is missing? The following example is from the EfficientAD paper (appendix). It shows that PatchCore isn't able to detect the missing transistor as an anomaly. I am wondering because the deviation between the normal and actual state is very large. I would expect that these cases are easy for image anomaly detection algorithms.

Currently I am struggling with the same issue in my use case. Can someone explain that to me? I read the papers of PaDiM and PatchCore but I do not find an answer on that.

[abc-125]

As I understand it, most existing anomaly detectors (like PatchCore) miss global context due to using features from lower levels of feature extractor (levels 2 and 3 of ResNet in the case of PatchCore). So they can detect a scratch, but cannot detect logical defects such as a missing part of the object or the whole object.

EfficientAD has a separate sub-model to deal with this, AutoEncoder, which successfully catches the global context, but it won't be able to detect smaller defects like scratches (there is a Student model for this).

You can read more on this here. Or in the EfficientAD paper.