Dinomaly is reconstructing too good --> no anomalies detected [just for an inidividual dataset]

[waschsalz]

When training Dinomaly on my own dataset, it never achieves a true anomaly detection capability.
It is a smaller dataset compaered to the MVTec categories, however it still contains about 100 training images. When training Dinomaly on the bottle subset of MVTec, it does a great job, but when training on my own images, it classifies every image as "OK" with very low anomaly scores.

I'm currently looking for possible solutions or trying to understand the mistakes made by myself.

[rajeshgangireddy]

Hi @waschsalz
We trained Dinomaly on other non-mvtec datasets and achieved good results but the key was to train fewer steps (around 500 to 100) than the default (5000) .

So, please try decreasing the number of epochs (or steps) you are training the model. You can pass max_epochs or max_steps to the Engine().

If it's "reconstructing too good" - You could also try increasing the bottleneck_dropout (default is 0.2) to around 0.4 ?
As per the Dinomaly paper :

The discard rate of Dropout in Noisy Bottleneck is 0.2 by default, which is increased to 0.4 for the diverse RealIAD.

Let me know how it goes.

[waschsalz]

Firstly, thank you very much for your quick reply.

I did decrease the amount of steps to 700 - 1000 and I also increased the bottleneck dropout throughout the different training processes. However my problem stays the same. The anomaly map is completely blue (normal) even for bigger defects, the scores also stay pretty low (~ 0.3 was the highest score so far).

Note:
I suppose I can confirm that my code for that example is not the problem as it is possible to plug-in other models like EfficientAD or Patchcore and it detects the anomalies with 100% recall.

I called it "reconstructing too good" as it seems to be most logical explanation to me. I am not sure if this suggestion is truly the cause.

[waschsalz]

Now, I received satisfying results. I was using an old datamodule setup which basically eliminates the usage of validation images. For my custom thresholding, this setup worked well for EfficientAD.
However for Dinomaly the lack of validation images boasted a big disadvantage of which the root cause seems to be clear at this point.

[rajeshgangireddy]

That is great.