Can´t reproduce EfficientAD results on MVTec Loco AD Dataset

[sbastianhuber]

I can´t reproduce the results of EfficientAD on the MvTec Loco data set; I am always significantly less accurate.

I achieve an AUROC of 80.7 for structural error detection (results in the paper 94.7) and an AUROC of 71.5 for logical errors (results in the paper 86.8).

What am I doing wrong?

I would appreciate your help!

from anomalib.engine imort Engine
from anomalib.engine import Engine
from anomalib.models import EfficientAd
from anomalib.data import Folder
from anomalib.data.utils import TestSplitMode

from anomalib.models.image.efficient_ad.torch_model import (
    EfficientAdModel,
    EfficientAdModelSize
)

datamodule = Folder(
    name="pushpins",
    root="/content/MvTec_Loco_AD/breakfast_box/",
    normal_dir="train/good)",
    train_batch_size=1,  # Number of images per training batch
    eval_batch_size=32,  # Number of images per validation/test batch
    num_workers=8,  # Number of parallel processes for data loading
)

datamodule.setup()

model = EfficientAd(teacher_out_channels=384,
    model_size=EfficientAdModelSize.M)

engine = Engine(max_epochs=200)

engine.fit(datamodule=datamodule, model=model)

[alexriedel1]

In the paper they are training for 70,000 steps. I don't know how many epochs this would be for MVTec LOCO but you could set min_steps=70000 and try if your training maybe wasn't just long enough