Torch Inference, pre_process function error of Efficiant_Ad model

[semaegrii]

Hello,
I want to do prediction with torch inference using Efficient Ad model. I used Padim model with torch inference in v0.7.0 version of Anomalib. But I found that Efficient Ad works very fast and I want to try it with torch inference in your latest version. I have trained my images with Efficient Ad with input size 1024x1024. I am getting error while testing. @samet-akcay @alexriedel1

for image_path in test_images_path:
    image = cv2.imread(image_path)
    start = time.time()
    torch.cuda.synchronize()        
    prediction = inferencer.predict(image)
    torch.cuda.synchronize()
    predLabel=prediction.pred_label
    predScore=prediction.pred_score
    end = time.time()
    print(f'prediction time: {round((end - start) * 1000, 2)} ms ')

The error is as follows:
Exception has occurred: AttributeError
'numpy.ndarray' object has no attribute 'to'
File “C:\Users\sema.egri\source\Workspaces\Anomalib-V1\anomalib\src\anomalib\deploy\inferencers\torch_inferencer.py”, line 213, in pre_process
return image.to(self.device)
File “C:\Users\sema.egri\source\Workspaces\Anomalib-V1\anomalib\src\anomalib\deploy\inferencers\torch_inferencer.py”, line 187, in predict
processed_image = self.pre_process(image)
File “C:\Users\sema.egri\source\Workspaces\Anomalib-V1\anomalib\tools\test.py”, line 35, in testTorch
output=current.predict(image)
File “C:\Users\sema.egri\source\Workspaces\Anomalib-V1\anomalib\tools\test.py”, line 107, in
labelList, predList, elapsedTime = testTorch(model_name, test_images_path, checkpoint_path)
AttributeError: 'numpy.ndarray' object has no attribute 'to'

[lweggartner]

Have you tried to convert the image to a tensor?

  image = torch.from_numpy(image)

[alexriedel1]

Hey, ok lets make a few things clear:

• Inference time in the EfficientAD paper is measured on images of 256x256 on an NVIDIA RTX A6000 GPU and is 2.2ms for the EfficientAD small and 4.5ms for the EfficientAD medium model. The GPU and image size might have a high influence on inference time.
• You are training your model on a CPU by setting accelerator="cpu". This might take a very long time for training, you should use your gpu by just removing the accelerator argument from your engine class.
• For measuring only the model inference time you should leave away everything data loading related. E.g. loading the image from files, appyling the test data transformations etc. That's how you get closer to the times described in the paper. I wrote a small script to measure the pure model timing, you can find it here: Slower inference/testing with EfficientAD than promised. #1183 (comment)
  be aware that inferencing on the gpu via cuda needs some "warm up" operations until the maximum speed is reached, you can find furthers details about this topic here: https://www.speechmatics.com/company/articles-and-news/timing-operations-in-pytorch
  In the script above this is done by excluding the first ten inferencing processes from the timing calculation

[semaegrii]

OK, I will apply what you said. I will write to you again according to the situation. The reason I trained the model on the cpu was because I got an “out of memory” error.
I also have one more question. Assuming I follow what you said, do you think the dfkde model or the efficient_ad model is faster. If you were to recommend a model, what do you think is faster?

[alexriedel1]

I dont' know which of the two is faster on your hardware and desired image resolution

[semaegrii]

I realize I'm taking up a lot of your time, but I have one more question. Are these inference times (2.2ms for EfficientAD small and 4.5ms for EfficientAD medium) for forward inference only, or do they include preprocessing, forward pass, and post-processing? When measuring prediction time, after obtaining predLabel and predScore as outputs of my np.array type image, I stop the timer. Here's how I do it:

for image_path in test_images_path:
    image = cv2.imread(image_path)
    start = time.time()
    processed_image = inferencer.pre_process(image)
    torch.cuda.synchronize()
    predictions = inferencer.forward(processed_image)
    torch.cuda.synchronize()
    output = inferencer.post_process(predictions, metadata=metadata)
    predScore = output['pred_score']
    predLabel = output['pred_label']
    end = time.time()
    elapsedTime.append((end - start) * 1000)

[alexriedel1]

I would only time the forward pass and I think the paper authors did so too. Pre and post-processing might be very different from user to user

[laogonggong847]

Hello @alexriedel1 :
I remember you mentioning at efficientAD that he needed to leverage some of imageNet's functionality. In the code before 0.7.0, all the "yaml" files had a parameter "normalisation: imagenet" to do this, but in the latest 1.0 series, has this parameter been removed? So how do I enable this parameter?

Also I would like to ask about my second question, when I train efficientAD on my own dataset, my task type is "classification". If I don't provide any anomaly images at this point and just provide OK images, will this have any effect on the accuracy of my model?

Looking forward to your reply, thank you very much！

[alexriedel1]

Hi!

1. Yes, right
2. Correct. However don't change parameters unless you have a good reason to. E.g. batch size should be 1 for training EfficientAD.

[laogonggong847]

Thank you very much！！！
Thank you for your patience, I just don't quite understand why the batch_size of efficientAD should be set to 1? And how much should I set my epoch to when there are 200 training sets?

[alexriedel1]

Do you mean 200 samples? The papers authors used 70,000 steps for training, which would be equal to 350 epochs in your case.
The authors use batch size 1 in their experiments.

[laogonggong847]

Thank you very much！
I can't wait to try out EfficientAD now, thanks for all your contributions! The integration of EfficientAD in Anomalib is without a doubt one of the major contributions of Anomalib, thank you again!

[laogonggong847]

Hello @alexriedel1 :

When I train my own dataset using efficientAD, my code looks as follows：

if __name__ == "__main__":
    datamodule = Folder(
        name="myData",
        root="../Dataset",
        normal_dir="512_ALL_OK",
        # abnormal_dir="crack",
        task="classification",
        train_batch_size=1,
        eval_batch_size=1,
        num_workers=8,
        image_size=(512, 512)
    )

    print(torch.hub.get_dir())

    # Setup the datamodule
    datamodule.setup()
    print("Preparation Dataset Success")

    model = EfficientAd()
    print("Model load Success")

    # engine = Engine(task="classification")
    engine = Engine(task="classification", accelerator="gpu", devices="4", max_epochs = 10)

    engine.train(datamodule=datamodule, model=model)

    engine.export(model, export_type=ExportType.ONNX, export_root="../Result/Ef_AD")

    print("Success")

The efficientAD ONNX model with three output branches. Among them, aside from the result corresponding to "output," the other two branches seem to be ineffective for inference. When exporting with EfficientAD, does it not remove branches that are ineffective for inference? Or is there an issue with my export code? Sorry to take up your valuable time again, and thank you very much. Looking forward to your reply.

By the way, I have 4 GPUs, and I want to use multiple GPUs for training. However, I saw in some other discussions that @samet-akcay mentioned that the anomalib 1.2 series would support multi-GPU training. Is my current code utilizing multiple GPUs for training? (My anomalib is still from the 1.1 series).

thank you!