Anomaly detection on high resolution images

[tominator95]

I want to train an anomaly detection model on high resolution images. The problem is that the images in this repo have a relatively low resolution (224x244 or 256x256). If I now have an input of 3000x2000px (for example of a PCB) and want to detect anomalies in it, the anomaly may simply disappear by scaling the image down.

Does anyone have an idea/suggestion how I can best implement this?

[djdameln]

You could experiment with Anomalib's tiling mechanism, which was designed for this purpose. Tiling subdivides the images into a grid of tiles, which are each processed by the model separately. This would allow you to retain more of the original image resolution without running into memory constraints. During testing and inference, the model predictions for each of the tile locations are stitched back together to form a single anomaly map for the pixel-level predictions.

Tiling is configured from the config.yaml. Just set dataset.tiling.apply to true and specify a tile size and stride.

For example, you could use the following configuration to tile your images into a 6x4 grid of tiles with a width and height of 500 pixels and no overlap between the tile locations:

  tiling:
    apply: true
    tile_size: 500
    stride: 500
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

Instead of resizing the full image to the dimensions specified in dataset.image_size, the resizing will now be applied to the individual tiles, which effectively increases the resolution of the image contents at seen by the input layer of the model. (In the current example of the 6x4 grid of tiles, when dataset.image_size is set to 256x256, the effective resolution of the images will become 1536x1024px, which is a lot better than the 256x256 resolution that would be used if tiling was disabled).

Please note that tiling is in experimental state and we do not have any benchmarking results to show. In our experience, tiling works best with models like STFPM and DRAEM, which involve fine-tuning of neural network weights. This is because a single model is trained on all tile locations combined. This makes tiling less suited for patch-based models like Padim and PatchCore, which assume that the contents of each spatial location within the image is fairly consistent across the dataset.

If you do want to use tiling with patch-based models, you could try to train an ensemble of models, where each model is trained on a distinct tile location. At the moment, Anomalib does not support this type of tiling, so you would have to implement your own wrapper utilities that perform the tiling and untiling separately.

[seanko29]

@djdameln Hello there! I have few questions regarding tiling you just replied to @tominator95

1. If tiling apply = True, then does the model ignore image input size of 256x256 or 224x224 when the input is given to the model?

2. If the image I want to train validate and test on is size of 512x512 pixels, and scale images down to 256x256 due to memory constraints, does applying tiling would increase performance?
   Or should I just retain my training image to 512x512 and reduce batch size instead? I am currently using RTX3090 with 24gb memory.

[tominator95]

@djdameln thanks for your suggestion! I will have a look into the tiling mechanism.

[djdameln]

@seanko29

1. After subdividing the original image into tiles, each tile itself is resized to the dimensions specified in dataset.image_size (I've updated my original answer to reflect this).
2. As I mentioned earlier, a single model will be trained on the multiple tile locations, which may have an adverse effect of the model's capability to learn the regularity in the dataset. In general I would therefore expect that better performance can be achieved by increasing the image size without tiling. Of course, at some point you will run into the limitations imposed by your available GPU memory, and this is where tiling may be useful. Enabling tiling will then allow you to further increase the effective resolution of the images as seen by the models.

[laogonggong847]

Hi, through your answer I have got some understanding of the tiling mechanism, but I still have doubts about the setting of the relevant parameters.

Suppose I have an image of 1920x1200 size and I don't want to reduce its size because the relevant features are inherently low in the whole image. If I directly reduce the size of 1920x1200 to 256x256, there is no doubt that this will directly lose a lot of details.

I have the following questions

1:For the image of the size I'm talking about (1920x1200), how should I set the relevant parameters if I use tiling

2：What is the meaning of the relevant parameters.

tiling.
    apply: true
    tile_size: 500
    stride: 500
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

3: I still don't understand the relationship between the relevant parameters and image_size in dataset when the tiling mechanism is turned on. Can you give me an example to help me understand better

Thank you very much and I look forward to your reply!

[monkeycc]

My project is also PCB and PCBA
I'm also studying

But my picture size
16000 * 16000

Annotation file
Picture cut into 512 * 512
Dimensioning

Camera loading, picture clipping, inference and prediction