How to do CLASSIFICATION task with config file in 2.0.0

[panp4n]

How to do classification task with config file in 2.0.0b3?
Is there a parameter in the config file of 2.0.0b3 like task: CLASSIFICATION in 1.2.0?
I update anomalib to 2.0.0b3 from 1.2.0, there are too much difference in usage. I think detailed usage documentation is what we need more.

My custom config yaml file:

seed_everything: true
trainer:
  accelerator: auto
  strategy: auto
  devices: auto
  num_nodes: 1
  precision: null
  logger: null
  callbacks: null
  fast_dev_run: false
  max_epochs: null
  min_epochs: null
  max_steps: -1
  min_steps: null
  max_time: null
  limit_train_batches: null
  limit_val_batches: null
  limit_test_batches: null
  limit_predict_batches: null
  overfit_batches: 0.0
  val_check_interval: null
  check_val_every_n_epoch: 1
  num_sanity_val_steps: null
  log_every_n_steps: null
  enable_checkpointing: null
  enable_progress_bar: null
  enable_model_summary: null
  accumulate_grad_batches: 1
  gradient_clip_val: null
  gradient_clip_algorithm: null
  deterministic: null
  benchmark: null
  inference_mode: true
  use_distributed_sampler: true
  profiler: null
  detect_anomaly: false
  barebones: false
  plugins: null
  sync_batchnorm: false
  reload_dataloaders_every_n_epochs: 0

model:
  class_path: anomalib.models.Patchcore
  init_args:
    backbone: wide_resnet50_2
    layers:
    - layer2
    - layer3
    pre_trained: true
    coreset_sampling_ratio: 0.1
    num_neighbors: 9
    pre_processor: true
    post_processor: true
    evaluator: true
    visualizer: true
data:
  class_path: anomalib.data.Folder
  init_args:
    name: hazelnut
    normal_dir: train/good
    root: datasets/MVTec/hazelnut
    abnormal_dir: ['test/crack', 'test/cut', 'test/hole', 'test/print']
    normal_test_dir: test/good
    mask_dir: null
    normal_split_ratio: 0.0
    extensions:
    - .png
    train_batch_size: 16
    eval_batch_size: 16
    num_workers: 8
    train_augmentations: null
    val_augmentations: null
    test_augmentations: null
    augmentations: null
    test_split_mode: from_dir
    test_split_ratio: 0.2
    val_split_mode: same_as_test
    val_split_ratio: 0.5
    seed: null
    
logging:
  log_graph: false
default_root_dir: results
ckpt_path: null

[haimat]

Hello, I have the same issue, don't know how to start with 2.0 -
have you had any luck in that regard so far?

[syshin0116]

Hi, same issue here, I would like to train a classification model without masks. any updates?

[djdameln]

Hi, you can just omit the task parameter. Anomalib will automatically recognize that it should ignore pixel-level evaluation due to absence of the mask_dir parameter.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[samet-akcay]

With Anomalib v2, task is handled under the hood, so you don't need to specify anything. Depending on what you pass as input data, Anomalib ensures whether it is a classification or segmentation task.

If you want to customise, things we split everything into the following categories:

• Pre-Processing
• Model Forward Padd
• Post-Processing
• Evaluation
• Visualization

So, each model now has the corresponding pre_processor, model, post_processor, evaluator and visualizer modules. If you want to customise them you can do via both CLI and API.

In your case, you don't pass ground truth masks, so you might want some customised metric evaluation and visualisation. Let's have a look how you could do it.

Custom Evaluation

via API

from anomalib.data import Folder
from anomalib.engine import Engine
from anomalib.metrics import Evaluator, F1Score
from anomalib.models import Patchcore

datamodule = Folder(
    name="hazelnut",
    normal_dir="train/good",
    root="datasets/MVTec/hazelnut",
    abnormal_dir=["test/crack", "test/cut", "test/hole", "test/print"],
    normal_test_dir="test/good",
)

evaluator = Evaluator(
    test_metrics=[
        F1Score(fields=["pred_score", "gt_label"]),
    ],
)

model = Patchcore(
    backbone="wide_resnet50_2",
    layers=["layer2", "layer3"],
    pre_trained=True,
    coreset_sampling_ratio=0.1,
    num_neighbors=9,
    evaluator=evaluator,
)
engine = Engine()

engine.train(model=model, datamodule=datamodule)

via CLI

You just need to pass the evaluator configuration to Patchcore model config:

model:
  class_path: anomalib.models.Patchcore
  init_args:
    backbone: wide_resnet50_2
    layers:
    - layer2
    - layer3
    pre_trained: true
    coreset_sampling_ratio: 0.1
    num_neighbors: 9
    pre_processor: true
    post_processor: true
    evaluator:
      class_path: anomalib.metrics.Evaluator
      init_args:
        test_metrics:
        - class_path: anomalib.metrics.F1Score
          init_args:
            fields: ["pred_score", "gt_label"]

Custom Evaluation

Anomalib handles the visualization for you, using the available fields in your dataset. In case you want some customization, here is how you could do that.

via API

from anomalib.visualization import ImageVisualizer

visualizer = ImageVisualizer(
    fields=["image", "anomaly_map"],
    overlay_fields=[("image", ["anomaly_map"]), ("image", ["pred_mask"])],
)

model.visualizer = visualizer

# The rest is the same..

via CLI

You can add the custom visualizer config to the model config:

model:
  class_path: anomalib.models.Patchcore
  init_args:
    backbone: wide_resnet50_2
    layers:
    - layer2
    - layer3
    pre_trained: true
    coreset_sampling_ratio: 0.1
    num_neighbors: 9
    pre_processor: true
    post_processor: true
    evaluator:
      class_path: anomalib.metrics.Evaluator
      init_args:
        test_metrics:
        - class_path: anomalib.metrics.F1Score
          init_args:
            fields: ["pred_score", "gt_label"]
    visualizer:
      class_path: anomalib.visualization.ImageVisualizer
      init_args:
        fields: ["image", "anomaly_map"]
        overlay_fields:
        - - image
          - [anomaly_map]
        - - image
          - [pred_mask]

[1713mz]

Hello, I've got another problem here (prob. related). If I would like to train the model via CLI using a config file, how should I specify multiple metrics I want for test and for validation? E.g., in the previous release one can simply add following lines into config file:

metrics:
image:

• AUPR
• AUROC
  pixel:
• AUPR
• AUROC
• PRO

How should I then modify the evaluator module in the config file in 2.0.1 to get the same result?

[samet-akcay]

Let me know if there are other parts you need clarification. Thanks!

[wenwu2021]

datamodule = Folder(
name="hazelnut",
normal_dir="train/good",
root="datasets/MVTec/hazelnut",
)
Can we train with only good samples?

[samet-akcay]

Yes, when you create Folder datamodule, you could create synthetic samples by passing test_split_mode=TestSplitMode.SYNTHETIC