Support training with only normal images,How to configure

[monkeycc]

#572

I only have normal pictures Diagram without defects

I want to automatically simulate defects
draem
anomaly_source_path:dtd

python tools/train.py --config anomalib/models/draemNO/config.yaml

# This is the config file for the new Anomalib CLI.
# To use this, run the following command:
# anomalib fit --config ./config/model.draem.yaml
ckpt_path: null
seed_everything: null

dataset:
  name: NAME
  format: folder
  path: F:/2022/
  normal_dir: OK # name of the folder containing normal images.
  abnormal_dir: null # name of the folder containing abnormal images.
  normal_test_dir: null # name of the folder containing normal test images.
  task: segmentation # classification or segmentation
  mask: null #optional
  extensions: null
  split_ratio: 0.2 # ratio of the normal images that will be used to create a test split
  image_size: 512
  train_batch_size: 8
  test_batch_size: 32
  num_workers: 8
  transform_config:
    train: null
    val: null
  create_validation_set: true
  tiling:
    apply: false
    tile_size: null
    stride: null
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16


model:
  class_path: anomalib.models.Draem
  init_args:
    anomaly_source_path: D:/0SDXX/dtd/ # optional, e.g. ./datasets/dtd

optimizer:
  class_path: torch.optim.Adam
  init_args:
    lr: 0.0001

post_processing:
  normalization_method: min_max # <null, min_max, cdf>
  threshold_method: adaptive # options: [adaptive, manual]
  manual_image_threshold: null
  manual_pixel_threshold: null

metrics:
  image_metrics:
    - F1Score
    - AUROC
  pixel_metrics:
    - F1Score
    - AUROC

visualization:
  show_images: False # show images on the screen
  save_images: True # save images to the file system
  log_images: False # log images to the available loggers (if any)
  mode: full # options: ["full", "simple"]

trainer:
  callbacks:
    - class_path: pytorch_lightning.callbacks.EarlyStopping
      init_args:
        patience: 50
        monitor: pixel_AUROC
        mode: max
  accelerator: auto # <"cpu", "gpu", "tpu", "ipu", "hpu", "auto">
  accumulate_grad_batches: 1
  amp_backend: native
  auto_lr_find: false
  auto_scale_batch_size: false
  auto_select_gpus: false
  benchmark: false
  check_val_every_n_epoch: 1
  default_root_dir: null
  detect_anomaly: false
  deterministic: false
  devices: 1
  enable_checkpointing: true
  enable_model_summary: true
  enable_progress_bar: true
  fast_dev_run: false
  gpus: null # Set automatically
  gradient_clip_val: 0
  ipus: null
  limit_predict_batches: 1.0
  limit_test_batches: 1.0
  limit_train_batches: 1.0
  limit_val_batches: 1.0
  log_every_n_steps: 50
  log_gpu_memory: null
  max_epochs: 100
  max_steps: -1
  max_time: null
  min_epochs: null
  min_steps: null
  move_metrics_to_cpu: false
  multiple_trainloader_mode: max_size_cycle
  num_nodes: 1
  num_processes: null
  num_sanity_val_steps: 0
  overfit_batches: 0.0
  plugins: null
  precision: 32
  profiler: null
  reload_dataloaders_every_n_epochs: 0
  replace_sampler_ddp: true
  strategy: null
  sync_batchnorm: false
  tpu_cores: null
  track_grad_norm: -1
  val_check_interval: 1.0

Traceback (most recent call last):
  File "tools/train.py", line 71, in <module>
    train()
  File "tools/train.py", line 49, in train
    config = get_configurable_parameters(model_name=args.model, config_path=args.config)
  File "D:\0SDXX\anomalib\anomalib\config\config.py", line 146, in get_configurable_parameters
    if config.project.get("seed") == 0:
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\dictconfig.py", line 355, in __getattr__
    self._format_and_raise(
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\base.py", line 231, in _format_and_raise
    format_and_raise(
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\_utils.py", line 900, in format_and_raise
    _raise(ex, cause)
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\_utils.py", line 798, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\dictconfig.py", line 351, in __getattr__
    return self._get_impl(
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\dictconfig.py", line 442, in _get_impl
    node = self._get_child(
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\basecontainer.py", line 73, in _get_child
    child = self._get_node(
  File "D:\anaconda31114\envs\anomalib_env\lib\site-packages\omegaconf\dictconfig.py", line 480, in _get_node
    raise ConfigKeyError(f"Missing key {key!s}")
omegaconf.errors.ConfigAttributeError: Missing key project
    full_key: project
    object_type=dict

[zhanghuayu-seu]

I have the same question, please help us solve the problem !!

[djdameln]

Hi, you could use the following dataset section in your config.yaml to train without anomalous images:

dataset:
  name: NAME
  format: folder
  path: F:/2022
  normal_dir: OK
  abnormal_dir: null
  normal_test_dir: null
  mask_dir: null
  extensions: null
  task: segmentation
  train_batch_size: 32
  eval_batch_size: 32
  num_workers: 8
  image_size: 256 # dimensions to which images are resized (mandatory)
  center_crop: null # dimensions to which images are center-cropped after resizing (optional)
  normalization: imagenet # data distribution to which the images will be normalized: [none, imagenet]
  transform_config:
    train: null
    eval: null
  test_split_mode: none # options: [from_dir, synthetic]
  test_split_ratio: 0.2 # fraction of train images held out testing (usage depends on test_split_mode)
  val_split_mode: synthetic # options: [same_as_test, from_test, synthetic]
  val_split_ratio: 0.5 # fraction of train/test images held out for validation (usage depends on val_split_mode)
  tiling:
    apply: false
    tile_size: null
    stride: null
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

By setting the test_split_mode to none, you will prevent Anomalib from trying to create a testing subset. Evaluation stage will be skipped because there are no anomalous samples that could be used to check the true positive rate of the model. It is recommended to set val_split_mode to synthetic, because this means that synthetic anomalous validation images will be generated from a part of the normal training data using Perlin noise augmentations. This helps the model to find a good anomaly score threshold that can be used during inference.

Let me know if this answers your question. We're happy to answer any additional questions that you may have.

[zhanghuayu-seu]

Thank you for your reply, it helps a lot !!!