Meaning of parameters in config file

[ikun0124]

What do the test_split_mode, test_split_ratio, val_split_mode, val_split_ratio parameters in the dataset section of the config file in anomalib version 1.0.0 represent? I can't get the desired replication results.

[samet-akcay]

Hi @loyal-ikun, can you elaborate desired replication results?

[ikun0124]

I was running anomaly detection experiments in my field on version 0.3.7 and got some excellent result. But when I downloaded the latest version 1.0.0, I used the same model and config model parameters and got poor results. I suspect it is due to wrong setting of test_split_mode, test_split_ratio, val_split_mode, val_split_ratio parameters values.

[samet-akcay]

can you share your config file that you use with 0.3.7 and 1.0.0dev please? Maybe we could try to reproduce what you did.

[ikun0124]

This is 0.3.7 version file:

dataset:
  name: folder
  format: folder
  path: 
  normal_dir: normal_train
  abnormal_dir: abnormal 
  normal_test_dir: normal_test
  task: classification
  mask: null
  extensions: null
  split_ratio: null
  image_size: 64
  train_batch_size: 256
  test_batch_size: 256
  inference_batch_size: 256
  num_workers: 8
  transform_config:
    train: null
    val: null
  create_validation_set: false
  tiling:
    apply: true
    tile_size: 64
    stride: null
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

model:
  name: ganomaly
  latent_vec_size: 210
  n_features: 64 
  extra_layers: 0
  add_final_conv: true
  early_stopping:
    patience: 8
    metric: image_AUROC
    mode: max
  lr: 0.0001
  beta1: 0.5
  beta2: 0.999 
  wadv: 1
  wcon: 60  
  wenc: 1
  normalization_method: none

metrics:
  image:
    - F1Score
    - AUROC
    - Accuracy
    - Precision
    - Recall
  threshold:
    method: adaptive #options: [adaptive, manual]
    manual_image: null

visualization:
  show_images: False # show images on the screen
  save_images: False # save images to the file system
  log_images: False # log images to the available loggers (if any)
  image_save_path: null # path to which images will be saved
  mode: full # options: ["full", "simple"]

project:
  seed: 42
  path: ./results

logging:
  logger: [csv] # options: [comet, tensorboard, wandb, csv] or combinations.
  log_graph: false # Logs the model graph to respective logger.

optimization:
  export_mode: null

# PL Trainer Args. Don't add extra parameter here.
trainer:
  accelerator: auto # <"cpu", "gpu", "tpu", "ipu", "hpu", "auto">
  accumulate_grad_batches: 1
  amp_backend: native
  auto_lr_find: false
  auto_scale_batch_size: false
  auto_select_gpus: false
  benchmark: false
  check_val_every_n_epoch: 2
  default_root_dir: null
  detect_anomaly: false
  deterministic: false
  devices: 1
  enable_checkpointing: true
  enable_model_summary: true
  enable_progress_bar: true
  fast_dev_run: false
  gpus: null # Set automatically
  gradient_clip_val: 0
  ipus: null
  limit_predict_batches: 1.0
  limit_test_batches: 1.0
  limit_train_batches: 1.0
  limit_val_batches: 1.0
  log_every_n_steps: 50
  log_gpu_memory: null
  max_epochs: 100 #100
  max_steps: null
  min_epochs: null
  min_steps: null
  move_metrics_to_cpu: false
  multiple_trainloader_mode: max_size_cycle
  num_nodes: 1
  num_processes: null
  num_sanity_val_steps: 0
  overfit_batches: 0.0
  plugins: null
  precision: 32
  profiler: null
  reload_dataloaders_every_n_epochs: 0
  replace_sampler_ddp: true
  strategy: null
  sync_batchnorm: false
  tpu_cores: null
  track_grad_norm: -1
  val_check_interval: 1.0

#####################################

This is 1.0.0 version file:

dataset:
  name: folder
  format: folder
  path: 
  normal_dir: normal_train
  abnormal_dir: abnormal 
  normal_test_dir: normal_test
  mask_dir: null
  extensions: null
  task: classification
  train_batch_size: 256
  eval_batch_size: 256
  inference_batch_size: 256
  num_workers: 8
  image_size: 64 # dimensions to which images are resized (mandatory)
  center_crop: null # dimensions to which images are center-cropped after resizing (optional)
  normalization: none # data distribution to which the images will be normalized: [none, imagenet]
  transform_config:
    train: null
    eval: null
  test_split_mode: from_dir # options: [from_dir, synthetic]
  test_split_ratio: 0 # fraction of train images held out testing (usage depends on test_split_mode)
  val_split_mode: same_as_test# options: [same_as_test, from_test, synthetic]
  val_split_ratio: 0 # fraction of train/test images held out for validation (usage depends on val_split_mode)
  tiling:
    apply: true
    tile_size: 64
    stride: null
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

model:
  name: k_trans
  latent_vec_size: 210
  n_features: 64
  extra_layers: 0
  add_final_conv: true
  early_stopping:
    patience: 8
    metric: image_AUROC
    mode: max
  lr: 0.0001
  beta1: 0.5
  beta2: 0.999
  wadv: 1
  wcon: 60
  wenc: 1
  normalization_method: none

metrics:
  image:
    - F1Score
    - AUROC
    - Accuracy
    - Precision
    - Recall
  threshold:
    method: adaptive #options: [adaptive, manual]
    manual_image: null

visualization:
  show_images: False # show images on the screen
  save_images: False # save images to the file system
  log_images: False # log images to the available loggers (if any)
  image_save_path: null # path to which images will be saved
  mode: full # options: ["full", "simple"]

project:
  seed: 42
  path: ./results

logging:
  logger: [] # options: [comet, tensorboard, wandb, csv] or combinations.
  log_graph: false # Logs the model graph to respective logger.

optimization:
  export_mode: null

# PL Trainer Args. Don't add extra parameter here.
trainer:
  enable_checkpointing: true
  default_root_dir: null
  gradient_clip_val: 0
  gradient_clip_algorithm: norm
  num_nodes: 1
  devices: 1
  enable_progress_bar: true
  overfit_batches: 0.0
  track_grad_norm: -1
  check_val_every_n_epoch: 2 # Don't validate before extracting features.
  fast_dev_run: false
  accumulate_grad_batches: 1
  max_epochs: 100
  max_steps: -1
  min_steps: null
  max_time: null
  limit_train_batches: 1.0
  limit_val_batches: 1.0
  limit_test_batches: 1.0
  limit_predict_batches: 1.0
  val_check_interval: 1.0 # Don't validate before extracting features.
  log_every_n_steps: 50
  accelerator: auto # <"cpu", "gpu", "tpu", "ipu", "hpu", "auto">
  strategy: null
  sync_batchnorm: false
  precision: 32
  enable_model_summary: true
  num_sanity_val_steps: 0
  profiler: null
  benchmark: false
  deterministic: false
  reload_dataloaders_every_n_epochs: 0
  auto_lr_find: false
  replace_sampler_ddp: true
  detect_anomaly: false
  auto_scale_batch_size: false
  plugins: null
  move_metrics_to_cpu: false
  multiple_trainloader_mode: max_size_cycle

[chenmo2]

Hello author, I have encountered a problem that cannot load verification sets normally for verification, how should I solve this problem