Custom Data Set : KeyError:'label'

[jasperZoinks]

Hello,
I'm trying to create a custom data set for your library.

However I'm having some trouble with this.

The error i got:
keyError

This is the config file I created:
ownConfigFile_Padim.txt

I'm working in Google Colab.

Thanks!

[preste-viunt]

I have the same problem

[samet-akcay]

Hi @jasperZoinks, I see that you set task to segmentation but don't provide mask folder. You could do one of the two

dataset:
  name: walNUT                                          
  format: folder                                        
  path: /content/drive/MyDrive/masterIIW/nutsDataSet/training 
  normal: normal # name of the folder containing normal images.
  abnormal: abnormal # name of the folder containing abnormal images.
->task: segmentation # classification or segmentation   
->mask: path/to/your/mask/annotation
  extensions: null
  split_ratio: 0.2  # ratio of the normal images that will be used to create a test split
  seed: 0
  image_size: 256
  train_batch_size: 32
  test_batch_size: 32
  num_workers: 8
  transform_config: null
  create_validation_set: true
  tiling:
    apply: false
    tile_size: null
    stride: null
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

dataset:
  name: walNUT                                          
  format: folder                                        
  path: /content/drive/MyDrive/masterIIW/nutsDataSet/training 
  normal: normal # name of the folder containing normal images.
  abnormal: abnormal # name of the folder containing abnormal images.
->task: classification # classification or segmentation   
->mask: null #optional
  extensions: null
  split_ratio: 0.2  # ratio of the normal images that will be used to create a test split
  seed: 0
  image_size: 256
  train_batch_size: 32
  test_batch_size: 32
  num_workers: 8
  transform_config: null
  create_validation_set: true
  tiling:
    apply: false
    tile_size: null
    stride: null
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

It would be safer if we check this condition. We'll create a PR for this

[jasperZoinks]

Thank you very much!
It's working properly now!

[samet-akcay]

Hi @preste-viunt, can you share your config file so I could have a look?

[preste-viunt]

Hello ! Thanks for quick reply !
Here it i:

dataset:
format: folder
name: mvtec_cleaned
path: "mvtec_cleaned/bottle"
normal: "support_set/good"
abnormal: "support_set/def"
category: bottle
task: classification
mask: null #"support_set/ground_truth"
extensions: null
split_ratio: 0.3
seed: 42
image_size: 256
train_batch_size: 16
test_batch_size: 1
num_workers: 36
transform_config:
create_validation_set: true
tiling:
apply: true
tile_size: 16
stride: null
remove_border_count: 0
use_random_tiling: False
random_tile_count: 16

model:
name: patchcore
backbone: wide_resnet50_2
layers:
- layer2
- layer3
coreset_sampling_ratio: 0.1
num_neighbors: 9
metric: auc
weight_file: weights/model.ckpt
normalization_method: min_max # options: [null, min_max, cdf]
threshold:
image_default: 0
pixel_default: 0
adaptive: true

project:
seed: 0
path: ./results
log_images_to: [local]
logger: false
save_to_csv: false

trainer:
accelerator: null
accumulate_grad_batches: 1
amp_backend: native
auto_lr_find: false
auto_scale_batch_size: false
auto_select_gpus: false
benchmark: false
check_val_every_n_epoch: 1 # Don't validate before extracting features.
checkpoint_callback: true
default_root_dir: null
deterministic: false
fast_dev_run: false
gpus: 1
gradient_clip_val: 0
limit_predict_batches: 1.0
limit_test_batches: 1.0
limit_train_batches: 1.0
limit_val_batches: 1.0
log_every_n_steps: 50
log_gpu_memory: null
max_epochs: 1
max_steps: -1
min_epochs: null
min_steps: null
move_metrics_to_cpu: false
multiple_trainloader_mode: max_size_cycle
num_nodes: 1
num_processes: 1
num_sanity_val_steps: 0
overfit_batches: 0.0
plugins: null
precision: 32
prepare_data_per_node: true
process_position: 0
profiler: null
progress_bar_refresh_rate: null
replace_sampler_ddp: true
stochastic_weight_avg: false
sync_batchnorm: false
terminate_on_nan: false
tpu_cores: null
track_grad_norm: -1
val_check_interval: 1.0 # Don't validate before extracting features.
weights_save_path: null
weights_summary: top

Also, this algorithm works only for one class ? how to make so it classifyies defect or not for several classes (in mvtec there are 15 classes and each defected or not) ?
Thanks in advance!!

[preste-viunt]

Looks like I found, my positive and negative classes were called differently that's why had to change 'normal' and 'abnormal' in the code