CUDA out of memory. when python tools/train.py on gpu

[cyj95]

python tools/train.py --model_config_path anomalib/models/patchcore/config_cloth.yaml --model patchcore

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 7.79 GiB total capacity; 5.42 GiB already allocated; 23.50 MiB free; 5.43 GiB reserved in total by PyTorch)
Epoch 0: 62%|██████▏ | 878/1417 [00:15<00:09, 56.28it/s]

[alexriedel1]

try: coreset_sampling_ratio: 0.01 in your config file

[cyj95]

try: coreset_sampling_ratio: 0.01 in your config file

still not solved

[alexriedel1]

limit_train_batches: 0.5 then might solve it.

[svmaskery]

Running into the same issue even though the train batch size is 1 and the coreset sampling ratio is 0.01 while using PatchCore.

RuntimeError: CUDA out of memory. Tried to allocate 1.28 GiB (GPU 0; 2.95 GiB total capacity; 1.55 GiB already allocated; 174.75 MiB free; 1.61 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[alexriedel1]

Running into the same issue even though the train batch size is 1 and the coreset sampling ratio is 0.01 while using PatchCore.

RuntimeError: CUDA out of memory. Tried to allocate 1.28 GiB (GPU 0; 2.95 GiB total capacity; 1.55 GiB already allocated; 174.75 MiB free; 1.61 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

train batch size 1 means 100% of the data, try batch size 0.1

[svmaskery]

The default values were 32. So when you say to change the batch size to 0.1, it doesn't make sense. Batch size is usually the number of images that go into the gpu.

[alexriedel1]

Ah, I'm talking about the hyperparameter limit_train_batches.
This will decrease the number of Training Samples and thus reduce Memory consumption. But anyways I guess 3GB RAM is just a bit too small training patchcore in a reasonable way with lots of data