[Bug]: Error in training own dataset #1011
Replies: 2 comments 5 replies
-
|
I had the same problem and it is because of the mask. You are supposed to provide masks for anomalous images (for metrics I suppose) and there is an error because you didn't. What I did is to set the mask to zero in src/anomalib/data/base/dataset.py", (line 133) by changing it to "mask = np.zeros(shape=image.shape[:2])". Keep in mind that by doing this your pixel metrics will be screwed |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much ! Thank you very much for your answer, and subsequently I have located the error here. My solution idea is essentially the same as yours. I was also interested in the ". /data/base/dataset.py", except that I directly changed line 118 of the code to "label_index =0". I'm not sure if there are any potential risks in doing this yet, but I did succeed. HAHA |
Beta Was this translation helpful? Give feedback.
-
|
If you don't have masks, you could set the task to task: classification # classification or segmentationAs @ZizmanTK, pointed out, this is to evaluate the performance of the model. To avoid metric computation issues, you could remove the pixel metrics from your config file. metrics:
image:
- F1Score
- AUROC
# or whichever metric you prefer.
threshold:
method: adaptive #options: [adaptive, manual]
manual_image: null
manual_pixel: null |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for your suggestion, it worked when I adopted @ZizmanTK's suggestion, and it also worked when I changed the task to "classification". Thank you both again for your help. But I am still very confused, my loss function keeps showing Also, when you say I look forward to hearing from you, and thank you very much for providing us with such an excellent and well-defined tool as |
Beta Was this translation helpful? Give feedback.
-
|
From what I know the padim model doesn't actually do any further training of pre-trained models. It just uses the weigths to get patchwise statistical representations of normal images. So it is normal to have no loss information. |
Beta Was this translation helpful? Give feedback.
-
|
Exactly, this is the main reason of getting NaN. There is no loss to report, so PyTorch Lightning logger logs NaN value. |
Beta Was this translation helpful? Give feedback.
-
|
Hello, thank you very much for all your help in the last week, this week I found a strange problem when training data with Please take a look at the following graph, it may have some Chinese information, but it does not affect the observation of CPU and GPU calls. How can I solve this problem? As you can see from the graph, my CPU takes up a lot of memory during training, but the GPU utilization is 0%, which I think is due to the fact that I am not calling CUDA correctly. This is also the case when I change the "num_workers" to 1. I am looking forward to your answer, thank you very much!
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Describe the bug
I have successfully reproduced the relevant code under the MVTec dataset using the relevant model. But when I am ready to train with my own dataset it reports an error after a little training time
Before training my own dataset I did the following
1: I created a folder called "Mydatasets" in the tools folder of the whole project, which holds all my normal and abnormal images. The structure of this folder is as follows（I have strictly followed the MVTec AD data format）:
2: Modified the corresponding yaml (I used the Padim model)，My yaml file is as follows: please note that I only modified the "dataset" part of the yaml as required
It is worthwhile to note that the official description of what changes should be made to the dataset in the yaml file when training your own dataset. The official modifications to the dataset do not include "normalization", "test_split_mode", "test_split_ratio", "val_split_mode" and "val_split_ratio". ratio", "val_split_mode" and "val_split_ratio" (which correspond to the last five lines of the Yaml I provided). But if I don't add them, it will report an error. So the yaml I provided is based on the official one with these five lines, and the corresponding path is modified to the Yaml.
At the time of the final training, he did run successfully for a while at first, as follows:
But later it will just report an error
Dataset
Other (please specify in the text field below)
Model
PADiM
Steps to reproduce the behavior
1: git latest version of anomalib
2: pip install -e .
OS information
MAC OS
python == 3.8
pytorch == 2.0.0
Expected behavior
I would like to be told the reason for this error, I still think there is something wrong with my Yaml setup. So please help me to see the exact cause of the error and the solution. Thank you very much!
I don't think the error is from the third-party library version because I have successfully run my code on MVTec AD
Screenshots
No response
Pip/GitHub
pip
What version/branch did you use?
latest
Configuration YAML
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions