Questions about the accuracy displayed in the 3D slicer after training the model

[Minxiangliu]

Thank you monailabel for providing a good tool for training the model. I have some original images and mask images that are not marked with 3D Slicer. I built the app according to the monailabel example, and I can also successfully train in 3D, but I want to know his accuracy What is the calculation based on, will he divide the train and test by himself?

[diazandr3s]

Thanks for opening this discussion, @Minxiangliu. We're glad you're using MONAI Label.
By default, MONAI Label takes 20% of the training set for validation. The accuracy you see is based on that.
This percentage can be changed in the Options tab in Slicer. Please watch this video to know more: https://youtu.be/KjwuFx0pTXU?t=64

[Minxiangliu]

Hi @diazandr3s ,
Thank you very much for your reply, I want to ask another question, since my data is very small (about 200 sets of 3D images), does monailabel have a cross validation method? Or is there no need to do this in monailabel? Thank you very much for your detailed explanation!

[diazandr3s]

Hi @Minxiangliu,

By default, there is no cross-validation. It just takes 20% of the training set and checks the accuracy. But you could certainly redefine this method: https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L327-L345
Just want to remind you that the sample app inherits the BasicTrainTask class: https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/trainers/segmentation.py#L42

[Minxiangliu]

ok, I guess changing the definition method might be a bit difficult for me but thanks for the tip.

[SachidanandAlle]

If you have your own datalist for validation (either json or pointing to json file) you can pass the same.. train_ds and val_ds
For example: https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/endoscopy/main.py#L315-L316

[Minxiangliu]

Hi @SachidanandAlle , According to your suggestion, if I want to use the example of radiology, do I need to change my json content before each training after adding the paths of train_ds and val_ds here(main.py)? Also, is there a json format for reference?

[diazandr3s]

Hi @Minxiangliu,

No, you don't need to change the JSON content. Just need to pass a list of dict for both train and validation. Please refer to this discussion: #990 (comment)