❓ What is the relationship between test_split_mode and val_split_mode？

[wenwu2021]

Your Question

class Folder(AnomalibDataModule):
"""Folder DataModule.

Args:
    name (str): Name of the dataset. Used for logging/saving.
    normal_dir (str | Path | Sequence): Directory containing normal images.
    root (str | Path | None): Root folder containing normal and abnormal
        directories. Defaults to ``None``.
    abnormal_dir (str | Path | None | Sequence): Directory containing
        abnormal images. Defaults to ``None``.
    normal_test_dir (str | Path | Sequence | None): Directory containing
        normal test images. Defaults to ``None``.
    mask_dir (str | Path | Sequence | None): Directory containing mask
        annotations. Defaults to ``None``.
    normal_split_ratio (float): Ratio to split normal training images for
        test set when no normal test images exist.
        Defaults to ``0.2``.
    extensions (tuple[str, ...] | None): Image extensions to include.
        Defaults to ``None``.
    train_batch_size (int): Training batch size.
        Defaults to ``32``.
    eval_batch_size (int): Validation/test batch size.
        Defaults to ``32``.
    num_workers (int): Number of workers for data loading.
        Defaults to ``8``.
    train_augmentations (Transform | None): Augmentations to apply dto the training images
        Defaults to ``None``.
    val_augmentations (Transform | None): Augmentations to apply to the validation images.
        Defaults to ``None``.
    test_augmentations (Transform | None): Augmentations to apply to the test images.
        Defaults to ``None``.
    augmentations (Transform | None): General augmentations to apply if stage-specific
        augmentations are not provided.
    test_split_mode (TestSplitMode): Method to obtain test subset.
        Defaults to ``TestSplitMode.FROM_DIR``.
    test_split_ratio (float): Fraction of train images for testing.
        Defaults to ``0.2``.
    val_split_mode (ValSplitMode): Method to obtain validation subset.
        Defaults to ``ValSplitMode.FROM_TEST``.
    val_split_ratio (float): Fraction of images for validation.
        Defaults to ``0.5``.
    seed (int | None): Random seed for splitting.
        Defaults to ``None``.

Example:
    Create and setup a folder datamodule::

        >>> from anomalib.data import Folder
        >>> datamodule = Folder(
        ...     name="custom",
        ...     root="./datasets/custom",
        ...     normal_dir="good",
        ...     abnormal_dir="defect",
        ...     mask_dir="mask"
        ... )
        >>> datamodule.setup()

    Get a batch from train dataloader::

        >>> batch = next(iter(datamodule.train_dataloader()))
        >>> batch.keys()
        dict_keys(['image', 'label', 'mask', 'image_path', 'mask_path'])

    Get a batch from test dataloader::

        >>> batch = next(iter(datamodule.test_dataloader()))
        >>> batch.keys()
        dict_keys(['image', 'label', 'mask', 'image_path', 'mask_path'])
"""

def __init__(
    self,
    name: str,
    normal_dir: str | Path | Sequence[str | Path],
    root: str | Path | None = None,
    abnormal_dir: str | Path | Sequence[str | Path] | None = None,
    normal_test_dir: str | Path | Sequence[str | Path] | None = None,
    mask_dir: str | Path | Sequence[str | Path] | None = None,
    normal_split_ratio: float = 0.2,
    extensions: tuple[str] | None = None,
    train_batch_size: int = 32,
    eval_batch_size: int = 32,
    num_workers: int = 8,
    train_augmentations: Transform | None = None,
    val_augmentations: Transform | None = None,
    test_augmentations: Transform | None = None,
    augmentations: Transform | None = None,
    test_split_mode: TestSplitMode | str = TestSplitMode.FROM_DIR,
    test_split_ratio: float = 0.2,
    val_split_mode: ValSplitMode | str = ValSplitMode.FROM_TEST,
    val_split_ratio: float = 0.5,
    seed: int | None = None,
) -> None:
    self._name = name
    self.root = root
    self.normal_dir = normal_dir
    self.abnormal_dir = abnormal_dir
    self.normal_test_dir = normal_test_dir
    self.mask_dir = mask_dir
    self.extensions = extensions
    test_split_mode = TestSplitMode(test_split_mode)
    val_split_mode = ValSplitMode(val_split_mode)
    super().__init__(
        train_batch_size=train_batch_size,
        eval_batch_size=eval_batch_size,
        num_workers=num_workers,
        train_augmentations=train_augmentations,
        val_augmentations=val_augmentations,
        test_augmentations=test_augmentations,
        augmentations=augmentations,
        test_split_mode=test_split_mode,
        test_split_ratio=test_split_ratio,
        val_split_mode=val_split_mode,
        val_split_ratio=val_split_ratio,
        seed=seed,
    )

      What is the relationship between   test_split_mode and  val_split_mode？

What is the relationship between test_split_ratio and val_split_ratio? Is the data get from this normal_test-dir
How to set ？

Forum Check

• I have searched the discussions forum for an answer to my question.

[github-actions[bot]]

Thanks for submitting this issue! It has been added to our triage queue. A maintainer will review it shortly.

[alfieroddan]

I think this a discussion not a bug, could you next time use the discussion tab?
https://github.com/open-edge-platform/anomalib/discussions

def setup(self, stage: str | None = None) -> None:
# ...
            self._setup(stage)
            self._create_test_split()
            self._create_val_split()
# ...

The Splits are called in this order. So to answer your question, first the test ratio is used to split the data. Then the val.

You can see the full code here: https://github.com/open-edge-platform/anomalib/blob/main/src/anomalib/data/datamodules/base/image.py#L288

[wenwu2021]

datamodule = Folder(
    name="bottle",
    root=r".\anomalib\anomalib-2.0.0\datasets\MVTecAD\bottle",
    normal_dir="train/good",
    abnormal_dir=None,
    train_batch_size=32,
    eval_batch_size=32,
    num_workers=0,
    normal_test_dir=None,
    mask_dir=None,
    normal_split_ratio=0.2,
    test_split_mode=TestSplitMode.SYNTHETIC,
    test_split_ratio=0.5,
    val_split_ratio=0.5,  # fraction of train/test images held out for validation (usage depends on val_split_mode)
    val_split_mode=ValSplitMode.FROM_TEST,

)

normal_dir Number of images are 100.
normal_split_ratio=0.2, 100 *0.2=20,
test_split_ratio=0.5, 100 *0.5=50, test_split_mode=TestSplitMode.SYNTHETIC, use 50?
val_split_mode=0.5 50 *0.5=25,val_split_mode=ValSplitMode.FROM_TEST

datamodule = Folder(
    name="bottle",
    root=r".\anomalib\anomalib-2.0.0\datasets\MVTecAD\bottle",
    normal_dir="train/good",
    abnormal_dir=None,
    train_batch_size=32,
    eval_batch_size=32,
    num_workers=0,
    normal_test_dir="test/good",,
    mask_dir=None,
    normal_split_ratio=0.2,
    test_split_mode=TestSplitMode.SYNTHETIC,
    test_split_ratio=0.5,
    val_split_ratio=0.5,  # fraction of train/test images held out for validation (usage depends on val_split_mode)
    val_split_mode=ValSplitMode.FROM_TEST,

)

normal_dir Number of images are 100.
normal_split_ratio=0.2, 100 *0.2=20,

normal_test_dir Number of images are 200.
test_split_ratio=0.5, 200 *0.5=100,test_split_mode=TestSplitMode.SYNTHETIC, use 100?
val_split_mode=0.5 200 *0.5=100,val_split_mode=ValSplitMode.FROM_TEST
is this?

[wenwu2021]

I think the is a discussion not a bug, could you next time use the discussion tab? https://github.com/open-edge-platform/anomalib/discussions

def setup(self, stage: str | None = None) -> None:
# ...
self._setup(stage)
self._create_test_split()
self._create_val_split()

...

The Splits are called in this order. So to answer your question, first the test ratio is used to split the data. Then the val.

You can see the full code here: https://github.com/open-edge-platform/anomalib/blob/main/src/anomalib/data/datamodules/base/image.py#L288

datamodule = Folder(
name="bottle",
root=r".\anomalib\anomalib-2.0.0\datasets\MVTecAD\bottle",
normal_dir="train/good",
abnormal_dir=None,
train_batch_size=32,
eval_batch_size=32,
num_workers=0,
normal_test_dir=None,
mask_dir=None,
normal_split_ratio=0.2,
test_split_mode=TestSplitMode.SYNTHETIC,
test_split_ratio=0.5,
val_split_ratio=0.5, # fraction of train/test images held out for validation (usage depends on val_split_mode)
val_split_mode=ValSplitMode.FROM_TEST,

)
normal_dir Number of images are 100.
normal_split_ratio=0.2, 100 *0.2=20,
test_split_ratio=0.5, 100 *0.5=50, test_split_mode=TestSplitMode.SYNTHETIC, use 50?
val_split_mode=0.5 50 *0.5=25,val_split_mode=ValSplitMode.FROM_TEST

datamodule = Folder(
name="bottle",
root=r".\anomalib\anomalib-2.0.0\datasets\MVTecAD\bottle",
normal_dir="train/good",
abnormal_dir=None,
train_batch_size=32,
eval_batch_size=32,
num_workers=0,
normal_test_dir="test/good",,
mask_dir=None,
normal_split_ratio=0.2,
test_split_mode=TestSplitMode.SYNTHETIC,
test_split_ratio=0.5,
val_split_ratio=0.5, # fraction of train/test images held out for validation (usage depends on val_split_mode)
val_split_mode=ValSplitMode.FROM_TEST,

)
normal_dir Number of images are 100.
normal_split_ratio=0.2, 100 *0.2=20,

normal_test_dir Number of images are 200.
test_split_ratio=0.5, 200 *0.5=100,test_split_mode=TestSplitMode.SYNTHETIC, use 100?
val_split_mode=0.5 200 *0.5=100,val_split_mode=ValSplitMode.FROM_TEST
is this?