[Task]: How to execute the code for all the categories in MVTec in single run?

[GauravGajbhiye]

What is the motivation for this task?

I have modified the model and would like to have the results of all categories in single run instead of running the code for each category. Would you please provide the solution for this?

Describe the solution you'd like

I would like to have the result of all categories in single file (xml,yaml, etc.).

Additional context

No response

[samet-akcay]

@GauravGajbhiye, there are two alternative approach for this:

CLI

you could use the benchmarking script for this purpose. For example, the following cli command would benchmark Padim and Patchcore on the MVTec AD categories.

python tools/benchmarking/benchmark.py --config tools/benchmarking/benchmark_params.yaml

API

If you want to do this via API, you could do something like the following by modifying the tools/train.py file:

    # Add the following to the existing imports
    from anomalib.data.mvtec import CATEGORIES
    #...other imports


    for category in CATEGORIES:
        config.dataset[category] = category

        datamodule = get_datamodule(config)
        model = get_model(config)
        experiment_logger = get_experiment_logger(config)
        callbacks = get_callbacks(config)

        trainer = Trainer(**config.trainer, logger=experiment_logger, callbacks=callbacks)
        trainer.fit(model=model, datamodule=datamodule)
        trainer.test(model=model, datamodule=datamodule)

I wrote this without testing though, so it would be worth to double check