trainer.test( ) not working

[purnasai-soulpageit]

I have used the below code

1. Dataclass below
   

2. Model Class below
   

3. Trainnig below
   

I have trained the model for 20 epochs, After that trainer. test( ) didn't run. I Got the Following error lines :

I have also tried the way suggested in Doc here But still, the error exists. This happening mostly in Google collab and in AwsInstance. In Local, It works sometimes and sometimes not.
Please let me know what am I missing here.

[rohitgr7]

hey @purnasai-soulpageit !

you need to pass in the datamodule to trainer.test.

trainer.test(datamodule=data_module)

since lightning no longer patches the data_modules passed during .fit you need to pass it there too.

[purnasai-soulpageit]

Hi @rohitgr7 I tried passing all including datamodule, ckpt_path earlier, but then for some reason, they threw me an error. I just checked again passing just datamodule and it worked.
Thanks

[rohitgr7]

great!
if you pass in ckpt_path now, does it fail?

[abhayraw1]

Recently update pl to 1.5.10 and i'm facing this issue. I'm using LightningCLI and I call trainer.test(self.model, datamodule=self.datamodule) inside the Trainer's after_fit method to run a evaluation at the end of training.
I launch my code using the following:

python trainer_contrastive.py fit --config configs/default.yaml --trainer configs/default_trainer.yaml --model configs/models/contrastive.yaml --data configs/data/office_home/product.yaml --exp_id delete_later

The training works but it throws the same exception when it reaches the after_fit method.

Traceback (most recent call last):
  File "trainer_contrastive.py", line 9, in <module>
    cli = Trainer(
  File "/mnt/sqnap_colomirror/abrawat/projects/transDA/pl_transda/trainer.py", line 17, in __init__
    super().__init__(
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/utilities/cli.py", line 528, in __init__
    self._run_subcommand(self.subcommand)
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/utilities/cli.py", line 787, in _run_subcommand
    after_fn()
  File "/mnt/sqnap_colomirror/abrawat/projects/transDA/pl_transda/trainer.py", line 112, in after_fit
    self.trainer.test()
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 911, in test
    return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 954, in _test_impl
    results = self._run(model, ckpt_path=self.tested_ckpt_path)
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1128, in _run
    verify_loop_configurations(self)
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/trainer/configuration_validator.py", line 42, in verify_loop_configurations
    __verify_eval_loop_configuration(trainer, model, "test")
  File "/mnt/sqnap_colomirror/abrawat/envs/pl_1.5.1/lib/python3.8/site-packages/pytorch_lightning/trainer/configuration_validator.py", line 186, in __verify_eval_loop_configuration
    raise MisconfigurationException(f"No `{loader_name}()` method defined to run `Trainer.{trainer_method}`.")
pytorch_lightning.utilities.exceptions.MisconfigurationException: No `test_dataloader()` method defined to run `Trainer.test`.

I tried passing the test_dataloaders manually but to no avail.

PS: I don't want to manually go and run a python trainer_contrastive test ... with the best checkpoint for the experiment/ Is that possible in after_fit?

[kxding]

Do you solve it now? i face the same problem

[jxtngx]

@kxding can you please post your stack trace. and, which version of lightning or pytorch lightning are you using? thank you 🙂

[kxding]

pytorch-lightning 1.5.9 and the warning like this raise MisconfigurationException(f"No{loader_name}()method defined to runTrainer.{trainer_method}.") pytorch_lightning.utilities.exceptions.MisconfigurationException: No test_dataloader()method defined to runTrainer.test.

[jxtngx]

have you passed in a dataloader as shown here: https://lightning.ai/docs/pytorch/1.5.9/common/trainer.html#test

[jxtngx]

Also, are you pinned to 1.5.9 or can you update to >= 2.0?

[kxding]

yeah， i updated it to 2.0，and it works