Clarification on reload_dataloaders_every_epoch

[thingsofleon]

I have a PyTorch Lightning DataModule instance that defines train_dataloader, val_dataloader, and test_dataloader.

Currently using a custom callback to reload the train_dataloader that will resample the data.

I saw that there is a Trainer flag called reload_dataloaders_every_epoch and soon to be reload_dataloaders_every_n_epochs.

Do these just reload the train_dataloader, or do the do all 3?

[carmocca]

Only the train and validation dataloaders:

https://github.com/PyTorchLightning/pytorch-lightning/blob/e4f3a8d3dd534d4ec2fe094280272513e652fba9/pytorch_lightning/trainer/training_loop.py#L168-L170

https://github.com/PyTorchLightning/pytorch-lightning/blob/e4f3a8d3dd534d4ec2fe094280272513e652fba9/pytorch_lightning/trainer/training_loop.py#L203-L207

[dianemarquette]

@carmocca Sorry to bother but I'm still a bit confused regarding the reload_dataloaders_every_epoch flag.

From the code snippets you've included above, I understand why the train dataloader is reset by the flag. However, to me, lines 206-207 only reset the validation dataloader if it wasn't previously defined and when the reload_dataloaders_every_epoch flag is set to False. If the reload_dataloaders_every_epoch flag really resets both dataloaders, wouldn't it be coded as?

# reset train dataloader 
if epoch != 0 and self.trainer.reload_dataloaders_every_epoch: 
    self.trainer.reset_train_dataloader(model)
    self.trainer.reset_val_dataloader(model)      # added line

Thanks in advance for your help :)

[carmocca]

This has changed since my last reply, these are the new relevant pieces:

pytorch_lightning//trainer/evaluation_loop.py:63:            if self.trainer.val_dataloaders is None or self.trainer.reload_dataloaders_every_epoch:
pytorch_lightning//trainer/training_loop.py:155:        if epoch != 0 and self.trainer.reload_dataloaders_every_epoch:

Note that the training loop calls the evaluation loop to run validation

[dianemarquette]

Thanks for the clarification! I hadn't seen the line of code in the evaluation loop. It's crystal-clear now :)