Process the DataLoaders in sequential order in training loop

[marcmk6]

Hi,

According to doc, it's possible to make lightning running through data loaders sequentially in val/test/predict loops:

def val_dataloader(self):
    loader_1 = DataLoader(dataset_1, collate_fn=collate_fn_1)
    loader_2 = DataLoader(dataset_2, collate_fn=collate_fn_2)
    return [loader_1, loader_2]


def validation_step(self, batch, batch_idx, dataloader_idx):
    ...

I'm wondering how to do this in training loops.

loader_1 = DataLoader(training_set_1, collate_fn=collate_fn_1)
loader_2 = DataLoader(training_set_2, collate_fn=collate_fn_2)

for loader in [loader_1, loader_2]:
    for batch in loader:
        # training

In other words, I'm wondering how to implement the logic above in lightning.

Thanks

[ndahiya3]

val/test/predict loops automatically do that already in lightning. If you pass a list of dataloaders to val/test/predict loops, you can check that the dataloader_idx will go from 0 to max_dataloders - 1 in sequence (print dataloader_idx in validation_step with each dataloader having a small number of batches). That is, it will first go through all the batches of first dataloader, then second dataloader and so on.

I think this applies to val/test/predict loops only and NOT to train loop. Train loop combines batches from all dataloaders even if they are passed as a list. I think there might be a place to pass in a 'sequential' parameter to get the train loop to all iterate over the dataloaders sequentially as well but I am not sure.