How do I use two Dataloaders during training?

[Breeze-Zero]

Recently, I have tried semi-supervised learning training, and the current scenarios are as follows: First, import labeled data for training, then import unlabeled data for training, it is easy to implement two dataloader in series under normal circumstances, but it seems to be complicated in Pytorch -lighting, is there any good solution?

[rohitgr7]

First, import labeled data for training, then import unlabeled data for training

do you mean import the send dataloader after certain epochs?

[Breeze-Zero]

Yes, it is somewhat similar. Take one epoch as an example, I first used the dataloader without labeling for training, and then changed it to the dataloader with labeling after training. I saw that there were similar requirements #12415 (comment), and he divided it into two epochs to realize, but I felt a little trouble at evey epoch redefined the Dataloader.

[rohitgr7]

you mean something like:

for epoch in range(max_epochs):
    dl_without_labels  = ...
    for batch in dl_without_labels:
        train_model()
    
    dl_with_labels = ...
    for batch in dl_with_labels:
        train_model()

[Breeze-Zero]

Yes, is there an efficient way to do it by Lightning?
edit: Maybe it's better this way

dl_without_labels  = ...
dl_with_labels = ...
for epoch in range(max_epochs):
    for batch in dl_without_labels:
        train_model()
    
    for batch in dl_with_labels:
        train_model()

[rohitgr7]

dl_without_labels  = ...
dl_with_labels = ...

we can think of maybe combining both of these somehow:

combined_dl = CyclicDataLoaders([dl_without_labels, dl_with_labels])
for epoch in range(max_epochs):
    for batch in combined_dl:
        train_model()

maybe this could work:

import itertools

def train_dataloader(self):
    dl_without_labels  = ...
    dl_with_labels = ...
    return itertools.chain(dl_without_labels, dl_with_labels)

but that might require you to set Trainer(reload_dataloaders_every_n_epochs=1) so that the iterator is initialized again.
haven't tried... do give it a try.