Train different models in parallel

[aliutkus]

Hello, thanks for the awesome work

I would like to train several models in parallel, that would ideally share the same dataloader, but have non shared parameters, and each one its loss.

Could be implemented like passing a list to a trainer somehow.

Giving a bit of context:

• Having just unrelated processes run in parallel would not be an option, because I additionally want the different models to communicate during their training, e.g. through some multiprocessing.Queue object
• Please notice I want to have several models trained (possibly one one single gpu), and not a single model trained on multiple gpus, which I understand is easily done through pl

Is there a way to do that ?

thanks