Integrating with PyTorch Lightning

[jaywonchung]

Hi,

I'm seeing if it's possible to integrate adaptdl with PyTorch Lightning (specifically the Deepspeech2 open source repo). Potential problems I see are:

1. Would the adaptdl specific the dataloader and the model be compatible?
2. Also, what should I do about the remaining_epochs_until iterator? Or, if I give up the remaining_epochs_until iterator and stop training at a specific validation metric, would that work?

Thanks a lot.

[aurickq]

Hi! I think integrating with Lightning would be a great idea. For your two questions:

1. Not sure about Lightning, but from our experience integrating with other frameworks, this is a likely area of friction. Usually, the model may be integrated just fine, but the data loading logic is often very diverse between different frameworks and require s few work-arounds.
2. I think the remaining_epochs_until iterator is still required, though in the future AdaptDL should relax this constraint. You should still be able to break out of the loop at any time though, i.e. when a certain validation metric is reached.

[Maximilian6]

@jaywonchung Did you make it work and did you publish it? I am interested in this as well.

[jaywonchung]

Nah, I just reverted to an older version of Deepspeech2 that didn't use PyTorch Lightning and integrated adaptdl there.

[Maximilian6]

Too bad, but thank you for the fast reply :)