Generating new data while training.

[pjovanovski1]

Hello!

I'm working on an inference engine where I don't have any data before I start training. The idea is that at each epoch, a new data point is generated by a simulator and that data is fed to the NN only at that epoch.

I'm wondering if there is a way to do this with pytorch-lightning. I looked at the IterableDataset class but the only thing I came up with is to generate the data beforehand and then iterate over it.

Thanks!

[edgarriba]

hi @pjovanovski1 in this case you could follow a similar strategy proposed in this example of lightning-bolts for RL
https://github.com/PyTorchLightning/lightning-bolts/blob/master/pl_bolts/models/rl/reinforce_model.py#L254

[pjovanovski1]

Hi @edgarriba. Thanks for your reply. This is what I was looking for.