Using LightningDataModule with custom data #19724
Unanswered
asusdisciple
asked this question in
Lightning Trainer API: Trainer, LightningModule, LightningDataModule
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
So at the moment I am working a lot with my own custom data. Of course I transferred that data into the classic pytorch dataset, but since I am using Lightning I wondered if there should be a way to use your custom data inside the LightningDataModule without relying on Pytorch Datasets.
Why?
Because for custom data I now have one custom class to create the pytorch dataset, which is then fed into my LightningDataModule to be used for training, testing etc. . This makes two classes for a task which could be handled by one. You could argue I would be able to introduce my pytorch datasetclass in the lightningdatamodule, however with a more complex structure, this makes it necessary to reimplement len() , getitem() and init() as well.
Is there a better workflow for this? Currently my datasets usually consist of a datafolder and .csv files with labels which have to be read in.
Beta Was this translation helpful? Give feedback.
All reactions