Select randomly percent of datasets in each percent of epochs

[kobrafarshidi]

I want to train my model with 4000 datas in 4 epochs. I want to train with percentage of datasets for percentage of epochs .For example it trains 2 the first of epoch only 2000 random data, then for 2 next epoch, 2000 random data.How can I do that? Can I do with changing dataloader or other way?

class DataModule(pl.LightningDataModule):

  def __init__(self, train_dataset, val_dataset,  batch_size = 2):

    super(DataModule, self).__init__()
    self.train_dataset = train_dataset
    self.val_dataset = val_dataset
    self.batch_size = batch_size

  def train_dataloader(self):
    return DataLoader(self.train_dataset, batch_size = self.batch_size, 
                      collate_fn = collate_fn, shuffle = True, num_workers = 2, pin_memory = True)
  
  def val_dataloader(self):
    return DataLoader(self.val_dataset, batch_size = self.batch_size,
                    collate_fn = collate_fn, shuffle = False, num_workers = 2, pin_memory = True)

[akihironitta]

You can reload your dataloaders every n epochs by specifying the following trainer argument. In your case, it'll look like this since you want to update your dataloaders every two epochs:

Trainer(..., reload_dataloaders_every_n_epochs=2)

See https://pytorch-lightning.readthedocs.io/en/1.8.3/common/trainer.html#reload-dataloaders-every-n-epochs

[kobrafarshidi]

Hi, Thank you so much for your relpy.
I have seen this page before but I can,t use that code because I will encouter 2 problem.
1) That code has 3 parts that contain datamodule,model and trainer and I write this codes in below and I don't know where to define this part that this page mention
*** Source that this site mention

for epoch in epochs:
    if not epoch % reload_dataloaders_every_n_epochs:
        train_loader = model.train_dataloader()
        # or if using data module: datamodule.train_dataloader()
    for batch in train_loader:
        ...

• This is part of datamodule

class DataModule(pl.LightningDataModule):

  def __init__(self, train_dataset, val_dataset,  batch_size = 2):

    super(DataModule, self).__init__()
    self.train_dataset = train_dataset
    self.val_dataset = val_dataset
    self.batch_size = batch_size
  def train_dataloader(self):
    return DataLoader(self.train_dataset, batch_size = self.batch_size, shuffle = True)
  
  def val_dataloader(self):
    return DataLoader(self.val_dataset, batch_size = self.batch_size, shuffle = False)

• And this is part of model

class model(pl.LightningModule):
  def __init__(self, config , learning_rate = 1e-4, max_steps = 100000//2):
    super(model, self).__init__()
    
    self.model=  model(config)
    self.training_losses = []
    self.max_steps = max_steps

  def configure_optimizers(self):
    return torch.optim.AdamW(self.parameters(), lr = self.hparams['learning_rate'])


  def forward(self, batch_dict):
    img =     batch_dict['img']
   out_vector = self.model(image_vector = img  )
    return out_vector 

  def calculate_metrics(self, prediction, labels):
      batch_size = len(prediction)
      ac_score = 0
      for (pred, gt) in zip(prediction, labels):
        ac_score+= calculate_acc_score(pred.detach().cpu(), gt.detach().cpu())
      ac_score = ac_score/batch_size
      return ac_score

  def training_step(self, batch, batch_idx):
    answer_vector = self.forward(batch)
    loss = nn.CrossEntropyLoss()(out_vector .reshape(-1,self.config['classes']), batch['answer'].reshape(-1))
    _, preds = torch.max(out_vector , dim = -1)
    train_acc = self.calculate_metrics(preds, batch['out'])
    train_acc = torch.tensor(train_acc)
    self.training_losses.append(loss.item())

    return loss

• And this is part of trainer

trainer = pl.Trainer(
    max_steps = max_steps,
    reload_dataloaders_every_n_epochs=2
)

2) As it is known, this line of code "reload_dataloaders_every_n_epochs" reloads the dataloader every 2 epochs. My question is, when I was able to use this code, how does it use percent of random data for every 2 epoch ؟Because my goal is to give only a fraction of the data for every 2 epoch?I think I don't know how to change it.

[akihironitta]

1. I'm not sure what you're asking. Would you mind elaborating more on it?
2. You could change the logic in your dataset so that every time your dataloader gets reloaded, the indices iterated by each reloaded dataloader will be mutually exclusive. Something like:

class DataModule(pl.LightningDataModule):
  def __init__(self, ...):
    self.count = 0
  def train_dataloader(self):
    if self.count % 2 == 0:
      index = [0, 1999]
    else:
      index = [2000, 3999]
    self.count += 1
    train_dataset = YourDataset(..., index)
    return DataLoader(train_dataset, batch_size = self.batch_size, shuffle = True)