Trainer not optimizing weights

[cwoolfo1]

Hello, I am running code to train a stacked GRU. I have defined and created train and validation data loaders. I initialized the trainer but when I run the code the accuracies for the validation loop and train loop don't change, they stay the same. I have compared my code to the documentation and I cannot seem to find the issue.

class DataModule(p1.LightningDataModule):
    def __init__(self):
        super().__init__()
        self.input = np.load('4thin.npy')
        self.output = np.load('4thout.npy')
        # Loads in npy files
        self.input = self.input[:,:,:10]
        self.output = self.output[:,:,:8]
        self.output = self.output.squeeze()
        self.batch_size = 1
        self.num_workers = 1
        # Set number of samples per batch
        
    def setup(self, stage):
        if stage == 'fit':
            split = len(self.input[:,1,1]) / 2
            split = int(split)
            self.trainX, self.trainy = self.input[:split, :, :], self.output[:split, :]
            self.valX, self.valy = self.input[split:, :, :], self.output[split:, :]
            # Splits the dataset into two smaller datasets for training.
            
        if stage == 'test' or stage is None:
            self.testX, self.testy = self.input, self.output
            # Returns the full dataset for testing.
            
    def train_dataloader(self):
        data = self.trainX, self.trainy
        return DataLoader(CustomDataset(data), batch_size = self.batch_size, num_workers=self.num_workers)
    # Loads data for training.
    
    def val_dataloader(self):
        data = self.valX, self.valy
        return DataLoader(CustomDataset(data), batch_size = self.batch_size, num_workers=self.num_workers)
    # Loads data for validation.
    
    def test_dataloader(self):
        data = self.testX, self.testy
        return DataLoader(CustomDataset(data), batch_size = self.batch_size, num_workers=self.num_workers)
    # Loads data for testing.        

parser = ArgumentParser()
parser.add_argument("--gpus", default=0)
parser.add_argument("--num_nodes", default=1)
args = parser.parse_args()
dm = DataModule()
dm.setup('fit')
train_dataloader = dm.train_dataloader()
val_dataloader = dm.val_dataloader()
model = NeuralNetwork(10, 10, 8, 2,)
# Creates a Stacked GRU model of the form (feature size, hidden size, output size, # of GRU layers)
tb_logger = p1_loggers.TensorBoardLogger("logs/")
# Defines the logger as a tensor board logger for training.
n = 1
trainer = Trainer(num_nodes = 1, gpus = args.gpus, logger=tb_logger, accumulate_grad_batches=1, reload_dataloaders_every_n_epochs=n) #, strategy="ddp")
# Uses tensorboard to log metrics and gradient accumulation to speed up processes.
trainer.fit(model, train_dataloader, val_dataloader)

Attached is the code I used to define my data module class and initialize and train my model. Is there any advice y'all can offer me for fixing this issue?

[rohitgr7]

can you share the code for NeuralNetwork?

[rohitgr7]

you are doing a softmax here:

x = F.softmax(self.outputlayer(x), dim=1)

and passing the probabilities to mseloss. is this correct?

[cwoolfo1]

Yes

[rohitgr7]

your code looks okay to me.. not sure what's causing the issue. is training and validation loss reducing with time?

[cwoolfo1]

No it is not. It goes from epoch to epoch but the training and validation loss do not change.

[rohitgr7]

oh! can you compile a small reproducible example in a colab notebook? I'll check why is it not training :)