get fancy csv logger every epoch end

[JNaranjo-Alcazar]

Hi Community,
I am trying to create a fancy csv logger with this structure:

lr, epoch, acc, val_acc

I would like that the info is written every epoch end. So far, I have this code:

In the main script for training:

logger = CSVLogger("lightning_logs", name="my_exp_name")
lr_monitor = LearningRateMonitor(logging_interval='epoch')
# Train
trainer = pl.Trainer(progress_bar_refresh_rate=20, max_epochs=5, 
                     logger=logger, callbacks=[lr_monitor])

Regarding the model class:

def training_step(self, batch, batch_idx):
        x, y = batch
        # forward, logits
        logits = self(x)
        acc = accuracy(logits, y)
        J = self.loss(logits, y)
        self.log("accuracy",  acc, prog_bar=True)
        return {"loss": J, "acc": acc}
    
    def training_epoch_end(self, outputs):
        avg_loss = torch.tensor([x["loss"] for x in outputs]).mean()
        avg_accuracy = torch.tensor([x["acc"] for x in outputs]).mean()
        self.log("acc_epoch", avg_accuracy, prog_bar=True)
        
    
    def validation_step(self, val_batch, batch_idx):
        return self.training_step(val_batch, batch_idx)

    def validation_epoch_end(self, outputs):
        avg_loss = torch.tensor([x["loss"] for x in outputs]).mean()
        avg_accuracy = torch.tensor([x["acc"] for x in outputs]).mean()
        self.log("val_acc_epoch", avg_accuracy, prog_bar=True)
        return {"val_loss": avg_loss}

The result I get in the csv is as follows:

lr-Adam,step,accuracy,epoch,val_acc_epoch,acc_epoch
0.01,0,,,,
,49,0.125,0,,
,62,0.0989999994635582,0,0.0982142835855484,
,62,,0,,0.095238097012043
0.01,63,,,,
,99,0.25,1,,
,125,0.0989999994635582,1,0.0982142835855484,

As it can be seen, it is not stored every epoch end and it does not save the information fancy. How can I solve this?

Thanks in advance

[akihironitta]

Hi @JNaranjo-Alcazar!

it is not stored every epoch end and it does not save the information fancy.

Could I ask what you mean by "fancy"? Could be related to #11605?

[JNaranjo-Alcazar]

I mean that I just want to store the info every epoch end, not every step or batch.

[akihironitta]

@JNaranjo-Alcazar Mind trying turning off step logging since by default it logs the metric every step according to the table here: https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html#id3

    def training_step(self, batch, batch_idx):
        x, y = batch
        # forward, logits
        logits = self(x)
        acc = accuracy(logits, y)
        J = self.loss(logits, y)
-       self.log("accuracy",  acc, prog_bar=True)
+       self.log("accuracy",  acc, prog_bar=True, on_step=False)
        return {"loss": J, "acc": acc}

    def training_epoch_end(self, outputs):
        avg_loss = torch.tensor([x["loss"] for x in outputs]).mean()
        avg_accuracy = torch.tensor([x["acc"] for x in outputs]).mean()
        self.log("acc_epoch", avg_accuracy, prog_bar=True)

    def validation_step(self, val_batch, batch_idx):
        return self.training_step(val_batch, batch_idx)

    def validation_epoch_end(self, outputs):
        avg_loss = torch.tensor([x["loss"] for x in outputs]).mean()
        avg_accuracy = torch.tensor([x["acc"] for x in outputs]).mean()
        self.log("val_acc_epoch", avg_accuracy, prog_bar=True)
        return {"val_loss": avg_loss}