How to Log Metrics (eg. Validation Loss, Accuracy) To TensorBoard Hparams?

[athenawisdoms]

I am using Pytorch Lightning 1.2.6 to train my models using DDP and TensorBoard is the default logger used by Lightning.

My code is setup to log the training and validation loss on each training and validation step respectively.

class MyLightningModel(pl.LightningModule):

    def training_step(self, batch):
        x, labels = batch
        out = self(x)
        loss = F.mse_loss(out, labels)
        self.log("train_loss", loss)
        return loss

    def validation_step(self, batch):
        x, labels = batch
        out = self(x)
        loss = F.mse_loss(out, labels)
        self.log("val_loss", loss)
        return loss

TensorBoard correctly plots both the train_loss and val_loss charts in the SCALERS tab. However, in the HPARAMS tab, on the left side bar, only hp_metric is visible under Metrics.

However, in the HPARAMS tab, on the left side bar, only hp_metric is visible under Metrics.

How can we add train_loss and val_loss to the Metrics section? This way, we will be able to use val_loss in the PARALLEL COORDINATES VIEW instead of hp_metric.

Image showing hp_metric and no val_loss:

Using Pytorch 1.8.1, Pytorch Lightning 1.2.6, TensorBoard 2.4.1

[SkafteNicki]

I think it is explained very well in this section of the documentation:
https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html#logging-hyperparameters
Basically, you just need to overwrite the hp_metric tag with whatever value you want to show up in the HPARAMS tab in tensorboard.

[athenawisdoms]

@SkafteNicki Thanks for pointing to the docs! Following the exmaple there, I am getting this error

TypeError: log_hyperparams() takes 2 positional arguments but 3 were given

after I added the log_parameters call to my pl.LightningModule

    def on_train_start(self):
        self.logger.log_hyperparams(self.hparams, {"val_loss": 0, "val_acc": 0})

My trainer is started with 2 loggers

tb_logger = TensorBoardLogger(
    OUTPUT_DIR, name=NAME, default_hp_metric=False
)
 trainer = pl.Trainer(
    logger=[tb_logger, another_logger],
)

I think self.logger is a pytorch_lightning.loggers.base.LoggerCollection object. In this case, must we somehow select the tensorboard logger only and then call log_hyperparams on the tensorboard logger? If so, how do we select the tensorboard logger?

Thanks!

[SkafteNicki]

when using multiple loggers you can always access the individual logger using indexing:

self.logger[0] # tb_logger
self.logger[1] # another logger

you therefore simply have to call self.logger[0].log_hyperparams(...) in you code :]