LightningModule.log does not work for validation metrics

[qpng]

I implemented the validation loop like this (using validation_epoch_end because the number of labels in every batch is variable):

class MyModel(pl.LightningModule):

    def validation_step(self, batch, batch_idx):
        return {'target': ... ,'pred': ...}

    def validation_epoch_end(self, outputs) -> None:
        targets = torch.cat([o['target'] for o in outputs])
        preds = torch.cat([o['pred'] for o in outputs])
        accuracy = (preds == targets).sum().item() / len(preds)
        self.log("val_acc", accuracy * 100)

I verified that accuracy is computed correctly (as a float). However, I can't see it in the console. The console output looks very odd, because the progress bar is repeated during validation (it looks okay during training), and the "Validating" prefix changed back to "Epoch 0" (the number of epochs also changed from 74 to 79).

Epoch 0:   6%|▋         | 5/79 [00:04<01:03,  1.16it/s, loss=6.29]
Validating: 0it [00:00, ?it/s]
Validating:   0%|          | 0/74 [00:00<?, ?it/s]
Epoch 0:  10%|█         | 8/79 [00:04<00:39,  1.80it/s, loss=6.29]
Epoch 0:  14%|█▍        | 11/79 [00:04<00:28,  2.41it/s, loss=6.29]
Epoch 0:  18%|█▊        | 14/79 [00:04<00:21,  2.98it/s, loss=6.29]
Epoch 0:  22%|██▏       | 17/79 [00:04<00:17,  3.51it/s, loss=6.29]
Epoch 0:  25%|██▌       | 20/79 [00:04<00:14,  4.04it/s, loss=6.29]
Epoch 0:  29%|██▉       | 23/79 [00:05<00:12,  4.52it/s, loss=6.29]
Epoch 0:  33%|███▎      | 26/79 [00:05<00:10,  5.00it/s, loss=6.29]
Epoch 0:  37%|███▋      | 29/79 [00:05<00:09,  5.45it/s, loss=6.29]
Epoch 0:  41%|████      | 32/79 [00:05<00:07,  5.88it/s, loss=6.29]
Epoch 0:  44%|████▍     | 35/79 [00:05<00:07,  6.27it/s, loss=6.29]
Epoch 0:  48%|████▊     | 38/79 [00:05<00:06,  6.65it/s, loss=6.29]
Epoch 0:  52%|█████▏    | 41/79 [00:05<00:05,  6.99it/s, loss=6.29]
Epoch 0:  56%|█████▌    | 44/79 [00:05<00:04,  7.34it/s, loss=6.29]
Epoch 0:  59%|█████▉    | 47/79 [00:06<00:04,  7.67it/s, loss=6.29]
Epoch 0:  63%|██████▎   | 50/79 [00:06<00:03,  7.99it/s, loss=6.29]
Epoch 0:  67%|██████▋   | 53/79 [00:06<00:03,  8.29it/s, loss=6.29]
Epoch 0:  71%|███████   | 56/79 [00:06<00:02,  8.58it/s, loss=6.29]
Epoch 0:  75%|███████▍  | 59/79 [00:06<00:02,  8.88it/s, loss=6.29]
Epoch 0:  78%|███████▊  | 62/79 [00:06<00:01,  9.15it/s, loss=6.29]
Epoch 0:  82%|████████▏ | 65/79 [00:06<00:01,  9.40it/s, loss=6.29]
Epoch 0:  86%|████████▌ | 68/79 [00:07<00:01,  9.63it/s, loss=6.29]
Epoch 0:  90%|████████▉ | 71/79 [00:07<00:00,  9.85it/s, loss=6.29]
Epoch 0:  94%|█████████▎| 74/79 [00:07<00:00, 10.10it/s, loss=6.29]
Epoch 0:  97%|█████████▋| 77/79 [00:07<00:00, 10.31it/s, loss=6.29]
Epoch 0: 100%|██████████| 79/79 [00:07<00:00, 10.41it/s, loss=6.29]
Epoch 1:   8%|▊         | 6/79 [00:03<00:46,  1.56it/s, loss=5.08]
Validating: 0it [00:00, ?it/s]
Validating:   0%|          | 0/74 [00:00<?, ?it/s]
Epoch 1:  11%|█▏        | 9/79 [00:04<00:31,  2.25it/s, loss=5.08]
Epoch 1:  15%|█▌        | 12/79 [00:04<00:23,  2.90it/s, loss=5.08]

What is the right way to implement validation accuracy? I'm new to lightning, but I looked at the documentation and couldn't find the answer. Thanks for your help.

A few related posts:

• How to show the training and validation accuracy as well as loss in the end of process bar?? #5774 (proposed solution is too complex)
• How to log performance metrics in the validation step/loop? #1906 (out of date, as validation_epoch_end shouldn't return anything now)
• https://forums.pytorchlightning.ai/t/computing-validation-accuracy-at-the-end-of-each-epoch/188/2 (out of date, as EvalResult has been removed)

Update:
I added self.log("val_loss", ...) to validation_step, but even this does not log anything in the progress bar (or anywhere in the console). It seems that self.log is not working at all.

Update:
I finally made it work by explicitly specifying all these parameters: self.log("val_acc", raw_accuracy * 100, prog_bar=True, on_step=False, on_epoch=True). (However, it still doesn't work for val_loss, which has on_step=True, on_epoch=False.) The repeated progress bar problem is not fixed.

[aribornstein]

@justusschock

[awaelchli]

Set prog_bar=True in self.log()
Or you can just open tensorboard to see the graphical logs

tensorboard --logdir lightning_logs

cheers