track_grad_norm with wandb

[surya-narayanan]

Hello,

I am using version 1.2.4 of PL. I have track_grad_norm = 1 in my Trainer object initialization, but don't see any of the grad norms in my wandb dashboard (There, I see just the things I log with self.log). I was able to see the grad norms in TensorBoard prior, and use a wandb_logger as wandb_logger = WandbLogger(blah).

Is it not supported yet?

[surya-narayanan]

It seems the self.log(grad_norms) work when we return a loss, not sure why.

[carmocca]

Can you try on the latest version? And if it still does not work, can you provide a reproducible example? You can adapt this script: https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/bug_report_model.py

I tried reproducing it and couldn't (using a different logger though):

def test_discussion(tmpdir):
    model = BoringModel()
    trainer = Trainer(
        default_root_dir=tmpdir,
        max_epochs=1,
        track_grad_norm=2,
        logger=CSVLogger(tmpdir)
    )
    trainer.fit(model)

    x = tmpdir / "default" / "version_0" / "metrics.csv"
    print(x.read_text('utf-8'))