How to log optimizer and learning rate when set by LightningCLI

[slinnarsson]

I'm using LightningCLI and setting the optimizer and learning rate using args like --optimizer AdamW --optimizer.lr 0.01. When running multiple runs with varying optimizers and learning rates, I would like these hyperparameters to show up in tensorboard, but I don't know how to make that happen. Using save_hyperparameters() in the __init__() of my module doesn't work, because the optimizer and learning rate are not parameters to __init_(). As I understand it, the optimizer is patched into the module class by LightningCLI. Can I make LightningCLI log these as hyperparameters?

Btw, the settings are saved correctly in config.yaml, they just don't show up in hparams.yaml.

Update: I found a workaround; I added optimizer and lr as arguments to __init__() even if I don't use them there. They get saved by save_hyperparameters() and show up in the Tensorboard logs.

[mauvilsa]

To have some custom logging of hyperparameters I would recommend to extend SaveConfigCallback to do this. Note that this callback has access to the full config. For more comments related to this see #13577 (comment)