where to add weight decay scheduler

[FredZZD]

I want to change weight decay during training, which hook should I overwrite, optimizer_step(), on_train_batch_end() or ... ?

[akihironitta]

@FredMushZhao I think either works if you want to schedule it on each step. Alternatively, you could use on_train_epoch_start if you want to update it on each epoch.

For reference on all available hooks: https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_module.html#hooks

[FredZZD]

thanks. :)