How are callback calls handled in multi-gpu mode?

[ktrapeznikov]

Say I have a callback that changes a hyper-parameter of the underlying model before every epoch.

class ChangeHyperParam(pl.Callback):
    def on_train_epoch_start(self, trainer, pl_module):
         pl_module.hyper_param1 = func(trainer.current_epoch)

What happens when I use this callback in DDP mode. Will this call be called on every GPU automatically? Or do I have to do something else?

[ananthsub]

Callbacks run on all ranks, so this would be called on each GPU in distributed training. As long as func returns the same value given the inputs, this should be fine