DeepSpeed Metrics Logging

[ktrapeznikov]

I am using deepspeed_stage_3_offload with multiple GPUs on a single node.
To log and sync metrics in the LightingModule, do I just add the following:
self.log(metric_name, value, sync_dist=True)

Or is there something special that needs to be done?

[tchaton]

No, it is all good !
Be aware sync_dist=True will force synchronisation for each batch, which might have performance degradation.
And by default, Lightning already does synchronisation on epoch end.

[ktrapeznikov]

Ok. Good to know about sync. So how are metrics logged in between syncs?