Horovod distributed data

[njgre6]

Hi there!
I am trying to set up a multi-node horovod cluster, running some proprietary code that is wrapped up in a LightningModule. I am wondering, how does the data in a lightning module get communicated between nodes, when the accelerator arg in the Trainer class is set to 'horovod'. Specifically, does each instance of the cluster need a local copy of the data?