Hello,
as far as I have understood your project correct, it is not possible to run the training with different number of GPUs per node. E.g. two nodes, first node 4 GPUs, second node 8 GPUs.
Is this correct? Are there any plans to implement this feature in future?