Parallel Training via mpirun Not Working

[AnuragKr]

Hi,

Command -- CUDA_VISIBLE_DEVICES=4,5,6,7 mpirun -np 4 dp train --mpi-log=workers input.json

Output --

Problem:- Unable to do parallel training via mpirun command. Horovod installation giving an error that's why trying via mpirun only.
Is it important to install HOROVOD even if we want to do parallel training via the mpirun command?

System Configuration -- DGX V100 GPU

[AnuragKr]

In the documentation, it is clearly written that without HOROVOD installation it will fall back to serial mode training.
Link -- https://docs.deepmodeling.com/projects/deepmd/en/master/install/install-from-source.html