run dp with multiple node

[jingganglan]

Hi,

I am trying to run dp with multiple nodes, each node has 1 gpu.
I am wondering if this is possible?

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
export CRAY_CUDA_MPS=1
export CUDA_VISIBLE_DEVICES=0

srun -N dp train water_se_a.json > log

this run script doesn't work. dp will only use one GPU

[jameswind]

Hi, right now the DeePMD-kit code does not support multi-GPU training. It only supports multi-GPU MD.

…

On Fri, Nov 29, 2019 at 6:25 PM jingganglan ***@***.***> wrote: Hi, I am trying to run dp with multiple nodes, each node has 1 gpu. I am wondering if this is possible? export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK export CRAY_CUDA_MPS=1 export CUDA_VISIBLE_DEVICES=0 srun -N dp train water_se_a.json > log this run script doesn't work. dp will only use one GPU — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/deepmodeling/deepmd-kit/issues/131?email_source=notifications&email_token=AEJ6DC7RXTVOJFTAATDGZILQWGQIFA5CNFSM4JTEACW2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H46426A>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJ6DC6TTJDGL727DS6FQBTQWGQIFANCNFSM4JTEACWQ> .

[tonystarkiss]

Does it still not support multi gpu training now?
I find it training on one GPU default, I don't know how to use multi GPU accelerate training?

[njzjz]

Now it's under development in the devel branch.

[tonystarkiss]

the multi GPU training feature is not included in v2.0.0? When will it release ?

[jingganglan]

thanks