Using atomate2 launch many calculations on slurm HPC center

[Asif-Iqbal-Bhatti]

Hello,

I have looked at the documentation and it is not clear to me how to launch many calculations on slurm. I know there is a way to run_locally. In this script jobs are launch one after the other in de loop en it takes a long time. Is there a way to launch all at once. There is something that mentiones about Fireworks or Fireworks-remote but there is no tutorial how to setup on slurm HPC center.

'''
struct_dir = Path("generate_polymorph_configurations/polymorph_configurations")

for i, inFile in enumerate(struct_dir.glob("*.vasp")):
structure_file = Structure.from_file(inFile)

relax_job = RelaxMaker(input_set_generator=input_generator).make(structure=structure_file)

static_job = StaticMaker(input_set_generator=my_custom_set).make(
    structure=relax_job.output.structure, 
    prev_dir=relax_job.output.dir_name
)

my_flow = Flow([relax_job, static_job], output=static_job.output)
run_locally(my_flow, store=job_store, create_folders=True)

'''

[JaGeo]

Actually, we do have video tutorials on jobflow-remote. https://github.com/materialsproject/atomate2?tab=readme-ov-file#tutorials

You can find more information on jobflow-remote here: https://github.com/Matgenix/jobflow-remote

[Asif-Iqbal-Bhatti]

Hello,

I follow fireworks, but it seems like there is no workaround for storing jobs. It always asks for Mongo or Mongo Atlas. Is there a way that I can use automate2 or fireworks to launch jobs without any jobs storing? I just want to launch many jobs on Slurm. I cannot find a way to do it.

[JaGeo]

I think jobflow-remote is more covenient. It still requires a MongoDB but one can use remote submission making the setup easier as you can install the DB on a workstation and remotely submit to a hpc. In both cases, a MongoDB is required.

[JaGeo]

Maybe also of interest: https://autoatml.github.io/autoplex/user/installation/installation.html#before-you-start-using-autoplex

[JaGeo]

We are working on improving the installation process. As this atomate2 is mostly developed based on research project funding and not specific software funding, it is not always easy to coordinate auch efforts. I hope you can understand this.

Nevertheless, I hope jobflow-remote might still be a useful way to accomplish your research goal.