Novelty/advancement, benchmarking, force calculations

[mikesha2]

In reference to #openjournals/joss-reviews#9480,

First, you have clearly put in a lot of work into this repository in terms of taking care of tedious stuff like cell-based neighbor calculations and hard coding a few potentials. The documentation looks complete enough, and the organization is as JOSS requires.

My main concern is of course the most subjective part of the evaluation: does this work "either [enable] some new research challenges to be addressed or makes addressing research challenges significantly better (e.g., faster, easier, simpler)." Why should I as a user choose your package over Molly.jl?

Questions I ask when choosing a software for molecular dynamics or quantum chemistry calculations:
Hardware considerations

1. How fast is the software? (unsure if you have any specific benchmarks you can point to, along with test system details)
2. What's the memory usage like? If I want to run modest-sized simulations locally, do I need to whip out my M4 Max with 128 GB RAM? What memory allocation would I need to assign on an HPC system? Or more specifically, what number do I put in my SLURM job script?
3. Is there GPU/FPGA acceleration?
   Day-to-day questions
4. How easily can I add custom forces? Do I need to manually calculate the forces from the energy, or does autodiff take care of that (JAX-MD)?
5. How easily can I restrict degrees of freedom/apply constraints?
6. Are there built-in analysis pipelines I can use?
   Numerical stability
7. How easily do NaNs pop up? Are forces/energies regularized so that my system doesn't blow up? Are NaNs propagated through the simulation, and how much of a performance hit is that?
   Style/usage, optional
8. Is there some special way the code makes you think, some abstraction which more closely mirrors how a biophysicist thinks about their problem?

From looking at your paper and the code, I don't have easy answers to these questions.

[Cavenfish]

Thank you for the recognition of the work I've put into this project! Also, thank you for agreeing to review this JOSS submission and doing so very quickly.

I'll first address your general question:

Why should I as a user choose your package over Molly.jl?

With Molly.jl specifically, a user might choose YASS over it since it has the capability to perform geometry optimizations, cell optimizations, and harmonic frequency calculations. As far as I'm aware Molly.jl does not have those particular capabilities.

When comparing to other commonly used packages:

• YASS is faster than pure ASE-based molecular dynamics
• YASS is easier to compile and interact with than LAMMPS and GROMACs
• YASS (through Optim.jl) provides more optimization algorithms for geometry/cell relaxations than JAX-MD

In some aspects YASS falls short compared with these other (older and more established) packages, but it still has something to offer.

In terms of your numbered questions:

1. No benchmarks have been done yet. This can be done and added somewhere in the documentation or readme.
2. Again, no explicit memory benchmarks have been done. In most of my slurm scripts I've used 2-4GB of memory depending on the potential, number of molecules, and simulation settings. If wanted, more precise measurements can be made and again added to the documentation or readme.
3. Sadly, no.
4. Very easily, an example is shown here in the documentation. In the example the forces are calculated analytically, however, a user can easily use one of the JuliaDiff packages to perform autodiff-style force calculations. Currently YASS does not do this automatically for the user.
5. This is a work in progress. Within the code there is the option to use fixed atom constraints, but nothing more as of now. YASS is setup to be flexible and easily customizable, so a user could in theory develop their own custom constraints but there is no documentation for this yet.
6. Currently, YASS only has velocity autocorrelation function (VACF) analysis and radial distribution function (RDF) as a built-in analysis pipelines.
7. The frequency of NaNs will heavily depend on the potential used. For the built-in potentials, they should (hopefully) never come up. However, there isn't any checks for it in place currently nor any tests on the performance hit they cause.
8. Tough for me to say. I feel the code is intuitive to work with, but I wrote it 😅. Honestly though, in this respect I don't think there is anything too special about YASS.

You mention:

From looking at your paper and the code, I don't have easy answers to these questions.

Would you like me to try and include some of these answers into the paper? The word count limit might present some challenges but I can still try to add some into it.

[mikesha2]

With Molly.jl specifically, a user might choose YASS over it since it has the capability to perform geometry optimizations, cell optimizations, and harmonic frequency calculations. As far as I'm aware Molly.jl does not have those particular capabilities.

When comparing to other commonly used packages:

• YASS is faster than pure ASE-based molecular dynamics

You would need specific data (benchmarks) to support this claim.

• YASS is easier to compile and interact with than LAMMPS and GROMACs
• YASS (through Optim.jl) provides more optimization algorithms for geometry/cell relaxations than JAX-MD

I think highlighting these specifics would be helpful in the paper, and wouldn't take too much space. I think you do need to highlight major features that these other packages offer which yours does not, such as distributed-memory models, support for heterogeneous compute, robust analysis pipelines, etc.

1. No benchmarks have been done yet. This can be done and added somewhere in the documentation or readme.
2. Again, no explicit memory benchmarks have been done. In most of my slurm scripts I've used 2-4GB of memory depending on the potential, number of molecules, and simulation settings. If wanted, more precise measurements can be made and again added to the documentation or readme.

I think just a comment in terms of memory usage as a function of system size would be helpful.

3. Sadly, no.

This would be the major limitation you would need to highlight to the user, as serious molecular dynamics calculations are rarely done without GPU acceleration.

4. Very easily, an example is shown here in the documentation. In the example the forces are calculated analytically, however, a user can easily use one of the JuliaDiff packages to perform autodiff-style force calculations. Currently YASS does not do this automatically for the user.

Something worth mentioning perhaps, as the user would then have to learn a separate interface for doing autodiff.

5. This is a work in progress. Within the code there is the option to use fixed atom constraints, but nothing more as of now. YASS is setup to be flexible and easily customizable, so a user could in theory develop their own custom constraints but there is no documentation for this yet.
6. Currently, YASS only has velocity autocorrelation function (VACF) analysis and radial distribution function (RDF) as a built-in analysis pipelines.
7. The frequency of NaNs will heavily depend on the potential used. For the built-in potentials, they should (hopefully) never come up. However, there isn't any checks for it in place currently nor any tests on the performance hit they cause.

Even simple and commonly used potentials like Lennard-Jones easily blow up if two atoms accidentally overlap just a little too much during force integration. Something to consider in terms of design if you want to incorporate some regularization.

8. Tough for me to say. I feel the code is intuitive to work with, but I wrote it 😅. Honestly though, in this respect I don't think there is anything too special about YASS.> You mention: > > From looking at your paper and the code, I don't have easy answers to these questions.
   Would you like me to try and include some of these answers into the paper? The word count limit might present some challenges but I can still try to add some into it.

I think most of these can be addressed in a sentence and/or a figure. I think any additional figures don't necessarily need to be in the paper, but I would like to see at least some simple benchmarks in the README markdown.

[Cavenfish]

I've made some edits to the documentation, readme, and paper to address your comments.

Please let me know if things can still be improved!