-
Notifications
You must be signed in to change notification settings - Fork 25
Running Xolotl
It is very easy to run Xolotl. After building Xolotl (see Build Configuration) you can use one of the benchmark parameter files (see Benchmark Problems), and all you have to do to run it is something like the following from the build directory:
Running in serial:
./xolotl <input_parameter_file>
Running in parallel:
mpiexec -n 4 ./xolotl <input_parameter_file>
This will work from any other directory if Xolotl is on your path.
This is an example of files needed to submit a job with multiple Xolotl simulations, assuming the Xolotl executable is available in /path_to_xolotl/xolotl. It has been tested on Cori.
First define where each simulation will be run in input_xolotl.txt:
/path/run_0
/path/run_1
/path/run_2
/path/run_3
Then, how to run each simulation in run_xolotl.sh:
cd $1
hostname >> nodeid.output
time srun -N 1 -n 32 /path_to_xolotl/xolotl/xolotl param.txt > log.output
Finally the job script submit.sbatch:
#!/bin/bash
#SBATCH --qos=debug
#SBATCH --constraint=haswell
#SBATCH -N 5
#SBATCH --ntasks-per-node 32
#SBATCH -t 00-0:02:00
module load parallel
parallel --joblog parallel_joblog /path_to_script/run_xolotl.sh {} < input_xolotl.txt
Because we are using PETSc as the solver and that it takes care of the MPI parallelization, there are a couple constraints to the grid geometry that can be used. The first one is that there should not be too few grid points per MPI process when running in parallel, in 1D it means that we should have at least 2 grid points per process. The second one is that in 3D, the number of grid points in the Y and Z direction should be a multiple of 3 (2*stencil_width + 1). The code will give an error if these rules are not respected.
As briefly described in Parameter File, it is possible to provide a time-dependent flux profile to Xolotl. Xolotl will read the information from the file which name is defined by the parameter file option "fluxFile", where the first column should correspond to the time in second and the second column to the flux amplitude in #/nm2/s. Here is an example:
0.5 1.0e6
1.0 5.0e5
In this case, Xolotl will use a constant flux amplitude of 1.0e6 #/nm2/s from 0s to 0.5s, then a linear decreasing ramp from 0.5s to 1s, starting at 1.0e6 #/nm2/s and decreasing to 5.0e5 #/nm2/s, and for anytime larger than 1s the flux will be 5.0e5 #/nm2/s .
As briefly described in Parameter File, it is possible to provide a time-dependent temperature profile to Xolotl. Similarly the the time-dependent flux, Xolotl will read the information from the file which name is defined by the parameter file option "tempFile", where the first column should correspond to the time in second and the second column to the temperature in K. It is important to note that the temperature will be the same on every grid point in Xolotl when using this option. Here is an example:
0.5 1000
1.0 1500
In this case, Xolotl will use a constant temperature of 1000 K from 0s to 0.5s, then a linear ramp from 0.5s to 1s, starting at 1000 K and increasing to 1500 K, and for anytime larger than 1s the temperature will be 1500 K .
When using the "fluxDepthProfileFilePath" parameter in the Parameter File, Xolotl can read-in implantation profiles from the file pointed to in the option (for instance "fluxDepthProfileFilePath=tridyn.dat"). In this case the other material parameters correspond to the material selected in the "material" parameter. Currently, Xolotl can simultaneously implant as many cluster as the user wants as long as they are not mixed clusters (He1 can be implanted, He1V1 can't). The format of the depth profile file is the following: there are two lines per cluster, with the first line corresponding to the type of cluster (He, D, T, V, I, Xe), followed by the wanted size and finally the flux factor which will be used to determine the effective flux amplitude of the given cluster by multiplying it by the value given by the parameter file "flux" option; the second line correspond to the fit parameters of an order 15 polynomial fit in ascending order, followed by one value in nm that is the depth cut-off beyond which the profile will be forced to 0.0. The amplitude of the fit doesn't matter as Xolotl will renormalize the profile. An example of tridyn.dat file can be found in Benchmark Problems.
The previous feature can be used for code integration purposes where an external code would provide the implantation profiles. Xolotl can then use the given profiles to evolve the sub-surface evolution of cluster concentrations and provide feedback of the material composition to the external code which can, in turn, update the implantation profiles. The customized PETSc option "-tridyn" can be used to create an HDF5 file containing the total concentration of He, D, T, V, I, and temperature at each depth, at each time step. An other option, when using Xolotl in 1D, is to use the checkpoint customized PETSc option "-start_stop" which will call the same monitor as "-tridyn" when it is done with the checkpoint file. The advantage of the latter is that a time frequency can be specified while the former will write a new file at every time step. More information on these customized PETSc options can be found at PETSC Options.