11# Using DFTK on compute clusters
22
3- This chapter summarises a few tips and tricks for running on compute clusters.
3+ This chapter summarises a few tips and tricks for running DFTK on compute clusters.
44It assumes you have already installed Julia on the machine in question
55(see [ Julia downloads] ( https://julialang.org/downloads/ )
66and [ Julia installation instructions] ( https://julialang.org/downloads/platform/ ) ).
7+ For a general presentation of using Julia on HPC systems from both the user
8+ and the admin perspective,
9+ see [ JuliaOnHPCClusters] ( https://juliahpc.github.io/JuliaOnHPCClusters/ ) .
710
8- We will use [ EPFL's scitas clusters] ( https://scitas-doc.epfl.ch/ ) as examples,
9- but the basic principles should apply to other systems as well .
11+ In this presentation we use [ EPFL's scitas clusters] ( https://scitas-doc.epfl.ch/ )
12+ as the example to explain the basic principles .
1013
1114## Julia depot path
1215By default on Linux-based systems
@@ -24,7 +27,7 @@ the `JULIA_DEPOT_PATH` to be a subdirectory of `/scratch`.
2427** EPFL scitas.**
2528On scitas the right thing to do is to insert
2629```
27- export JULIA_DEPOT_PATH="/scratch/$USER/.julia"
30+ export JULIA_DEPOT_PATH="$JULIA_DEPOT_PATH: /scratch/$USER/.julia"
2831```
2932into your ` ~/.bashrc ` .
3033
@@ -148,6 +151,23 @@ using MPIPreferences
148151MPIPreferences. use_system_binary ()
149152```
150153
154+ ## Using the ` --heap-size-hint ` flag
155+ Julia uses dynamic memory management,
156+ which means that unused memory may not be directly released back to the operating system.
157+ Much rather a garbage collection takes care of doing such cleanup periodically.
158+ If you are in a memory-bound situation,
159+ it can thus be helpful to employ the ` --heap-size-hint ` flag,
160+ which provides Julia with a hint of the maximal memory Julia may use.
161+ For example the call
162+ ```
163+ julia --heap-size-hint 40G
164+ ```
165+ tells Julia to optimise the garbage collection,
166+ such that the overall memory consumption remains below ` 40G ` .
167+ Note, however that this is just a * hint* ,
168+ i.e. no hard limits are enforced.
169+ Furthermore using too small a heap size hint can have a negative impact on performance.
170+
151171## Running slurm jobs
152172This example shows how to run a DFTK calculation on a slurm-based system
153173such as scitas. We use the MKL for FFTW and BLAS and the system-provided MPI.
@@ -241,7 +261,7 @@ Finally the jobscript `job.sh` for a slurm job looks like this:
241261
242262# IMPORTANT: Set Julia's depot path to be a local scratch
243263# direction (not your home !).
244- export JULIA_DEPOT_PATH=" /scratch/$USER /.julia"
264+ export JULIA_DEPOT_PATH=" $JULIA_DEPOT_PATH : /scratch/$USER /.julia"
245265
246266# Load modules to setup julia (this is specific to EPFL scitas systems)
247267module purge
0 commit comments