How to estimate SARvey memory requirements per core/dataset?

[epehlivanli]

Currently unclear how to anticipate memory needs. Is there a way to estimate memory need based on:

• Number of pixels / points (e.g., Grove dataset: 13,344)

• Number of arcs (e.g., ~392,246)

• Number of nearest neighbors (e.g., 50)

• Number of cores?

From testing:

• Grove dataset: 13k points, 392k arcs, 137 dates -> ~120 GB RAM needed for Step 1/2 with 24–48 cores.

Grove dataset with 24 Cores:

Memory Usage on Step 1

              total        used        free	  shared  buff/cache   available
Mem:           187Gi	   123Gi        60Gi	   851Mi       6.1Gi	    63Gi
Swap:             0B          0B          0B


Memory Usage on Step 2

               total        used        free	  shared  buff/cache   available
Mem:           187Gi	   120Gi        63Gi	   851Mi       6.1Gi	    66Gi
Swap:             0B          0B          0B

Grove dataset with 48 Cores:

Memory Usage on Step 1

               total        used        free	  shared  buff/cache   available
Mem:           187Gi       123Gi        60Gi	   851Mi       6.1Gi	    63Gi
Swap:             0B          0B          0B


Memory Usage on Step 2

               total        used        free	  shared  buff/cache   available
Mem:           187Gi       120Gi        63Gi	   851Mi       6.1Gi	    66Gi
Swap:             0B          0B          0B

Grove dataset:

wget http://149.165.154.65:/data/HDF5EOS/Emirhan/Grove_subset/Grove_newcon.tar.gz
tar -xzf Grove_newcon.tar.gz
cd Grove_newcon
sarvey -f config.json 0 2

[Andreas-Piter]

Dear Emirhan,
Thank you for sharing the insights in the memory usage for your dataset. It seems to me that you are using a one-step unwrapping approach, is that correct? This approach is intended for small areas only, due to the following: During the temporal unwrapping the parameters (relative dem error and relative velocity) of all arcs need to be integrated to get the parameters at the scatterers. This integration requires the inversion of a matrix with the size num_points x num_points. Such a matrix will require a lot of memory and the computational time of the inversion is O(n^3), here with n being the number of points. Therefore, you need to keep the number of points small to make the processing feasible. You could do that e.g. by making the spatial extend of your dataset smaller. You can crop the stack or use a spatial mask around your area of interest. If your area is larger, please consider applying the two-step unwrapping approach.
Best, Andreas