[Help]:

[Adre17]

Describe the problem you encountered
Provide a brief description of the issue you're facing.

OS and software version
Please specify your operating system and the version of PyGMTSAR you are using, if applicable.

• OS:
• ubuntu
• PyGMTSAR version:
• last

Steps to reproduce the problem
Reprodice Golden Valley Subsidence out of google colab

Log file
Syncing NetCDF 2D/3D Dataset: 100%|█████████████| 52/52 [00:43<00:00, 1.18it/s]
📌 SBAS salvati in /workspace/SBAS_cumulative_displacement.png
NaN: 0.0 / 16995482.0 (0.00%)
Inf: 0.0 / 16995482.0 (0.00%)
Traceback (most recent call last):
File "/workspace/Subsidence_1.py", line 261, in
processing_sub_1.run_dask_processing(sbas, AOI, DEM)
File "/workspace/processing_sub_1.py", line 174, in run_dask_processing
velocity_sbas = sbas.velocity(disp_sbas)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/pygmtsar/Stack_detrend.py", line 1098, in velocity
velocity = nanoseconds_per_year*data.polyfit('date', 1).polyfit_coefficients .sel(degree=1).astype(np.float32).rename('trend')
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/xarray/core/dataarray.py", line 5801, in polyfit
return self._to_temp_dataset().polyfit(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/xarray/core/dataset.py", line 89 20, in polyfit
return polyfit_impl(self, dim, deg, skipna, rcond, w, full, cov)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/xarray/computation/fit.py", line 203, in polyfit
rank = np.linalg.matrix_rank(lhs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/numpy/linalg/_linalg.py", line 2 108, in matrix_rank
S = svd(A, compute_uv=False, hermitian=hermitian)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/numpy/linalg/_linalg.py", line 1 822, in svd
s = _umath_linalg.svd(a, signature=signature)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/numpy/linalg/_linalg.py", line 1 13, in _raise_linalgerror_svd_nonconvergence
raise LinAlgError("SVD did not converge")
numpy.linalg.LinAlgError: SVD did not converge

[AlexeyPechnikov]

'SVD did not converge' commonly means all empty interferograms. I regularly update Google Colab notebooks when some Python libraries or Google Colab updates broke the processing; you would try the recent notebook version.
If it still does not help, there are PyGMTSAR Docker images (see stable ones, not automatically generated) for reproducible runs.

[SuvamBRB]

Respected community,
Does anyone run PyGMTSAR on an HPC system? I have set up PyGMTSAR on my allotted system using Apptainer, but I am encountering the following issues while processing around 30 SLC images with the full IW1 sub-swath:

1. The processing speed does not improve even with increased resources (40 cores + 256 GB RAM).
2. The geocoding step gets stuck.
   Question: Are there any recommended workarounds to handle large stacks efficiently without triggering deadlocks or “too many open files” errors?
   Thank you for your guidance.

[AlexeyPechnikov]

'ulimit -n' controls the allowed number of opened files. You need to configure your Dask cluster for your hardware for better performance.

[SuvamBRB]

Thank you, I will try that.

Do you have any specific recommendations for Dask configuration in this case? The resources available on my node are:

• CPU: 2 × Intel Xeon Gold 6148 (Skylake), 20 cores each @ 2.4 GHz
• Total cores: 40
• RAM: 96 GB
  In particular, I would appreciate guidance on:
• Number of Dask workers vs threads per worker
• Any recommended limits on open files (ulimit -n) or worker memory
  Also, could you give the expected processing time for a full IW1 sub-swath with ~30 SLC images on a single HPC node?
  Thank you very much for your constant guidance.

[AlexeyPechnikov]

You can set ulimit to 10000 as the minimum. Use 8-16GB RAM per worker. Processing time for PyGMTSAR examples on different configurations is available at https://hub.docker.com/r/pechnikov/pygmtsar

[SuvamBRB]

Thank you very much, I will try and update you.