Skip to content
This repository was archived by the owner on Oct 15, 2025. It is now read-only.

Commit df7cb1e

Browse files
Matt GarthwaiteMatt Garthwaite
authored andcommitted
update documentation for release
1 parent 9fcabc0 commit df7cb1e

File tree

12 files changed

+177
-252
lines changed

12 files changed

+177
-252
lines changed

docs/PyRate_plot_screenshot.png

242 KB
Loading

docs/dependencies.rst

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,20 @@
11
Dependencies
22
------------
33

4-
The following dependencies need to be on your system prior to `PyRate` installation:
4+
The following dependencies need to be on your system (or in your working
5+
environment) prior to `PyRate` installation:
56

67
- Python_, versions 3.6, 3.7 or 3.8.
78
- GDAL_, versions 3.0.2 or 3.0.4
89
- `Open MPI`_, versions 2.1.6, 3.0.4, 3.1.4 or 4.0.2
910

10-
The versions of each package stated above have been tested to work.
11+
The versions of each package stated above have been tested to work using
12+
`Travis CI`_ continuous integration testing.
1113

1214
.. _Python: https://www.python.org/downloads/
1315
.. _GDAL: https://gdal.org/download.html
1416
.. _`Open MPI`: https://www.open-mpi.org/software/ompi/v4.0/
17+
.. _`Travis CI`: https://travis-ci.org/github/GeoscienceAustralia/PyRate
1518

16-
Other Python dependencies that will be installed are listed in ``PyRate/requirements.txt``.
19+
Other Python dependencies that will be installed are listed in
20+
``PyRate/requirements.txt``.

docs/history.rst

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Release History
44
===============
55

6-
0.5.0 (2020-08-31)
6+
0.5.0 (2020-09-08)
77
------------------
88
Added
99
+++++
@@ -34,11 +34,16 @@ Changed
3434
files are saved to disk, regardless of whether ``cohmask`` is 0 or 1.
3535
- Parallelisation capability is refactored - MPI and multiprocessing both now use a common
3636
tiling framework for ``stack``, ``timeseries`` and ``mst`` algorithms.
37+
- Introduced a simplified and standardised file naming format for files produced by the
38+
``prepifg`` step. Information previously saved in the filename (e.g. ``ifglksx``, ``ifglksy``,
39+
and ``ifgcropopt`` values) is now saved to the geotiff header instead.
3740

3841
Removed
3942
+++++++
4043
- Redundant ``tscal`` config parameter was deprecated - not needed now there is a ``timeseries``
4144
step invokable on the command line.
45+
- Unused ``Pillow``, ``cython`` and ``glob2`` dependencies.
46+
- Deprecated function ``pyrate.prepifg_helper.prepare_ifgs``, which is no longer needed.
4247

4348
0.4.3 (2020-08-04)
4449
------------------

docs/hpc.rst

Lines changed: 9 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -7,47 +7,34 @@ HPC platforms may differ.
77

88
.. _`National Computational Infrastructure (NCI)`: https://nci.org.au/
99

10-
Login to Gadi and clone the `PyRate` repository:
11-
12-
::
10+
Login to Gadi and clone the `PyRate` repository::
1311

1412
ssh <user_name>@gadi.nci.org.au
1513
cd ~
1614
git clone https://github.com/GeoscienceAustralia/PyRate.git
1715

18-
Load the required Gadi modules (this will also remove the default NCI GDAL
19-
Python bindings so we can build and use our own):
20-
21-
::
22-
23-
source ~/PyRate/scripts/nci_load_modules.sh
24-
25-
Create a Python virtual environment:
26-
27-
::
16+
Create a Python virtual environment::
2817

2918
python3 -m venv ~/PyRateVenv
3019
source ~/PyRateVenv/bin/activate
3120

32-
Install `PyRate`:
21+
Load the required Gadi modules (this will also remove the default NCI GDAL
22+
Python bindings so we can build and use our own)::
23+
24+
source ~/PyRate/scripts/nci_load_modules.sh
3325

34-
::
26+
Install `PyRate`::
3527

3628
cd ~/PyRate
3729
python3 setup.py install
3830

39-
4031
Following this, `PyRate` will be available for PBS jobs. To verify the
41-
installation, first run an interactive session:
42-
43-
::
32+
installation, first run an interactive session::
4433

4534
qsub -I -q express -l walltime=01:00:00,mem=16Gb,ncpus=4,wd
4635

4736
Once the session has started, you will need to reactivate your virtual
48-
environment and reload the required modules:
49-
50-
::
37+
environment and reload the required modules::
5138

5239
source ~/PyRateVenv/bin/activate
5340
source ~/PyRate/scripts/nci_load_modules.sh

docs/installation.rst

Lines changed: 14 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@ Installation
22
============
33

44
This is an installation guide to get `PyRate` running on various platforms.
5-
Follow the instructions to install `PyRate` and run a small toy example.
5+
Follow the instructions to install `PyRate` and run a small toy example using
6+
the test dataset included in the repository.
67

78
.. include:: dependencies.rst
89

@@ -12,14 +13,13 @@ Platforms
1213
PyPI
1314
^^^^
1415

15-
`PyRate` and its Python dependencies can be installed directly from the Python Package Index (PyPI_):
16-
17-
.. _PyPI: https://pypi.org/project/Py-Rate/
18-
19-
::
16+
`PyRate` and its Python dependencies can be installed directly from the
17+
Python Package Index (PyPI_)::
2018

2119
pip install Py-Rate
2220

21+
.. _PyPI: https://pypi.org/project/Py-Rate/
22+
2323
.. include:: ubuntu.rst
2424
.. include:: docker.rst
2525
.. include:: hpc.rst
@@ -28,21 +28,17 @@ PyPI
2828
Verify Installation
2929
-------------------
3030

31-
To verify `PyRate` has been successfully installed, run the workflow with the
32-
example config file and the small dataset included in the repository:
31+
To verify `PyRate` has been successfully installed, run the full processing
32+
workflow with the example config file and the small dataset included
33+
in the repository. If you compiled the ``pyrate`` executable program::
34+
35+
pyrate workflow -f input_parameters.conf
3336

34-
::
37+
If you installed from the Python Package Index (PyPI_)::
3538

36-
pyrate conv2tif -f input_parameters.conf
37-
pyrate prepifg -f input_parameters.conf
38-
pyrate correct -f input_parameters.conf
39-
pyrate timeseries -f input_parameters.conf
40-
pyrate stack -f input_parameters.conf
41-
pyrate merge -f input_parameters.conf
39+
python3 pyrate/main.py workflow -f input_parameters.conf
4240

4341
If the installation has been successful, this workflow will complete without
44-
errors and geotiff files will be available in:
42+
errors and geotiff files will be available in::
4543

46-
::
47-
4844
~/PyRate/out
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
pyrate correct Script
1+
PyRate Correct Script
22
========================
33

4-
.. automodule:: pyrate.process
4+
.. automodule:: pyrate.correct
55
:members:
66
:undoc-members:
77
:show-inheritance:
Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,14 @@
11
PyRate Main Script
22
==================
33

4-
.. automodule:: pyrate.__main__
4+
.. automodule:: pyrate.main
55

66
.. rubric:: Functions
77
.. autosummary::
88

99
conv2tif
1010
prepifg
11-
process
11+
correct
12+
timeseries
13+
stack
1214
merge

docs/scripts.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,6 @@ PyRate Scripts
66

77
pyrate.conv2tif
88
pyrate.prepifg
9-
pyrate.process
9+
pyrate.correct
1010
pyrate.merge
11-
pyrate.__main__
11+
pyrate.main

docs/troubleshooting.rst

Lines changed: 7 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -7,41 +7,6 @@ If your issue is not covered below, please contact the `Geoscience Australia InS
77

88
.. _`Geoscience Australia InSAR Team`: mailto:[email protected]
99

10-
Corrections being skipped
11-
-------------------------
12-
**Problem**: When running the ``process`` step, many corrections are reported as ``Skipped: interferograms already corrected``, but I want to try different processing parameters!
13-
14-
::
15-
16-
>> pyrate correct -f input_parameters.conf
17-
16:43:16 main 97 24732 INFO 0/0 Verbosity set to INFO.
18-
16:43:16 shared 1294 24732 INFO 0/0 Running process serially
19-
16:43:17 process 86 24732 INFO 0/0 Found 13 unique epochs in the 17 interferogram network
20-
16:43:17 process 134 24732 INFO 0/0 Searching for best reference pixel location
21-
16:43:18 process 155 24732 INFO 0/0 Selected reference pixel coordinate: (38, 58)
22-
16:43:18 process 170 24732 INFO 0/0 Calculating orbital correction
23-
16:43:18 shared 1255 24732 INFO 0/0 Skipped: interferograms already corrected
24-
16:43:18 process 198 24732 INFO 0/0 Calculating reference phase
25-
16:43:19 shared 1255 24732 INFO 0/0 Skipped: interferograms already corrected
26-
16:43:19 process 105 24732 INFO 0/0 Calculating minimum spanning tree matrix
27-
16:43:19 process 342 24732 INFO 0/0 Calculating the temporal variance-covariance matrix
28-
16:43:20 process 391 24732 INFO 0/0 Calculating time series using SVD method
29-
16:43:20 timeseries 152 24732 INFO 0/0 Calculating timeseries in serial
30-
16:43:21 process 323 24732 INFO 0/0 Calculating rate map from stacking
31-
16:43:21 process 326 24732 INFO 0/0 Stacking of tile 0
32-
16:43:21 stack 64 24732 INFO 0/0 Calculating stack rate in serial
33-
16:43:21 process 314 24732 INFO 0/0 PyRate workflow completed
34-
35-
**Reason**: `PyRate` updates the phase values in the input interferogram geotiff files as corrections are applied during the ``process`` step. Metadata is then added to the geotiff header to indicate the correction has been applied. This metadata is then checked upon subsequent runs to see if the correction should be applied.
36-
37-
**Solution**: Start again from ``prepifg`` step, creating new cropped/multi-looked interferograms that have not been corrected.
38-
39-
.. note::
40-
41-
We plan to change this workflow behaviour in a future `PyRate` release, recognising that
42-
it would be convenient to be able to quickly test the impact of parameter changes.
43-
44-
4510
ValueError: too many values to unpack (expected 2)
4611
--------------------------------------------------
4712
**Problem**: During ``prepifg`` step, the following error is encountered:
@@ -104,12 +69,12 @@ Stack Rate map appears to be blank/empty
10469

10570
**Reason**: The ``maxsig`` parameter is too low, resulting in stack rate values being replaced by NaNs. ``maxsig`` is a threshold for masking stack rate pixels according to the corresponding stack error estimate saved in ``out/tmpdir/stack_error_*.npy``.
10671

107-
**Solution**: Increase ``maxsig``, then re-run ``process`` and ``merge`` steps. Maximum permittable value for ``maxsig`` is 1000 mm.
72+
**Solution**: Increase ``maxsig``, then re-run the ``merge`` step. Maximum permittable value for ``maxsig`` is 1000 mm.
10873

10974

11075
Failure of APS spatial low pass filter
11176
---------------------------------------
112-
**Problem**: Atmospheric corrections during “process“ fails due to the interpolated grid used for correction being empty:
77+
**Problem**: Atmospheric corrections during ``correct`` step fails due to the interpolated grid used for correction being empty:
11378

11479
::
11580

@@ -155,8 +120,8 @@ Failure of APS spatial low pass filter
155120
In general, users are advised to input a network of small-baseline interferograms
156121
that has at least 2 interferometric connections per SAR image epoch. Furthermore,
157122
make sure that ``ts_pthr``, ``pthr`` and ``tlpfpthr`` are smaller than the number
158-
of image epochs. To check that ``process`` worked correctly, users can check that
159-
the ``tsincr_*.npy`` and ``tscuml*.npy`` arrays in the ``/<outdir>/tmpdir`` contain numeric values and not NaNs.
123+
of image epochs. To check that the spatio-temporal filters worked correctly, users can check that
124+
the numpy arrays saved at ``/<outdir>/aps_error/*aps_error.npy`` contain numeric values and not NaNs.
160125

161126

162127
Out of memory errors
@@ -167,12 +132,12 @@ Out of memory errors
167132

168133
joblib.externals.loky.process_executor.TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker. The exit codes of the workers are {EXIT(1), EXIT(1), EXIT(1)}
169134

170-
**Solution**: Increase the amount of memory available. On HPC systems this can be done by increasing the value provided to the ``mem`` argument when submitting a PBS job, e.g.:
171-
172-
::
135+
**Solution**: Increase the amount of memory available. On HPC systems this can be done by increasing the value provided to the ``mem`` argument when submitting a PBS job, e.g.::
173136

174137
mem=32Gb
175138

139+
If no more memory can be called upon, users can try running the job in serial, or reducing the size of the interferograms by increasing the multi-looking factors applied during ``prepifg`` (parameters ``ifglksx`` and ``ifglksy``).
140+
176141
Incorrect modules loaded on Gadi
177142
----------------------------------
178143
**Problem**: `PyRate` requires certain versions of Python, GDAL and Open MPI to be loaded on Gadi and other HPC systems. While sourcing the `PyRate/scripts/nci_load_modules.sh` script will load the correct modules, you may need to unload previously unloaded modules.

docs/ubuntu.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,5 @@ Clone the repository, install required packages and install `PyRate`:
1717
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/usr/include/gdal
1818

1919
python3 setup.py install
20+
21+
These commands will compile the executable program ``pyrate``.

0 commit comments

Comments
 (0)