Skip to content

Commit 07a2b44

Browse files
Overhaul documentation for how to make a release
1 parent 49e0cc6 commit 07a2b44

File tree

4 files changed

+355
-252
lines changed

4 files changed

+355
-252
lines changed

INSTALLGUIDE.rst

Lines changed: 36 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Precompiled binaries (Linux and macOS)
1212

1313
ASP builds are for Linux and OSX (with the Intel processor).
1414
ASP was verified to work under Microsoft Windows using the Windows
15-
Subsystem for Linux, and under the OSX M1 processor using Rosetta.
15+
Subsystem for Linux, and under the OSX M1 processor using Rosetta 2.
1616

1717
Choose either the latest build (recommended) or a stable release
1818
from:
@@ -22,8 +22,8 @@ from:
2222
No installation steps or administrative rights are necessary. Extract
2323
the archive, and run the executables in the ``bin`` subdirectory as::
2424

25-
tar -xvf StereoPipeline-3.2.0-2022-12-30-x86_64-Linux.tar.bz2
26-
./StereoPipeline-3.2.0-2022-12-30-x86_64-Linux/bin/stereo --help
25+
tar xvf StereoPipeline-3.3.0-2023-08-16-x86_64-Linux.tar.bz2
26+
./StereoPipeline-3.3.0-2023-08-16-x86_64-Linux/bin/stereo --help
2727

2828
The result of the last command should be a help message.
2929

@@ -55,7 +55,10 @@ to process terrestrial images, skip to :numref:`dg_tutorial`.
5555
To perform pre-processing (radiometric calibration, ephemeris
5656
processing, etc.), of non-terrestrial images prior to running Stereo
5757
Pipeline, you will need to install :term:`ISIS`. Just as with our
58-
binaries, you can use the ISIS binaries as-is.
58+
binaries, you can use the ISIS binaries as-is. See the ISIS installation
59+
instructions at:
60+
61+
https://github.com/DOI-USGS/ISIS3
5962

6063
ASP has its own self-contained version of the ISIS libraries, and on
6164
occasion it is behind the latest ISIS (see the release notes in
@@ -65,14 +68,17 @@ those work with the version of ISIS shipped with ASP, assuming no
6568
major changes have taken place in the data formats or camera models by
6669
the ISIS developers. At the very least, you should be able to install
6770
the older standalone version of ISIS that ASP uses if a failure is
68-
found. To do so, follow the ISIS installation instructions, but
69-
create a new conda environment (not the one with your current ISIS),
70-
and right before you would run ``conda install isis``, run ``conda
71-
search isis`` to find all of the versions of ISIS available for
72-
installation. For example, if you wanted to install ISIS 7.1.0, and
73-
it is available in the ``conda search isis`` listing, you can run
74-
``conda install isis=7.1.0`` and then follow the remainder of the ISIS
75-
installation instructions.
71+
found.
72+
73+
To install a specific version of ISIS, follow the ISIS installation
74+
instructions, but create a new conda environment (not the one with your current
75+
ISIS), and right before you would run ``conda install isis``, run ``conda search
76+
isis`` to find all of the versions of ISIS available for installation. For
77+
example, if you wanted to install ISIS 8.0.0, run::
78+
79+
conda install -c usgs-astrogeology -c conda-forge isis=8.0.0
80+
81+
and then follow the remainder of the ISIS installation instructions.
7682

7783
In closing, running the Stereo Pipeline executables only requires
7884
that you have downloaded the ISIS secondary data and have
@@ -92,10 +98,10 @@ Using the packaged ASP tarball
9298
https://github.com/NeoGeographyToolkit/StereoPipeline/releases
9399

94100
#. Fetch ISIS binaries and install, following
95-
https://github.com/USGS-Astrogeology/ISIS3#installation
101+
https://github.com/DOI-USGS/ISIS3#installation
96102

97103
#. Fetch ISIS data, as detailed at
98-
https://github.com/USGS-Astrogeology/ISIS3#the-isis-data-area
104+
https://github.com/DOI-USGS/ISIS3#the-isis-data-area
99105

100106
#. Add the ISIS executables to your path:
101107

@@ -110,9 +116,9 @@ Using the packaged ASP tarball
110116
111117
Check that you have the directory ``$ISISDATA/base``.
112118

113-
#. Untar Stereo Pipeline::
119+
#. Extract Stereo Pipeline::
114120

115-
tar xzvf StereoPipeline-<VERSION>-<ARCH>-<OS>.tar.gz
121+
tar xvf StereoPipeline-<VERSION>-<ARCH>-<OS>.tar.bz2
116122

117123
#. Add Stereo Pipeline to your path:
118124

@@ -135,9 +141,9 @@ Quick start for Digital Globe users
135141
#. Fetch Stereo Pipeline from
136142
https://github.com/NeoGeographyToolkit/StereoPipeline/releases
137143

138-
#. Untar Stereo Pipeline::
144+
#. Extract Stereo Pipeline::
139145

140-
tar xzvf StereoPipeline-<VERSION>-<ARCH>-<OS>.tar.gz
146+
tar xvf StereoPipeline-<VERSION>-<ARCH>-<OS>.tar.bz2
141147

142148
#. Add StereoPipeline to your path, as above.
143149

@@ -209,7 +215,7 @@ Fetching pre-compiled ASP with conda
209215

210216
ASP's official releases can be fetched with ``conda``, for Linux and OSX with the Intel processor. See :numref:`precompiled_binaries` for how other platforms may be handled.
211217

212-
The latest conda release is version 3.2.0, published on December 30, 2022. See
218+
The latest conda release is version 3.3.0, published on December 30, 2022. See
213219
:numref:`news` for what changed since then. The newer functionality
214220
is available only with the daily build (:numref:`precompiled_binaries`).
215221

@@ -272,12 +278,19 @@ and not all environments.
272278

273279
Install ASP with the command::
274280

275-
conda install stereo-pipeline==3.2.0
281+
conda install \
282+
-c nasa-ames-stereo-pipeline \
283+
-c usgs-astrogeology \
284+
-c conda-forge \
285+
stereo-pipeline==3.3.0
276286

277-
This will install ASP 3.2.0 together with ISIS 7.1.0. Note that the
287+
This will install ASP 3.3.0 together with ISIS 8.0.0. Note that the
278288
latest build (see above) may have more fixes or features than this
279289
official release.
280290

291+
Alternatively, consider using ``mamba`` instead of ``conda``. It is
292+
must faster though it is not always guaranteed to work.
293+
281294
If using ISIS, the environmental variable ISISROOT should be set to
282295
point to this distribution, such as::
283296

@@ -293,11 +306,11 @@ found as a set of .yaml files in the ``conda`` subdirectory of the
293306
Stereo Pipeline GitHub repository. So, alternatively, the installation
294307
can happen as::
295308

296-
conda env create -f asp_3.2.0_linux_env.yaml
309+
conda env create -n asp -f asp_3.3.0_linux_env.yaml
297310

298311
or::
299312

300-
conda env create -f asp_3.2.0_osx_env.yaml
313+
conda env create -n asp -f asp_3.3.0_osx_env.yaml
301314

302315
depending on your platform. Then invoke, as earlier::
303316

RELEASEGUIDE

Lines changed: 1 addition & 196 deletions
Original file line numberDiff line numberDiff line change
@@ -1,196 +1 @@
1-
HOW TO MAKE A RELEASE
2-
3-
Here's a developer's checklist for performing a binary Ames Stereo
4-
Pipeline (ASP) release.
5-
6-
1.) Modify the build system:
7-
8-
Inside both the ASP and VW src/CMakeLists.txt set the new
9-
version. After the last release (x.y.0), the ASP version is likely
10-
x.y.0-alpha, and so you just need to change it to x.y.0 if this is a
11-
feature release. If this is a bugfix release or a major release
12-
instead, then set the version number of the release appropriately (see
13-
https://semver.org for guidance).
14-
15-
2.) Update NEWS.rst.
16-
17-
3.) Commit all changes.
18-
19-
4.) Read and update the documentation.
20-
21-
5.) Build and package the software and documentation:
22-
23-
- Fetch the latest ISIS using conda.
24-
25-
- Fetch the recipes for all ASP dependencies from:
26-
27-
https://github.com/NeoGeographyToolkit
28-
29-
(all of the repositories there ending with "feedstock"). Ensure
30-
that all versions there agree with the versions from the ISIS
31-
environment. Rebuild them with conda-build and commit them to:
32-
33-
https://anaconda.org/nasa-ames-stereo-pipeline
34-
35-
This is described in detail in :numref:`conda_build` in the
36-
documentation.
37-
38-
- Make the conda dependencies of:
39-
40-
https://github.com/NeoGeographyToolkit/MultiView
41-
42-
compatible with the dependencies so far, then build and install
43-
that package in the same directory as VisionWorkbench and
44-
StereoPipeline.
45-
46-
- Create a local conda environment and install there all the
47-
packages just built, including VisionWorkbench and StereoPipeline.
48-
This will ensure there are no conflicts.
49-
50-
The obtained current environment can be exported with the command:
51-
52-
conda env export > asp.yml
53-
54-
A copy of this file can be made, called ``asp_deps.yml``,
55-
with the VisionWorkbench and StereoPipeline packages removed from
56-
it, then edited to change the ``name`` field on top and removing
57-
the ``prefix`` field at the bottom. Then this file can be used to
58-
create an environment having only the ASP dependencies with the
59-
command:
60-
61-
conda env create -f asp_deps.yml
62-
63-
Save a record of these in StereoPipeline/conda. It helps with future
64-
reproducibility.
65-
66-
- Check out https://github.com/NeoGeographyToolkit/BinaryBuilder
67-
in your $HOME.
68-
69-
- ISIS expects a full Python distribution to be shipped. For now
70-
we handle this in an awkward way. A fresh conda environemnt
71-
is created having only Python, numpy, with versions
72-
as expected by current ISIS, for example, as:
73-
74-
conda create -n python_isis7 -y python=3.9.15 numpy=1.23.5
75-
76-
This directory is packaged later with the build with make-dist.py
77-
(that tool needs to be edited to ensure the path to this
78-
environemnt is used). See also BinaryBuilder/README.
79-
80-
- Update the path to your local conda environment having the latest
81-
ASP dependencies in BinaryBuilder/auto_build/utils.sh.
82-
83-
- Update this path as well in StereoPipelineTest, in the
84-
release*conf files, to be used for the nightly regressions.
85-
86-
- Ensure your conda environment does not have VisionWorkbench and
87-
Stereo Pipeline include files, libraries, or executables, as those
88-
will conflict with building locally.
89-
90-
- Update the ISIS version in README.rst, INSTALLGUIDE.rst, and
91-
building_asp.rst (there can be many mentions in each).
92-
93-
6.) Build and package the ASP release tarball. See:
94-
95-
BinaryBuilder/README.
96-
97-
- Build the documentation in StereoPipeline/docs. See
98-
https://stereopipeline.readthedocs.io/en/latest/installation.html#building-the-documentation
99-
for how to fetch sphinx. Run:
100-
101-
make html
102-
builds a local copy of the online documentation
103-
104-
make latexpdf
105-
builds 'the pdf book'
106-
107-
- Copy the pdf book from docs/_build/latex/asp_book.pdf to
108-
$HOME/BinaryBuilder/dist-add/asp_book.pdf.
109-
110-
Ideally all the dependencies are already built and installed
111-
with conda as above. VisionWorkbench can be built and installed
112-
either with conda or with BinaryBuilder.
113-
114-
Use the compilers provided by conda (on Linux).
115-
116-
Test all binary releases on different machines by running
117-
StereoPipelineTest.
118-
119-
NOTE: All items in this step are done automatically by running on
120-
lunokhod1 the script
121-
122-
$HOME/projects/BinaryBuilder/auto_build/launch_master.sh
123-
124-
The resulting release tarballs will go to BinaryBuilder/asp_tarballs.
125-
126-
7.) Check that the ASP version is correct in the documentation
127-
file.
128-
129-
8.) Commit and tag the release in the VW and ASP repos. Example:
130-
131-
git tag 3.2.0
132-
133-
Push the tags to the remote server with:
134-
135-
git push origin 3.2.0 # commit to your branch
136-
git push god 3.2.0 # commit to main branch
137-
138-
(Here it is assumed that 'origin' points to your own fork and 'god'
139-
points to the parent repository.)
140-
141-
If more commits were made and it is desired to apply this tag to a
142-
different commit, first remove the exiting tag with:
143-
144-
git tag -d 3.2.0
145-
git push origin :refs/tags/3.2.0
146-
git push god :refs/tags/3.2.0
147-
148-
9.) Create a new release on GitHub, and upload to it the builds and
149-
the pdf documentation. Use the just-created tag. Do not just rename
150-
the latest uploaded daily build, as that will create an incorrect
151-
Zenodo link. Paste in the release area the current release notes from
152-
NEWS.txt.
153-
154-
Files can be uploaded to a release from the browser, or from the
155-
command line, with the GitHub "gh" tool. Here's an example usage:
156-
157-
cd BinaryBuilder
158-
for file in asp_tarballs/StereoPipeline-3.2.0-2023-01-01-x86_64-Linux.tar.bz2 \
159-
asp_tarballs/StereoPipeline-3.2.0-2023-01-01-x86_64-OSX.tar.bz2 \
160-
dist-add/asp_book.pdf; \
161-
do
162-
gh release upload 3.2.0 $file \
163-
-R git@github.com:NeoGeographyToolkit/StereoPipeline.git
164-
done
165-
166-
Wait a few minutes for Zenodo to mint a link, then visit the Zenodo
167-
page at https://zenodo.org/badge/latestdoi/714891 (which will always
168-
point to the 'latest' DOI) and find there this release's URL and put
169-
it at the top of README.rst, in the appropriate ``target`` field.
170-
Increment the version in the ``image`` field right above that.
171-
Push this update.
172-
173-
Get the text on the GitHub release page right the first time, as once
174-
Zenodo picks it up, the Zenodo page cannot be modified. But the binaries
175-
attached to the release can be modified or added later.
176-
177-
10.) Send an announcement of the new release to the mailing list, at
178-
https://groups.google.com/forum/\#!forum/ames-stereo-pipeline-support
179-
and the old stereo-pipeline@lists.nasa.gov, listing the changes
180-
specified in NEWS.rst.
181-
182-
11.) Modify the build system.
183-
184-
After the release has been done, prep the repo for the next phase of
185-
development, by updating the version number in:
186-
187-
- src/CMakeLists.txt
188-
189-
in both the VW and ASP repositories.
190-
191-
If version 3.2.0 just got released, we expect that the next feature
192-
release will be 3.3.0, if a major release, or 3.2.1 if a minor
193-
release, so the version tag should be updated to 3.2.1-alpha in
194-
anticipation (see https://semver.org for guidance).
195-
196-
12.) Update this document as it becomes necessary.
1+
See https://stereopipeline.readthedocs.io/en/latest/building_asp.html.

0 commit comments

Comments
 (0)