Skip to content

Commit ac3126e

Browse files
davidfrantzdavidfrantz
committed
dev history wrap-up
1 parent 696c8ca commit ac3126e

File tree

2 files changed

+145
-131
lines changed

2 files changed

+145
-131
lines changed

‎docs/source/history/v3/v378.rst‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
.. _v378:
2+
3+
FORCE v. 3.7.8
4+
==============
5+
6+
Release: 05.09.2022
7+
8+
- **General**
9+
10+
- *master* branch was renamed to *main*
11+
12+
- FORCE comes with some default output file formats,
13+
most noteably the striped GTiff default format with LZW compression.
14+
15+
It is now possible that users can define their own output file format with custom GDAL options.
16+
There is now a new parameter: ``FILE_OUTPUT_OPTIONS``.
17+
This expects a file, and will only be activated when ``OUTPUT_FORMAT = CUSTOM``.
18+
19+
The text file should be styled in tag and value notation like this:
20+
21+
| DRIVER = GTiff
22+
| EXTENSION = tif
23+
| BIGTIFF = YES
24+
| COMPRESS = ZSTD
25+
| PREDICTOR = 2
26+
| ZLEVEL = 1
27+
| INTERLEAVE = BAND
28+
| TILED = YES
29+
| BLOCKXSIZE = 1024
30+
| BLOCKYSIZE = 1024
31+
32+
**Important:** the file needs at least the ``DRIVER`` (GDAL short driver name) and ``EXTENSION``,
33+
and then a variable number of GDAL options (up to 32 - this should be enough, right?).
34+
35+
Some thoughts of caution: with opening this up to the user,
36+
it is now possible to give invalid or conflicting options that result in the failure of creating files.
37+
38+
Thanks to Max Freudenberg for suggesting and testing this feature.
39+
40+
- A new feature was added to write a provenance table to the output directory to trace input-output relations
41+
42+
- **FORCE L1AS**
43+
44+
- Stefan Ernst added a note to ``force-level1-csd`` that warns the user when attempting to download Landsat data.
45+
See here: https://github.com/davidfrantz/force/discussions/167
46+
47+
- **FORCE L2PS**
48+
49+
- scattered nodata pixels occured in Sentinel-2 imagery at random locations.
50+
This is because the cirrus band TOA reflectance can be 0 or negative over dark surfaces (e.g. water or shadow).
51+
As this is physically impossible (and we cannot recover from this during atmospheric correction), FORCE screens
52+
for *bad* pixels and masks them out.
53+
54+
This condition was now relaxed for the cirrus band.
55+
Thanks to Max Helleis for bringing this up.
56+
57+
- Added a small hiccup when parsing sensor ID.
58+
Was only relevant when it failed by giving wrong input data.
59+
60+
- Fixed an issue when outputting overview thumbnails (OVV) AND using a geographic output projection (i.e. resolution < 1).
61+
Thanks to GitHub user dkczk for bringing this up.
62+
63+
- **FORCE HLPS**
64+
65+
- deprecation note:
66+
67+
The option to use the SPLITS algorithm to derive LSP metrics in ``force-higher-level``
68+
will be deprecated with version 3.7.9.
69+
70+
Please use the polar-based LSP retrieval method instead (recommended anyway).
71+
72+
- in ``force-higher-level``:
73+
since compiling against python for developing the UDF sub-module,
74+
``force-higher-level`` did not honor ``ctrl+c`` anymore to abort the
75+
processing, and the program needed to be killed explicitly.
76+
Florian Katerndahl provided a fix for this!
77+
``ctrl+c`` works again.
78+
79+
- in ``force-higher-level``, TSA sub-module:
80+
81+
- New indices were added:
82+
83+
- CCI (provided by J. Antonio Guzmán Q.)
84+
- EVI2 (suggested by Jonas Ardo)
85+
- Continuum Removal of SWIR1
86+
87+
- A recipe for adding a new index was added to the repository in order to facilitate users the implementation of additional indices.
88+
- J. Antonio Guzmán Q. additionally changed the scaling factor for simple ratio indices like CIre (was 10000, is 1000).
89+
90+
- in ``force-higher-level``, TSA sub-module:
91+
added the harmonic interpolation method from Zhu et al. 2015 (http://dx.doi.org/10.1016/j.rse.2015.02.009).
92+
This can be used with ``INTERPOLATE = HARMONIC``.
93+
``HARMONIC_MODES = 3`` defines how many modes per season are used,
94+
i.e. uni-modal (1), bi-modal (2), or tri-modal (3).
95+
96+
- in ``force-higher-level``, TSA sub-module:
97+
added a simple near-real time monitoring component.
98+
When using the harmonic interpolation, the user can fit the harmonic to a subset of the time period with
99+
``HARMONIC_FIT_RANGE``.
100+
101+
For example, if the analysis timeframe is ``DATE_RANGE = 2015-01-01 2022-06-20``,
102+
all data from 2015-2022 will be considered. If ``HARMONIC_FIT_RANGE = 2015-01-01 2017-12-31``,
103+
the harmonic will only be fitted to the first 3 years of data.
104+
The new NRT product (``OUTPUT_NRT = TRUE``) will then contain the residual between the
105+
extrapolated harmonic and the actual data from 2018 on.
106+
107+
This can be used to identify changes in the present relative to the "usual" seasonality observed in the past.
108+
109+
This method is inspired by a presentation of K. Ose, R. Dutrieux, and J.-B. Féret (INRAE) at LPS2022.
110+
111+
- in ``force-higher-level``, TSA sub-module (and probably others):
112+
Stefan Ernst noted that the TSA submodule did not produce any output when using Landsat 7, Landsat 9 and Sentinel-2 simultaneously.
113+
This gives us the unusual situation that we have >= 3 observations per day.
114+
115+
This caused a divide-by-0 error in the linear interpolation that is used for detecting remaining noise in the time series.
116+
This is now fixed by simply computing the average when we have more then 2 obs/day.
117+
118+
- in ``force-higher-level``, ML sub-module:
119+
a stack smashing bug occured when using more than 8 modelsets.
120+
This is now fixed. Thanks to Fabian Thiel for finding this.
121+
122+
- **FORCE AUX**
123+
124+
- in ``force-qai-inflate``:
125+
changed output nodata from 1 (which is a valid value) to 255.
126+
Thanks to Fabian Thiel for bringing this up.
127+
128+
- in ``force-lut-modis``:
129+
Vincent Schut reportet that the program does not properly detect a 504 response from the server,
130+
and hangs infinitely.
131+
Thanks to Florian Katerndahl for adding a fix that catches HTTP responses >= 400.
132+
133+
- in ``force-cube``:
134+
If a resulting image is completely nodata, it will automatically be removed.
135+
This works now for both raster-to-raster and vector-to-raster.
136+
Empty tiles will be removed as well.
137+
Thanks for László Henits for bringing this up.
138+
Thanks to Stefan Ernst for suggesting a fix.
139+
140+
- in ``force-cube``:
141+
Changed the default resampling to nearest neighbor in accordance with the gdalwarp utility.
142+
**Make sure to adapt your workflows if necessary.**
143+
Thanks to Marcel Schwieder for suggesting and modifying.
144+

‎docs/source/history/vdev.rst‎

Lines changed: 1 addition & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -3,134 +3,4 @@
33
Develop version
44
===============
55

6-
- **General**
7-
8-
- *master* branch was renamed to *main*
9-
10-
- FORCE comes with some default output file formats,
11-
most noteably the striped GTiff default format with LZW compression.
12-
13-
It is now possible that users can define their own output file format with custom GDAL options.
14-
There is now a new parameter: ``FILE_OUTPUT_OPTIONS``.
15-
This expects a file, and will only be activated when ``OUTPUT_FORMAT = CUSTOM``.
16-
17-
The text file should be styled in tag and value notation like this:
18-
19-
| DRIVER = GTiff
20-
| EXTENSION = tif
21-
| BIGTIFF = YES
22-
| COMPRESS = ZSTD
23-
| PREDICTOR = 2
24-
| ZLEVEL = 1
25-
| INTERLEAVE = BAND
26-
| TILED = YES
27-
| BLOCKXSIZE = 1024
28-
| BLOCKYSIZE = 1024
29-
30-
**Important:** the file needs at least the ``DRIVER`` (GDAL short driver name) and ``EXTENSION``,
31-
and then a variable number of GDAL options (up to 32 - this should be enough, right?).
32-
33-
Some thoughts of caution: with opening this up to the user,
34-
it is now possible to give invalid or conflicting options that result in the failure of creating files.
35-
36-
Thanks to Max Freudenberg for suggesting and testing this feature.
37-
38-
- A new feature was added to write a provenance table to the output directory to trace input-output relations
39-
40-
- **FORCE L2PS**
41-
42-
- scattered nodata pixels occured in Sentinel-2 imagery at random locations.
43-
This is because the cirrus band TOA reflectance can be 0 or negative over dark surfaces (e.g. water or shadow).
44-
As this is physically impossible (and we cannot recover from this during atmospheric correction), FORCE screens
45-
for *bad* pixels and masks them out.
46-
47-
This condition was now relaxed for the cirrus band.
48-
Thanks to Max Helleis for bringing this up.
49-
50-
- Added a small hiccup when parsing sensor ID.
51-
Was only relevant when it failed by giving wrong input data.
52-
53-
- Fixed an issue when outputting overview thumbnails (OVV) AND using a geographic output projection (i.e. resolution < 1).
54-
Thanks to GitHub user dkczk for bringing this up.
55-
56-
- **FORCE HLPS**
57-
58-
- deprecation note:
59-
60-
The option to use the SPLITS algorithm to derive LSP metrics in ``force-higher-level``
61-
will be deprecated with version 3.7.9.
62-
63-
Please use the polar-based LSP retrieval method instead (recommended anyway).
64-
65-
- in ``force-higher-level``:
66-
since compiling against python for developing the UDF sub-module,
67-
``force-higher-level`` did not honor ``ctrl+c`` anymore to abort the
68-
processing, and the program needed to be killed explicitly.
69-
Florian Katerndahl provided a fix for this!
70-
``ctrl+c`` works again.
71-
72-
- in ``force-higher-level``, TSA sub-module:
73-
74-
- New indices were added:
75-
76-
- CCI (provided by J. Antonio Guzmán Q.)
77-
- EVI2 (suggested by Jonas Ardo)
78-
- Continuum Removal of SWIR1
79-
80-
- A recipe for adding a new index was added to the repository in order to facilitate users the implementation of additional indices.
81-
- J. Antonio Guzmán Q. additionally changed the scaling factor for simple ratio indices like CIre (was 10000, is 1000).
82-
83-
- in ``force-higher-level``, TSA sub-module:
84-
added the harmonic interpolation method from Zhu et al. 2015 (http://dx.doi.org/10.1016/j.rse.2015.02.009).
85-
This can be used with ``INTERPOLATE = HARMONIC``.
86-
``HARMONIC_MODES = 3`` defines how many modes per season are used,
87-
i.e. uni-modal (1), bi-modal (2), or tri-modal (3).
88-
89-
- in ``force-higher-level``, TSA sub-module:
90-
added a simple near-real time monitoring component.
91-
When using the harmonic interpolation, the user can fit the harmonic to a subset of the time period with
92-
``HARMONIC_FIT_RANGE``.
93-
94-
For example, if the analysis timeframe is ``DATE_RANGE = 2015-01-01 2022-06-20``,
95-
all data from 2015-2022 will be considered. If ``HARMONIC_FIT_RANGE = 2015-01-01 2017-12-31``,
96-
the harmonic will only be fitted to the first 3 years of data.
97-
The new NRT product (``OUTPUT_NRT = TRUE``) will then contain the residual between the
98-
extrapolated harmonic and the actual data from 2018 on.
99-
100-
This can be used to identify changes in the present relative to the "usual" seasonality observed in the past.
101-
102-
103-
- in ``force-higher-level``, TSA sub-module (and probably others):
104-
Stefan Ernst noted that the TSA submodule did not produce any output when using Landsat 7, Landsat 9 and Sentinel-2 simultaneously.
105-
This gives us the unusual situation that we have >= 3 observations per day.
106-
107-
This caused a divide-by-0 error in the linear interpolation that is used for detecting remaining noise in the time series.
108-
This is now fixed by simply computing the average when we have more then 2 obs/day.
109-
110-
- in ``force-higher-level``, ML sub-module:
111-
a stack smashing bug occured when using more than 8 modelsets.
112-
This is now fixed. Thanks to Fabian Thiel for finding this.
113-
114-
- **FORCE AUX**
115-
116-
- in ``force-qai-inflate``:
117-
changed output nodata from 1 (which is a valid value) to 255.
118-
Thanks to Fabian Thiel for bringing this up.
119-
120-
- in ``force-lut-modis``:
121-
Vincent Schut reportet that the program does not properly detect a 504 response from the server,
122-
and hangs infinitely.
123-
Thanks to Florian Katerndahl for adding a fix that catches HTTP responses >= 400.
124-
125-
- in ``force-cube``:
126-
If a resulting image is completely nodata, it will automatically be removed.
127-
This works now for both raster-to-raster and vector-to-raster.
128-
Empty tiles will be removed as well.
129-
Thanks for László Henits for bringing this up.
130-
Thanks to Stefan Ernst for suggesting a fix.
131-
132-
- in ``force-cube``:
133-
Changed the default resampling to nearest neighbor in accordance with the gdalwarp utility.
134-
Thanks to Marcel Schwieder for suggesting and modifying.
135-
136-
#-- No further changes yet.
6+
-- No further changes yet.

0 commit comments

Comments
 (0)