You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -9,21 +9,15 @@ This tutorial explains how to generate and use processing masks in the FORCE Hig
9
9
10
10
.. admonition:: Info
11
11
12
-
*This tutorial uses FORCE v. 3.0*
13
-
14
-
15
-
.. important::
16
-
17
-
EDIT: Part of this tutorial needs updating: Option 2.
18
-
12
+
*This tutorial uses FORCE v. 3.7.6*
19
13
20
14
21
15
What are processing masks?
22
16
--------------------------
23
17
24
18
In the FORCE Higher Level Processing System, processing masks can be used to restrict processing and analysis to certain pixels of interest.
25
19
The masks need to be in datacube format, i.e. they need to be raster images in the same grid as all the other data.
26
-
The masks can - but don’t need to - be in the same directory as the other data.
20
+
The masks can - but don't need to - be in the same directory as the other data.
27
21
The masks should be binary images.
28
22
The pixels that have a mask value of 0 will be skipped.
29
23
@@ -40,49 +34,71 @@ What is the advantage of using processing masks?
40
34
As an example, when computing a tree species classification, you can speed up processing substantially if you provide a forest masks.
41
35
- Processing masks decrease data volume substantially.
42
36
- In the processed products, the pixels of no interest have a nodata value.
43
-
As all FORCE output is compressed (unless you choose to output in ENVI format; I don’t recommend to do this), the compression kicks in nicely if you have used processing masks.
37
+
As all FORCE output is compressed (unless you choose to output in ENVI format; I don't recommend to do this), the compression kicks in nicely if you have used processing masks.
44
38
You can easily decrease data volume by several factors.
45
39
- Processing masks facilitate analyzing the processed data.
46
40
- In the processed products, the pixels of no interest have a nodata value.
47
-
Thus, you don’t need to sort the pixels on your own, e.g. computing confusion matrices and classification accuracy is more straightforward to implement.
41
+
Thus, you don't need to sort the pixels on your own, e.g. computing confusion matrices and classification accuracy is more straightforward to implement.
48
42
49
43
50
44
Generate processing masks
51
45
-------------------------
52
46
53
-
Option 1: from shapefile to mask
54
-
""""""""""""""""""""""""""""""""
47
+
Option 1: from vector data to mask
48
+
""""""""""""""""""""""""""""""""""
55
49
56
-
FORCE comes with a program to generate processing masks from a shapefile:
50
+
FORCE comes with a program to generate processing masks from vector data (e.g. shapefile or geopackage):
57
51
58
52
.. code-block:: bash
59
53
60
-
force-cube
54
+
force-cube -h
55
+
56
+
Usage: force-cube [-hvirsantobj] input-file(s)
57
+
58
+
optional:
59
+
-h = show this help
60
+
-v = show version
61
+
-i = show program's purpose
62
+
-r = resampling method
63
+
any GDAL resampling method for raster data, e.g. cubic (default)
64
+
is ignored for vector data
65
+
-s = pixel resolution of cubed data, defaults to 10
66
+
-a = optional attribute name for vector data. force-cube will burn these values
67
+
into the output raster. default: no attribute is used; a binary mask
68
+
with geometry presence (1) or absence (0) is generated
69
+
-l = layer name for vector data (default: basename of input, without extension)
70
+
-n = output nodate value (defaults to 255)
71
+
-t = output data type (defaults to Byte; see GDAL for datatypes;
72
+
but note that FORCE HLPS only understands Int16 and Byte types correctly)
73
+
-o = output directory: the directory where you want to store the cubes
74
+
defaults to current directory
75
+
'datacube-definition.prj' needs to exist in there
76
+
-b = basename of output file (without extension)
77
+
defaults to the basename of the input-file
78
+
cannot be used when multiple input files are given
79
+
-j = number of jobs, defaults to 'as many as possible'
80
+
81
+
mandatory:
82
+
input-file(s) = the file(s) you want to cube
83
+
84
+
-----
85
+
see https://force-eo.readthedocs.io/en/latest/components/auxilliary/cube.html
output-dir: the directory you want to store the cubes;
65
-
datacube-definition.prj needs to exist in there
66
-
resample: resampling method
67
-
(1) any GDAL resampling method for raster data, e.g. cubic
68
-
(2) rasterize for vector data
69
-
resolution: the resolution of the cubed data
70
87
71
88
72
89
``force-cube`` imports raster or vector data into the datacube format needed by FORCE.
73
90
The output directory needs to contain a copy of the datacube definition (see datacube tutorial).
74
91
75
-
The ``rasterize`` resampling option rasterizes polygon vector geometries.
76
-
It burns the occurence of the geometry into a raster image, i.e. it assigns the value *1* to all cells that are covered by a geometry, *0* if not.
92
+
If used with vector data, the tool rasterizes the polygon vector geometries.
93
+
By default, it burns the occurence of the geometry into a raster image, i.e. it assigns the value *1* to all cells that are covered by a geometry, *0* if not.
77
94
The resulting masks are compressed GeoTiff images.
78
95
Do not worry about data volume when converting from vector to raster data, because the compression rate is extremely high.
79
96
80
97
In the following example, we generate a processing mask for the administrative area of Vienna, Austria.
*Mask of Vienna generated from a shapefile. Overlayed with the processing grid in green*
122
144
123
145
124
-
Option 2: from raster to mask
125
-
"""""""""""""""""""""""""""""
126
-
127
-
As of now, FORCE does not come with a handy tool to generate masks from a raster image with continuous values (this is on my to-do list though).
128
-
However, you can follow this recipe to accomplish this.
129
-
130
-
.. important::
131
-
132
-
EDIT: This tool already exists for a while, ``force-procmask``. This part of the tutorial needs updating.
133
-
146
+
Option 2: from raster data to mask
147
+
"""""""""""""""""""""""""""""""""
134
148
135
-
In the example given below, our input image is a multiband continuous fields dataset, which gives the percentages of built-up land (urban), high vegetation (trees), and low vegetation (grass, agriculture).
136
-
Point 1) may be skipped if the data are already in datacube format, which is the case in this example.
149
+
FORCE comes with a program to generate processing masks from a raster image with continuous values:
137
150
138
-
1. If the data are not already in the datacube format, use ``force-cube`` to import the data (see the usage above).
139
-
Use a raster resampling option to trigger the raster import, e.g. ``cubic`` (bc it's all about cubes, eh?).
151
+
.. code-block:: bash
140
152
141
-
2. Go to the parent directory of the cubed images (this is important for the next point), and generate a list with the filenames:
-s = pixel resolution of cubed data, defaults to 10
159
+
-l = input-layer: band number in case of multi-band input rasters,
160
+
defaults to 1
161
+
-d = input directory: the datacube directory
162
+
defaults to current directory
163
+
'datacube-definition.prj' needs to exist in there
164
+
-o = output directory: the directory where you want to store the cubes
165
+
defaults to current directory
166
+
-b = basename of output file (without extension)
167
+
defaults to the basename of the input-file,
168
+
appended by '_procmask'
169
+
-j = number of jobs, defaults to 'as many as possible'
147
170
171
+
Positional arguments:
172
+
- input-basename: basename of input data
173
+
- calc-expr: Calculation in gdalnumeric syntax, e.g. 'A>2500'"
174
+
The input variable is 'A'
175
+
For details about GDAL expressions, see
176
+
https://gdal.org/programs/gdal_calc.html
148
177
149
-
In this example, the image covers 597 tiles:
178
+
-----
179
+
see https://force-eo.readthedocs.io/en/latest/components/auxilliary/procmask.html
150
180
151
-
.. code-block:: bash
152
181
153
-
wc -l files.txt
182
+
In the example given below, our input image is a multiband continuous fields dataset,
183
+
which gives the percentages of built-up land (urban), high vegetation (trees), and low vegetation (grass, agriculture).
154
184
155
-
597 files.txt
185
+
.. note::
186
+
If the data are not already in the datacube format, use ``force-cube`` to import the data (see the usage above).
187
+
Use a raster resampling option to trigger the raster import, e.g. ``cubic`` (bc it's all about cubes, eh?).
156
188
189
+
In our case, the data are already in datacube format, covering 597 tiles:
157
190
158
191
.. code-block:: bash
159
192
160
-
head files.txt
193
+
cd /data/europe/pred
194
+
ls X*/*.tif | head
161
195
162
196
X0052_Y0045/CONFIELD_MLP.tif
163
197
X0052_Y0046/CONFIELD_MLP.tif
@@ -171,44 +205,38 @@ In this example, the image covers 597 tiles:
171
205
X0053_Y0045/CONFIELD_MLP.tif
172
206
173
207
174
-
3. Generate the masks using a command similar to the example below.
175
-
The 1st part of the command uses the list from point 2), and parallely calls the command in parentheses ``"..."``.
176
-
The curly braces ``{//}`` replace the input image with its dirname, i.e. with the tile ID.
177
-
A directory for the tile is generated if it is not already existing.
178
-
The ``gdal_calc.py`` command handles simple raster algebra.
179
-
The ``-A`` and ``--A_band`` options specify the image and band on which to operate the calculation specified by ``--calc`` (in our input image, the tree percentage is in band 2).
180
-
A binary image (= mask) will be generated, wherein all pixels larger than 3000 (i.e. 30%) are set to *1*.
181
-
The ``--creation-option`` parameters are options that specify compression etc.
182
-
The blocksize parameters should best reflect the blocksize used for the datacube (see datacube tutorial).
183
-
*As said before, a tool for this will likely be implemented in a not-so-far future version of FORCE.*
208
+
We generate the masks using ``force-procmask``, which internally uses ``gdal_calc.py`` for executing the raster algebra.
209
+
Thus, the arithmetic expression must be given in gdalnumeric syntax, e.g. 'A>3000'.
210
+
``A`` refers to our input image.
211
+
If this is a multiband file, the desired band can be specified with the ``-b`` option
212
+
(if not given, the first band is used).
213
+
In our example input image, the tree percentage is in band 2 and the percentage values are scaled by 100 (i.e. 100% = 10000).
214
+
To generate a mask with tree cover > 30%, we use the following:
0 commit comments