Skip to content

Commit 532d8c4

Browse files
JanssenBrmJanssenBrm
committed
icg: updated data hosting guidelines
1 parent 3afe8b2 commit 532d8c4

File tree

1 file changed

+216
-1
lines changed

1 file changed

+216
-1
lines changed

‎interoperability/datahosting.md‎

Lines changed: 216 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ fostering wider adoption and enabling advanced use cases in downstream applicati
4949
</tbody>
5050
</table>
5151

52-
Table: Interoperability requirements for data providers
52+
Interoperability requirements for data providers
5353
:::
5454

5555
:::{.callout-tip title="File Format Recommendations"}
@@ -95,3 +95,218 @@ EarthCODE and EOEPCA, to ensure consistency across projects and promote the adop
9595
- [STAC Best Practices](https://github.com/radiantearth/stac-best-practices/blob/main/README.md)
9696
- [EOEPCA+ Datacube Access Best Practices](https://github.com/EOEPCA/datacube-access/blob/main/best_practices/stac_best_practices.md)
9797
- [ESA PRR Collection Specifications](https://eoresults.esa.int/prr_collection_specifications.html)
98+
99+
:::{#tbl-metadata}
100+
<table class="requirements">
101+
<thead>
102+
<tr>
103+
<th>ID</th>
104+
<th>Scenario</th>
105+
<th>Level</th>
106+
<th>Requirement</th>
107+
<th>Description</th>
108+
</tr>
109+
</thead>
110+
<tbody>
111+
<tr>
112+
<td colspan="5"><b>General</b></td>
113+
</tr>
114+
<tr>
115+
<td>METADATA-REC-01</td>
116+
<td></td>
117+
<td>Collection / Item</td>
118+
<td>The STAC collection and items should use STAC 1.1 or higher.</td>
119+
<td></td>
120+
</tr>
121+
<tr>
122+
<td>METADATA-REC-02</td>
123+
<td></td>
124+
<td>Collection</td>
125+
<td>The STAC collection must follow the [ESA PRR collection specifications](https://eoresults.esa.int/prr_collection_specifications.html).</td>
126+
<td>This guarantees the collection is compatible for upload and registration in the ESA Project Results Repository.</td>
127+
</tr>
128+
<tr>
129+
<td>METADATA-REC-03</td>
130+
<td></td>
131+
<td>Collection / Item</td>
132+
<td>Collections must be homogeneous: each item has the same assets and uses the same asset keys.</td>
133+
<td>Consistent and homogeneous definition of assets simplifies client-side handling and supports datacube generation.</td>
134+
</tr>
135+
<tr>
136+
<td>METADATA-REC-04</td>
137+
<td></td>
138+
<td>Item</td>
139+
<td>Each item must have at least one asset where the role is set to *data*.</td>
140+
<td>This allows for accurate identification of assets containing the data.</td>
141+
</tr>
142+
<tr>
143+
<td>METADATA-REC-05</td>
144+
<td></td>
145+
<td>Item</td>
146+
<td>
147+
Each asset must include:
148+
<ul>
149+
<li>type</li>
150+
<li>role</li>
151+
<li>title</li>
152+
<li>file:size</li>
153+
</ul>
154+
Optionally properties:
155+
<ul>
156+
<li>file:checksum</li>
157+
</ul>
158+
</td>
159+
<td>These properties help tools and platforms accurately import the dataset. Furthermore, the *file* properties allow
160+
the ESA PRR to perform extra quality checks.</td>
161+
</tr>
162+
<tr>
163+
<td>METADATA-REC-06</td>
164+
<td>Data Visualisation</td>
165+
<td>Item</td>
166+
<td>
167+
<p>In the case of categorical datasets,
168+
<a href="https://github.com/stac-extensions/classification">classification extension</a> is recommended to identify
169+
the different classes used in the asset. </p>
170+
<p>For additional visualization support, it is recommend setting the *title* and *color_hint* properties to allow
171+
external tools, such as the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md) to properly visualise
172+
the data.</p>
173+
</td>
174+
<td>The classification extension supports the proper interpretation of categorical data that is included in the
175+
collection, item or asset.</td>
176+
</tr>
177+
<tr>
178+
<td>METADATA-REC-07</td>
179+
<td>Data Visualisation</td>
180+
<td>Item / Asset</td>
181+
<td>
182+
<p>To support the visualization of the dataset, the
183+
<a href="https://github.com/stac-extensions/render">render extension</a> is recommended.</p>
184+
<p>The render extension allows the definition of the following properties:</p>
185+
<ul>
186+
<li>rescale: 2 dimensions array of delimited Min,Max range per band. If not provided, the data will not be rescaled.</li>
187+
<li>Colormap: that must be applied for a raster band</li>
188+
</ul>
189+
</td>
190+
<td>The render extension supplies rendering tools, like the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md),
191+
with key data to auto-configure visualization settings, including rescaling and colour maps.</td>
192+
</tr>
193+
<tr>
194+
<td colspan="5"><b>Datacube Formats (netCDF, ZARR, ...)</b></td>
195+
</tr>
196+
<tr>
197+
<td>METADATA-REC-08</td>
198+
<td>Data Processing</td>
199+
<td>Collection / Item</td>
200+
<td>
201+
<p>The <a href="https://github.com/stac-extensions/datacube"></a>datacube extension (v2.x) should be used to
202+
properly describe the datacube:</p>
203+
<ul>
204+
<li>For a single variable: only use *cube:dimensions*</li>
205+
<li>For multiple variables: *cube:variables* and *cube:dimensions*. Each variable should be a separate datacube,
206+
no attempt should be made to combine variables automatically.</li>
207+
</ul>
208+
</td>
209+
<td>
210+
<p>The extension enables correct data parsing into a datacube by the platform or tool.</p>
211+
<p>A variable can be bands in EO data or meteorological variables like rain or temperature in meteorological data
212+
sets.</p>
213+
</td>
214+
</tr>
215+
<tr>
216+
<td colspan="5"><b>Raster Formats (COG)</b></td>
217+
</tr>
218+
<tr>
219+
<td>METADATA-REC-09</td>
220+
<td>Data Processing,Data Visualisation</td>
221+
<td>Item</td>
222+
<td>
223+
<p>The <a href="https://github.com/stac-extensions/projection">projection extension</a> must be used to identify
224+
the CRS, raster bounds and shape.</p>
225+
<p>At minimum the following must be defined:</p>
226+
<ul>
227+
<li>*proj:code*</li>
228+
<li>*proj:bbox*</li>
229+
<li>*proj:shape*</li>
230+
</ul>
231+
</td>
232+
<td>
233+
<p>The projection extension ensures that any tools accessing the data can accurately determine key raster
234+
properties without the performance overhead of inspecting the raster file.</p>
235+
<p>If the goal is to visualise your data through the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md),
236+
please consider the projections that are currently supported, as defined in [EXPLORER-REQ-07](./geospatial_explorer.qmd).</p>
237+
</td>
238+
</tr>
239+
<tr>
240+
<td>METADATA-REC-10</td>
241+
<td>Data Processing,Data Visualisation</td>
242+
<td>Item</td>
243+
<td>To incorporate a time dimension, the item must define a *datetime*, *start_datetime* and *end_datetime* at the
244+
item level. Both properties contain a single value using ISO8601 time intervals.</td>
245+
<td>
246+
<p>These properties enable tools to accurately identify the data's temporal dimension, simplifying search and
247+
filtering within the STAC collection.</p>
248+
<p>For temporal dimensions, it is recommended to maintain the original level of granularity; data should not be
249+
aggregated from daily records to a single label unless specifically instructed by the user or noted in the metadata.
250+
When combining data and overlap exists, the user must indicate the methodology unless indicated in the metadata.</p>
251+
</td>
252+
</tr>
253+
<tr>
254+
<td>METADATA-REC-11</td>
255+
<td>Data Processing,Data Visualisation</td>
256+
<td>Item</td>
257+
<td>
258+
<p>The <a href="https://github.com/radiantearth/stac-best-practices/blob/main/best-practices-asset-and-link.md#bands">*bands*</a>
259+
array must be used to identify band information in the raster, keep the order as identified in the array.</p>
260+
<ul>
261+
<li>Use the *name* property, if provided. Alternatively, use *eo:common_name*. As a last resort, use the array
262+
indices.</li>
263+
<li>Ensure homogeneous data types across bands, choosing the most precise one.</li>
264+
</ul>
265+
</td>
266+
<td></td>
267+
</tr>
268+
<tr>
269+
<td>METADATA-REC-12</td>
270+
<td></td>
271+
<td>Item</td>
272+
<td>For other dimensions, the datacube extension must be provided.</td>
273+
<td></td>
274+
</tr>
275+
<tr>
276+
<td>METADATA-REC-13</td>
277+
<td>Data Processing,Data Visualisation</td>
278+
<td>Item</td>
279+
<td>
280+
<p>The <a href="https://github.com/stac-extensions/raster">raster extension</a> must be used to accurately specify
281+
the following attributes associated with the raster file: </p>
282+
<ul>
283+
<li>*raster:spatial_resolution* (if multiple resolutions are provided in the item, otherwise specify it in the
284+
Item)</li>
285+
<li>*raster:scale*</li>
286+
<li>*raster:offset*</li>
287+
<li>*raster:nodata value*</li>
288+
</ul>
289+
</td>
290+
<td>The raster extension offers valuable information about the dataset, eliminating the need to directly access or
291+
analyse the data itself. For instance, when visualising, details like scale and offset can help convert raw values
292+
into real-world figures.</td>
293+
</tr>
294+
<tr>
295+
<td colspan="5"><b>Statistical Data (FlatGeobuff, GeoJSON)</b></td>
296+
</tr>
297+
<tr>
298+
<td>METADATA-REC-14</td>
299+
<td>Data Visualisation</td>
300+
<td>Item</td>
301+
<td>It is recommended that STAC records for statistics include a *datetime* property that matches the time stamp of
302+
the source data that the statistics are derived from. The description should also make reference to the boundary
303+
datasets (e.g. NUTS) that the statistics represent.</td>
304+
<td>This information offers clear insights into the statistical data and its applications and also assists in integrating
305+
this data into the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md).
306+
</td>
307+
</tr>
308+
</tbody>
309+
</table>
310+
311+
Metadata Recommendations
312+
:::

0 commit comments

Comments
 (0)