DGGRS Indexing

[jerstlouis]

Topic for discussion

Continuing the discussion started in cf-convention/cf-conventions#433 and cf-convention/vocabularies#290...

Possibilty to make the proposed HEALPix indexing DGGRS-agnostic

Is it still possible at this stage to make the HEALPix indexing proposed for 1.13 agnostic of any particular DGGRS?

This would primarily mean three straightforward things:

• Renaming healpix_index to something more generic like discrete_global_grid_index or dggrs_global_index
• Adding a new contextual attribute e.g., dggrs_uri pointing to a registered DGGRS (e.g., with the proposed OGC DGGRS registry) which implies a specific Discrete Global Grid Hierarchy (the mathematical structure defining the multi-resolution partitioning of the globe -- the discrete global grids, with precisely defined zone boundaries mapped to a specific reference model, commonly the WGS84 ellipsoid), as well as the Zone Identifier Reference System (the name of the zones, unique across all refinement levels), e.g., https://www.opengis.net/def/dggrs/OGC/1.0/HEALPix_NUNIQ
• Allowing the zone identifier / grid mapping to be either integer or text, as the definition of a DGGRS does not require an integer-based ZIRS

This mechanism would allow using netCDF for DGGS data in a manner agnostic of any particular DGGRS, and is needed to support the netcdf3-dggs-zoneids and netcdf4-dggs-zoneids profiles defined in (approved/published) OGC API - DGGS netCDF requirements class.

If this proposal is too late to modify the current 1.13 proposal, please consider deprecating the healpix_index in favor of this DGGRS-agnostic approach in a future version.

DGGS sub-zone based grid mapping (for compact zone data packets)

As a proposal for a subsequent version, could an additional grid mapping mechanism be considered to support the need of the netcdf3-dggs and netcdf4-dggs profiles defined in OGC API - DGGS netCDF requirements class? These profiles are based on the concept of a root zone (a coarser zone for which data is being downloaded, for example based on a client selecting a set of large zones currently visible in the user's viewport), containing data values of all sub-zones (finer zones at least partially overlapping the coarser zone).

This approach needs three contextual elements:

• The DGGRS (we would reuse the dggrs_uri tag here), which already implies a particular Deterministic Sub-Zone Order (as defined in OGC API - DGGS),
• The global zone identifier of the root zone e.g., dggrs_root_index,
• The relative depth of the sub-zones e.g., dggs_relative_depth (the sub-zone's refinement level minus the root zone refinement level)

And then the "coordinates" of the actual dimension being mapped (e.g., grid mapping could be called dggrs_subzone_index) are highly compressible integers from 0 to nSubZones - 1, where nSubZones is the count of sub-zones for the root zone at that particular depth. For most DGGRSs, that count of sub-zones is a pre-determined value consistent for all root zones of a given type (e.g., hexagons or pentagons for hexagonal DGGRS) and relative depth (of the order of aperture^depth).

[allixender]

Hi @jerstlouis I'd like to raise your awareness towards a Zarr convention approach analogous to NetCDF-CF but with support for nested attributes (Zarr supports Dict-like attributes where NetCDF-CF only supports String-based key-value pairs: https://github.com/zarr-conventions/dggs

[jerstlouis]

@allixender It would be great to have a consistent convention across Zarr, netCDF and Coverage-JSON to support the two categories of profiles defined in OGC API-DGGS:

Using global zone IDs

This requires one contextual property:

• dggrs_uri (this is a URI like e.g., https://www.opengis.net/def/dggrs/OGC/1.0/HEALPix_NUNIQ)

And a dimension mapping allowed to be either text or integer:

• dggrs_zone_id (global zone ID as per the DGGRS)

What each property is named does not really matter, but it would be great if the names can be consistent across netCDF / Zarr / CoverageJSON.

netCDF profiles: netcdf3-dggs-zoneids and netcdf4-dggs-zoneids
Zarr profile: zarr2-dggs-zoneids
CoverageJSON profile: covjson-dggs-zoneids

This approach can package a whole dataset in a single file, but at the cost of one extra ID per individual set of data values. If the global zone id follows some kind of global space filling curve and the format supports a start..end range then the cost of the zone ID / zone can be avoided, assuming the global zone IDs do not have to be explicitly specified. Depending on the space filling curve, a single zone range may or may not correspond to "root zones" data packets as for OGC API - DGGS .../dggs/{dggrId}/zones/{zoneId}/data (e.g., if the space filling curve exits and re-enters a root zone later).

Using deterministic sub-zone order

netCDF profiles: netcdf3-dggs and netcdf4-dggs
Zarr profile: zarr2-dggs
CoverageJSON profile: covjson-dggs

This requires the following three contextual properties:

• dggrs_uri (same as for the global approach)
• dggrs_root_zone_id (this is always text)
• dggs_relative_depth (always an integer, sub-zone level - root zone level)

And a dimension mapping which is always a sequential integer from 0..nSubZones-1 (or 1 - nSubZones -- only the order that matters, the convention just needs to specify 0 or 1-based index):

• dggrs_subzone_index (index into the deterministic sub-zone order of the root zone as per the DGGRS)

This approach is much more compact, especially if the sub-zone order index can be omitted using a start..end range. The payload is then only the data values (except for the contextual elements for the whole root zone), but enforces one file/blob/shard with its contextual information per DGGRS root zone. A root zone + relative depth always corresponds to a.../dggs/{dggrId}/zones/{zoneId}/data data packet.

cc. @tik65536 @jsorel @fmigneault @tinaok @nicolasvila

[jsorel]

Just a note, All the links and docs posted talk about dggrs attributes in formats which is good.
But it looks like everyone forgot all the other reference systems.

DGGRS is only one kind of ReferencingByIdentifier (Apache-SIS name) (also called Identifier based reference system in Coverage-JSON which are defined in ISO 19112 (Gazetter).

We already use a lot of other Identifier based reference system, postal codes, mgrs, geohash, country codes, ....

The only difference DGGRS has is it's capability to store zone ids in a more compact way using a rootid+relativeDepth.
And even that, other Identifier based reference system could do it, like geohash.

So I believe the correct way to go would be to have :

• rs_uri : same as dggrs_uri but not restricted to dggrs

On the zone id dimension, two different cases :

• and exhaustive list of all ids
• a set a parameters describing how it could be computed
  • root zone id + depth
  • array of startIdx+count
  • ...

[fmigneault]

Is there an advantage of the rs_uri abstraction? Wouldn't this abstraction require the introduction of another attribute to distinguish DGGS-like or not reference systems? If there is no such distinction between these systems (or not sufficiently different to need another attribute), why couldn't they be classified/inventoried as a DGGS as well?

Maybe rs_uri is too generic and could lose interpretability when mixed with other "[coordinate|temporal|vertical|...] reference systems"? At the very least, I would consider "identifier reference system".

[jsorel]

Sure, "Identifier reference system" would work.

I'm saying this because coverage-json already has IRS, and DGGRS is one type of IRS.
And no you can't map all IRS as DGGRS. Many are not global, like country codes or postal codes, so it breaks one of the constraints of dggrs.