Plugins: Add support for other tiled formats

[gkjohnson]

2D Formats

• Protomaps
  • https://protomaps.com/
  • 2d vectorized data with text / metadata information
• Overture Maps Data
  • https://overturemaps.org/
• XYZ MVT Data
  • OpenStreetMap also provides vector tiles in the MVT format.
  • Another vector format? here
• Cloud Optimized GeoTiff
  • Tiled TIFF file with different levels of detail embedded
  • https://developers.planet.com/docs/planetschool/an-introduction-to-cloud-optimized-geotiffs-cogs-part-1-overview/
• 3D OpenStreetMap Data
  • https://wiki.openstreetmap.org/wiki/3D
• Mapbox Tilejson Format
  • https://github.com/mapbox/tilejson-spec
• LERC
  • Definition: https://github.com/Esri/lerc (including JS parser)
  • Files: https://services.arcgisonline.com/arcgis/rest/services/WorldElevation3D
    • structured as WMTS
• WMS
  • Add Support WMS Tiled Image Format #1299 1299
• Other
  • Ellipsoid projection
  • Repeat tiles on X and Y
• WMTS Capabilities
  • Support the WMTS Capabilities format to infer tile mtrices, etc without explicit user specification or url templating. See sample here and GIBS capabilities here.
  • Add a parser to enumerate all the layer information and options / dimensions.
  • Allow for passing the list directly into the WMTSLayer for construction or passing the capabilities url in directly and specifying the keys to use.
  • https://services.arcgisonline.com/arcgis/rest/services
• WMTS tiles for rendering 2d map data
  • Can support planar or ellipsoid projection
  • https://api.oneatlas.airbus.com/guides/oneatlas-basemap/g-oab-wmts-basic/
  • https://opengeospatial.github.io/e-learning/wmts/text/main.html
  • Seems to be TMS but flipped on Y
  • Cesium (and many other libraries) seem to avoid the capabilities file and just require passing in the url parameters (see cesium's implementation here).
  • It seems Cesium also recommends the "OpenLayers" parser here or here. Or a leaflet integration capability here.
  • WMTS works currently with XYZ format template but would be good to support this more officially / directly. Eg class or string interpolation arguments (matrix set, time).
• TMS data
  • Supported in Cesium Ion "imagery"
  • https://cesium.com/learn/3d-tiling/tiler-data-formats/
• XYZ data
  • Max zoom level is 19
  • https://wiki.openstreetmap.org/wiki/Tiles
  • https://gis.stackexchange.com/questions/447421/convert-a-point-on-a-flat-2d-web-mercator-map-image-to-a-coordinate
• Deep Zoom Images for rendering high resolution images
  • Plugins: Add support for Deep Zoom Image format #945
  • If needed could add support for Collections, DisplayRects

DEM Formats

• TerrainDEM
  • https://docs.mapbox.com/data/tilesets/reference/mapbox-terrain-dem-v1/
• AWS Open Terrain Tiles
  • https://registry.opendata.aws/terrain-tiles/
• QuantizedMesh tiles for rendering ion terrain
  • https://github.com/CesiumGS/quantized-mesh
  • Provide an option to construct a solid volume CSG support.

3D Formats

• PoTree
  • Point cloud hierarchy structure
  • https://github.com/cpj001/potree-develop/blob/master/docs/potree-file-format.md
• COPC
  • COG-inspired point cloud format
  • https://copc.io/
• I3S
  • Generalized 3d hierarchy system similar to 3d tiles.
  • https://www.ogc.org/publications/standard/i3s/

Coordinate Reference Systems

• EPSG:3031, used for Antarctic data sets. Some ArcGIS data uses this.
• EPSG:21781, used for Swiss data sets.

Data Sources

• ESA Data: https://esa-worldcover.org/en/data-access
• Terrascope: https://docs.terrascope.be/Developers/WebServices/OGC/WMTSv2.html
• Mapbox
  • Jonni Walker's data sets (see here)
• OpenStreetMap tiles (here)
• ArcGIS data sets (here)
• NASA GIBs data sets (here)
• Cesium Ion
• https://geotiles.citg.tudelft.nl/
• Protomaps (here)
• OpenDroneMap.org
• Esri data format - may require custom vector data format
  • https://www.arcgis.com/home/item.html?id=33064a20de0c48d2bb61efa8faca93a8
  • https://www.arcgis.com/home/group.html?id=f2d67dbc6a764031969b2e3a891fea90#overview
• Swiss EPSG data sets
  • 3D tiles of vectorized location names
  • WMTS in EPSG:21781 (Swiss projection), in particular SWISSIMAGE (swiss aerial imagery)

[jo-chemla]

Regarding OSM tiles, they indeed introduced a few months ago a production vector tiles endpoint which can be styled client-side: eg tilekiln-shortbread-demo from Paul Norman, and the official OSM forum thread, where style can be live-edited via TileJSON editors like Maplibre Maputnik, mappable or maptiler: https://vector.openstreetmap.org/shortbread_v1/{z}/{x}/{y}.mvt.

Regarding other types of Tiled 3D data, I just wanted to quote that loaders-gl has a section dedicated to tiled-3d-data loaders here, with support already live for OGC 3D Tiles and I3S. In their v5.0 roadmap, they aim for supporting other tiled formats like

• COPC (Cloud Optimized PointCloud)
• or Potree (v1, v2).

Also for what it is worth, I dreamt once about a TiTiler-like middleware but for 3d, but it did not gather attention (TiTiler is given an endpoint to a COG Cloud Optimized Geotiffs, and returns a live TMS endpoint with on-the-fly tile selection, mosaicking, merging and reprojection), to dissociate data format from consuming end-user application - zero-traction post and associated quick discussion in loader-gl repo.

[gkjohnson]

Thanks for the info! Getting Potree working would be great considering some of the discussions in #912, as well. I'm less familiar with COPC - is this a widely used format? It looks fairly new, still. I'll have to take a look at the I3S format a bit more, too. The spec looked a bit difficult to understand and I'm not sure if it's really designed for web use.

TileJSON editors like Maplibre Maputnik, mappable or maptiler: https://vector.openstreetmap.org/shortbread_v1/{z}/{x}/{y}.mvt

Do you have a link to the spec that OSM uses for tiling? Is it something similar (or the same as) WMTS?

TiTiler is given an endpoint to a COG Cloud Optimized Geotiffs

Supporting Cloud Optimized Geotiffs directly should be possible, as well, with no middleware (looks like there are embedded LoDs). Sounds like something like TiTiler is good for enabling support in applications that only support WMTS but some of the these tiling formats are close enough to be on-the-fly converted client side.

[jo-chemla]

From my point of view, Potree succeeded because it was one of the first, easiest way to tile massive pointclouds for web viewing, with a viewer and library with the right amount of tooling for measurement taking, clipping, POIs, and easiness to deploy for non tech-savvy users. So a lot of public and private organizations are using it, which result in a pretty large collection of datasets in that spec. The older v1 format stored the hierarchy tiles as individual files which were named by their index, which was pretty heavy on the filesystem during transfers, compression etc. Markus introduced the v2 format a few years back to alleviate these, resulting in a single file tiled pointcloud.

COG is simply a geotiff but with pyramid overviews (downsampled version of the file) and block memory-contiguity (for faster retrieval of a subset of the data), standardized. Titiler indeed takes advantage of that internal tiles structure to decide which portions of the file to read, retrieve them, reproject and merge on the fly, with gdal wizardy - I even got TiTiler to ingest not only COGs, but also other TMS like in this discussion.

COPC has been created by the team at hobu (folks behind eg PDAL), which has been dealing with massive pointclouds from a long time, eg 3DEP, the lidar coverage of the US as open-data. My understanding is they first created Entwine as an open spec, similar to potree, with extensibility in mind, but then got the inspiration from COG to create COPC. They extended the LAS file format, and describe how tiles should be described in header. If an app does not know COPC, it can just parse the file as a standard las. COPC os getting used more and more by national mapping agencies to publish their Lidar coverage, especially in US/Europe. PDAL is usually used to convert a standard pointcloud to COPC.

From my understanding, I3S never got traction outside of the ESRI ecosystem, which inceptionned it, especially in comparison to 3D-Tiles, which was pushed more by Cesium - especially with all the ecosystem of plugins for 3D render engines, now on Unreal, Unity, O3DE, Omniverse in addition to the web libs.

The /{z}/{x}/{y} used by OSM is indeed the standard TMS or XYZ tiling scheme (I often find myself looking back at this ref), in web mercator EPSG:3857 - with sometimes a Y-inversion with Google tiling, y_google = 2**zoom - y_tms - 1.

Hope, this all makes sense!

[gkjohnson]

Great, thanks! This is all helpful information.

The /{z}/{x}/{y} used by OSM is indeed the standard TMS or XYZ tiling scheme (I often find myself looking back at this ref),

Can you explain the difference between TMS and WMTS, then? I've tried looking it up before but it wasn't super clear to me. A simple explainer would help.

[jo-chemla]

TMS and XYZ are endpoint naming schemes/conventions, where tile indices (zoom-level=z, column=x and row=y) are usually represented surrounded by brackets. Any standard file server can act as a TMS server as long as tile files are stored with the described hierarchy directories, eg upper level directory for zoom, which each contain one folder per column, which itself contain one file per row - eg here satellite https://1.aerial.maps.ls.hereapi.com/maptile/2.1/maptile/newest/satellite.day/{z}/{x}/{y}/512/jpg. Other url examples for TMS include google satellite https://www.google.cn/maps/vt?lyrs=s@189&gl=cn&x={x}&y={y}&z={z}, where url-params are used rather than hierarchical directory.

WMS/WMTS is a more complex server specification, where the server has to serve raster tiles via the GetTile endpoint, but also other endpoints, like GetCapabilities which describe what the server can produce and CRS it handle. URLs are usually in the form https://wxs.ign.fr/essentiels/geoportail/wmts?SERVICE=WMTS&REQUEST=GetTile &VERSION=1.0.0&LAYER=ORTHOIMAGERY.ORTHOPHOTOS&TILEMATRIXSET=PM &TILEMATRIX={z}&TILECOL={x}&TILEROW={x}&STYLE=normal&FORMAT=image/jpeg. WMTS is a WMS server with tiles cached.

You can see a collection of WMS vs TMS endpoints on NextGIS QMS, with satellite endpoints for google, here, esri, bing (based on quadkey rather than xyz) etc.

[jo-chemla]

Quick note regarding 2.5D DTM/DSM formats (quantized-mesh is one of them rather than real 3D like the others, COPC, potree, 3dtiles, I3S etc): there are two existing standards for encoding height in RGB 3-channel data image, with a pretty good height accuracy. They are usually served as tiles, via TMS, WMTS endpoints etc.

• MapTiler/Mapbox TerrainRGB: elevation = -10000 + ((R * 256 * 256 + G * 256 + B) * 0.1), mapbox serves their terrain data following this standard
• Terrarium, elevation = (R * 256 + G + B / 256) - 32768 eg Amazon Open Data Terrain Tiles https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{z}/{x}/{y}.png.

I recently discovered within Titiler (middleware initially meant to ingest COGs, Cloud Optimized Geotiffs, and serve on-the-fly a TMS endpoint) that algorithms can be applied to the data, and therefore it is possible to encode/decode standard-height/terrarium/terrainrgb - see this discussion and PR fixing terrain-rgb encoding. Note it is also capable of computing hillshade or slope also on the fly, and not only ingest DSM COGs but also TMS for height data.