Add multiple projections support

[nyurik]

Following from #611 proposal to add projection support, lets move the complex discussion here as it allows threads. These are some of the questions I personally had, but I'm sure there are a lot more options to consider.

IMPORTANT please use threads where appropriate, rather than creating a new top level answer (unless its a new topic)

Configuration vs Per-Request

Do we force site admins to explicitly allow which sources should support which projection, or do we let users do that?

• Allowing users to request "anything" creates much bigger scope for abuse / DDOS
• It might not make sense to allow all projections for a specific source
• It will make caching less effective, because now the same resource can be requested with two different URLs - e.g. /src/z/x/y and /src/3857/z/x/y

I suspect we should really let the site admin pre-configure additional sources due to above, but not certain yet.

URL pattern

• /src/proj/z/x/y or /src@proj/z/x/y -- The @ is like saying "this source in this projection". Symbol IDs currently cannot contain this symbol, so that's ok.
• no changes? If we only allow admins to define projections, and the the number of different projections per source is not too big, perhaps we should just keep the existing /src/z/x/y, and let the admins define the needed ones explicitly in the config? E.g. they could create mysource and mysource_4326 - keeping the system far simpler, and solving many unknowns except the configuration one.
• What does OpenLayers use here?

Config

This one is really tricky, and I don't know a good answer. I think we should adapt OGC work as much as possible if applicable, especially since OpenLayers supports a lot of it - and could use this API out of the box(?). Defining projections is tricky, so we should:

• allow "named" or "numbered" projections where it makes sense, e.g. epsg:4326 should "just work". We may also (eventually) allow to do local custom definitions - martin:foo_bar is a custom martin projection defined in the config?
• multiple sources may reuse the same projection spec, same tree-like inheritance is needed here

Tile storage format

• What are the specifications for MVT - would it even work?
• How does this work with archives like .mbtiles and .pmtiles?
• Raster images should be simpler
• Coordinate with the MLT efforts?

Re-projections

• Should Martin be able to reproject from one projection to another? Which should it support? Should it use proj? Something else?
• raster vs vector?

PG support

• How different would the PG SQL queries should be to support this? Would ST_AsMVT be able to handle it? (More specifically - ST_AsMVTGeom)

See also

Which other systems have already done this, esp those that worked well? Links needed.

• openlayers (?)
• geoserver (?)
• OGC standarts (?)
• bbox by @pka (?)
• postgis (?)

[nyurik]

🧵 How to define a projection?

[sharkAndshark]

EPSG CODE IS Enough

Writting down the EPSG CODE in our yaml is enough for the most users.

EPSG CODE IS NOT ENOUGH

We couldn't get the tile bbox with only epsg code, the extent tile size and origin are needed.

I mean maybe we shoud say we are Adding multiple tile grid support instead of Adding multiple projections support

tile_grid:
  custom_grid1: 
    epsg: 4326
    bounds:
        - minx
        - miny
        - maxx
        - maxy
    tilesize: 512

[nyurik]

🧵 Other implementations that worked well?

[justb4]

My $2ct..
MapProxy has ample support for projections and grids, even transformations among caches internally. A "tiling scheme" is more than a Projection. Besides a Projection (EPSG) it includes a Coverage, Origin, Resolutions, etc.
See https://mapproxy.github.io/mapproxy/latest/configuration.html#id6

Martin may not need all this complexity. In my current Martin setup with PostGIS functions it is already possible to support alternative Projections and Tiling Schemes via ST_TileEnvelope(). I experimented with this for the Dutch Projection EPSG:28992 and the Dutch Tiling scheme which has a custom coverage extent and thus tile numbering:

 SELECT ST_AsText(ST_TileEnvelope(9,277,180, 'SRID=28992;LINESTRING(-285401.920 22598.080, 595401.920 903401.920)'));
 POLYGON((191126.7200000001 592024,191126.7200000001 593744.3200000001,192847.04000000004 593744.3200000001,192847.04000000004 592024,191126.7200000001 592024))

gives the exact bbox for the (Dutch) WMTS tile z,x,y 9,277,180.

The only thing lacking now for Martin, is an 'Id' to determine the tiling scheme to use when Martin calls the configured PG function for the Layer. This could be an id passed in the URL and then forwarded with z, x, y to the PG function. This 'Id' would function like a TileMatrixSet identifier as used in WMTS (or like a `grid identifier in MapProxy).

So https://srv.com/<id>/<z>/<x>/<y>.png would call for example this slice:

CREATE OR REPLACE
    FUNCTION map5.vt_landcover(id text, z integer, x integer, y integer)
    RETURNS bytea AS $$
DECLARE
  mvt bytea;
BEGIN
-- figure out srs and extent for id, e.g. from a table
  SELECT INTO mvt ST_AsMVT(tile, 'landcover', 4096, 'geom') FROM (
     SELECT
        ST_AsMVTGeom(
            ST_Transform(geom, <the srs for id>),
            ST_TileEnvelope(z, x, y, 'the srid and extent from id'),
            4096, 64, true) AS geom,
.
.

Offcourse there are many more 'grid' variants not possible with this, and Martin can use non-PG Sources...

[blacha]

I work on a tile service for the New Zealand government (linz/basemaps), we primarily operate in the raster (TIFF/COG) space but also serve vector tiles.

Some projections have multiple in-use tile grids associated with them, I am aware of around five different grids for EPSG:2193 NZTM2000 and in some cases multiple different tile grids are being used on the same projects.

Having just a EPSG code could cause confusion on those projections.

We have been using the tile matrix identifier for our tile grids

/v1/tiles/:tileSet/:projection/:z/:x/:y.:ext

eg
/v1/tiles/topographic/WebMercatorQuad/2/3/2.pbf

morecantile has a collection of tile grids https://github.com/developmentseed/morecantile/tree/main/morecantile/data

titler is another common Tiff/COG title server, which follows a similar pattern for endpoints https://developmentseed.org/titiler/endpoints/cog/

[nyurik]

🧵 APIs?

[sharkAndshark]

The OGC Tiles API is something like /tiles/tile matrix set id/tile matrix/tile row/tile col, eg, /tiles/{tile grid}/z/x/y

[sharkAndshark]

API	easy to use	easy to config	added/changes to configure	ddos&&cache
/{source@tileGrid}/{z/{x}/{y}	⭐⭐	⭐⭐	many pre-configured tilegrid	⭐ The spam user could request many different tile grid (already pre-configured in yml)on same source to add the burden of cache
/{source}/{tilegrid}/{z/{x}/{y}	⭐⭐	⭐⭐	many pre-configured tilegrid	⭐ The spam user could request many different tile grid (already pre-configured in yml)on same source to add the burden of cache
/{source}/{tilegrid}/{z}/{x}/{y}?tilegrid=custom1...	⭐	⭐⭐	many pre-configured tilegrid	⭐ The spam user could request many different tile grid (already pre-configured in yml)on same source to add the burden of cache
/{source}/{z}/{x}/{y}	⭐⭐	⭐	many pre-configured tilegrid postgres.auto_publish.tilegrid. postgres.tables.table1.tilegrid	⭐⭐ the tilegrid of source is pre-configured,not easy to do ddos

I personally like the /source/tielgrid/z/x/y(OGC USE THIS EITHER) and /source/z/x/y.

[CommanderStorm]

With /source/tielgrid/z/x/y we would also not have this mess /source@tileGrid1,source@tileGrid2/z/x/y (tileGrid1!=tileGrid2) where I have no clue what should happen..

[nyurik]

I was thinking /source1,source2@tileGrid -- i.e. the tileGrid applies to all sources -- IF we go this route. As outlined above - the issue with this is that we now have to support "all" tilegrids, and it becomes a user-exposed feature (i.e. part of the API). If we do not allow tileGrid param at all, only site admins have to deal with it - smaller API exposure. We can always optimize this a bit in the configuration:

postgres:
  connection_string: 'postgresql://postgres@localhost:5432/db'
  # This is the current default
  default_tile_grid: 3857
  # This adds a SUFFIX to each published source, e.g. source_4326
  # Each value could be either a number or a string
  extra_tile_grids: [ 4326 ]

The above could be used with postgres, mbtiles, and others - if we are certain it will cover all our use cases.