New feature: Check location of OSM objects against list of regions

This commit introduces a new feature: Locators. A locator is initialized
with one or more regions, each region has a name and a polygon or
bounding box. A geometry of an OSM object can then be checked against
this region list to figure out in which region(s) it is located.
This check is much faster than it would be to do this inside the
database after import.

Locators can be used for all sorts of interesting features:
* Read larger OSM file but import only data inside some area.
* Annotate each OSM object with the country (or other region) it
  is in. This can then, for instance, be used to show special
  highway shields for each country.
* Use the information which region the data is in for further
  processing, for instance setting of default values for the
  speed limit or using special language transliterations rules
  based on country.

Locators are created in Lua with `define_locator()`. Bounding boxes can
be added with `add_bbox()`. Polygons can be added from the database by
calling `add_from_db()` and specifiying an SQL query which can return
any number of rows each defining a region with the name and the
(multi)polygon as columns.

A locator can then be queried using `all_intersecting()` returning a
list of names of all regions that intersect the specified OSM object
geometry. Or the `first_intersecting()` function can be used which only
returns a single region for those cases where there can be no
overlapping data or where the details of objects straddling region
boundaries don't matter.

Several example config files are provided in the flex-config/locator
directory.

Author: joto

.github/workflows/luacheck.yml

@@ -15,5 +15,5 @@ jobs:
           sudo apt-get install -yq --no-install-suggests --no-install-recommends lua-check
 
       - name: Run luacheck
-        run: luacheck flex-config/*.lua flex-config/gen/*.lua tests/data/*.lua tests/lua/tests.lua
+        run: luacheck flex-config/*.lua flex-config/*/*.lua tests/data/*.lua tests/lua/tests.lua
 

flex-config/locator/README.md

@@ -0,0 +1,9 @@
+# Flex Output Configuration for Locator
+
+These are config file examples for use with the *locator* functionality.
+
+First use the `import-countries.lua` to import country boundaries, then
+use these with the other config files.
+
+You should be able to run this on the planet file or on any extracts you want.
+

flex-config/locator/buildings.lua

@@ -0,0 +1,74 @@
+-- This config example file is released into the Public Domain.
+
+-- This file shows how to use a locator to tag all buildings with the country
+-- they are in.
+
+-- Define the "countries" locator and get all country geometries from the
+-- database. Use the import-countries.lua file to import them first, before
+-- you run this.
+local countries = osm2pgsql.define_locator({ name = 'countries' })
+
+-- The SELECT query must return the regions the locator will use. The first
+-- column must contain the name of the region, the second the geometry in
+-- WGS84 lon/lat coordinates.
+-- To improve the efficiency of the country lookup, we'll subdivide the
+-- country polygons into smaller pieces. (If you run this often, for instance
+-- when doing updates, do the subdivide first in the database and store the
+-- result in its own table.)
+countries:add_from_db('SELECT code, ST_Subdivide(geom, 200) FROM countries')
+
+-- You have to decide whether you are interested in getting all regions
+-- intersecting with any of the objects or only one of them.
+--
+-- * Getting all regions makes sure that you get everything, even if regions
+--   overlap or objects straddle the border between regions. Use the function
+--   all_intersecting() for that.
+-- * Getting only one region is faster because osm2pgsql can stop looking
+--   for matches after the first one. Use first_intersecting() for that.
+--   This makes sense if you only have a single region anyway or if your
+--   regions don't overlap or you are not so concerned with what happens at
+--   the borders.
+--
+-- Just for demonstration, we do both in this example, in the "country" and
+-- "countries" columns, respectively.
+local buildings = osm2pgsql.define_area_table('buildings', {
+
+    -- This will contain the country code of the first matching country
+    -- (which can be any of the countries because there is no order here).
+    { column = 'country', type = 'text' },
+
+    -- This array will contain the country codes of all matching countries.
+    { column = 'countries', sql_type = 'text[]' },
+    { column = 'tags', type = 'jsonb' },
+    { column = 'geom', type = 'polygon', not_null = true },
+})
+
+local function add(geom, tags)
+    buildings:insert({
+        country = countries:first_intersecting(geom), -- or use geom:centroid()
+
+        -- We have to create the format that PostgreSQL expects for text
+        -- arrays. We assume that the region names do not contain any special
+        -- characters, otherwise we would need to do some escaping here.
+        countries = '{' .. table.concat(countries:all_intersecting(geom), ',') .. '}',
+
+        tags = tags,
+        geom = geom,
+    })
+end
+
+function osm2pgsql.process_way(object)
+    if object.tags.building then
+        add(object:as_polygon(), object.tags)
+    end
+end
+
+function osm2pgsql.process_relation(object)
+    if object.tags.building then
+        local geom = object:as_multipolygon()
+        for p in geom:geometries() do
+            add(p, object.tags)
+        end
+    end
+end
+

flex-config/locator/iceland.lua

@@ -0,0 +1,32 @@
+-- This config example file is released into the Public Domain.
+
+-- This file shows how to use a locator with a bounding box to import only
+-- the data for a region. In this case only highways in Iceland are imported
+-- even if you run this on the full planet file.
+
+local iceland = osm2pgsql.define_locator({ name = 'iceland' })
+
+iceland:add_bbox('IS', -25.0, 62.0, -12.0, 68.0)
+
+local highways = osm2pgsql.define_way_table('highways', {
+    { column = 'hwtype', type = 'text', not_null = true },
+    { column = 'name', type = 'text' },
+    { column = 'ref', type = 'text' },
+    { column = 'geom', type = 'linestring', not_null = true },
+})
+
+function osm2pgsql.process_way(object)
+    local t = object.tags
+    if t.highway then
+        local geom = object:as_linestring()
+        local region = iceland:first_intersecting(geom)
+        if region then
+            highways:insert({
+                hwtype = t.highway,
+                name = t.name,
+                ref = t.ref,
+                geom = geom,
+            })
+        end
+    end
+end

flex-config/locator/import-countries.lua

@@ -0,0 +1,35 @@
+-- This config example file is released into the Public Domain.
+
+-- This file is part of the examples for using the locator function of
+-- osm2pgsql. It is used to import all country boundaries into a database.
+
+local countries = osm2pgsql.define_relation_table('countries', {
+    -- For the ISO3166-1 Alpha-2 country code
+    -- https://en.wikipedia.org/wiki/ISO_3166-1
+    { column = 'code', type = 'text', not_null = true },
+    -- Because we want to use the geometries for the locator feature they
+    -- must be in 4326! We use a polygon type here and will later split
+    -- multipolygons into their parts.
+    { column = 'geom', type = 'polygon', not_null = true, projection = 4326 },
+})
+
+function osm2pgsql.process_relation(object)
+    local t = object.tags
+
+    if t.boundary == 'administrative' and t.admin_level == '2' then
+        local code = t['ISO3166-1']
+
+        -- Ignore entries with syntactically invalid ISO code
+        if not code or not string.match(code, '^%u%u$') then
+            return
+        end
+
+        for geom in object:as_multipolygon():geometries() do
+            countries:insert({
+                code = code,
+                geom = geom,
+            })
+        end
+    end
+end
+

flex-config/locator/motorway-colours.lua

@@ -0,0 +1,40 @@
+-- This config example file is released into the Public Domain.
+
+-- This file shows how to use a locator to find the country-specific colour
+
+-- Define the "countries" locator and get all country geometries from the
+-- database. Use the import-countries.lua file to import them first, before
+-- you run this.
+local countries = osm2pgsql.define_locator({ name = 'countries' })
+countries:add_from_db('SELECT code, ST_Subdivide(geom, 200) FROM countries')
+
+local highways = osm2pgsql.define_way_table('highways', {
+    { column = 'hwtype', type = 'text' },
+    { column = 'country', type = 'text' },
+    { column = 'colour', type = 'text' },
+    { column = 'geom', type = 'linestring', not_null = true },
+})
+
+-- Each country uses their own colour for motorways. Here is the beginning
+-- of a list of some countries in Europe. Source:
+-- https://en.wikipedia.org/wiki/Comparison_of_European_road_signs
+local cc2colour = {
+    BE = '#2d00e5',
+    CH = '#128044',
+    DE = '#174688',
+    FR = '#333b97',
+    NL = '#064269',
+}
+
+function osm2pgsql.process_way(object)
+    if object.tags.highway then
+        local geom = object:as_linestring()
+        local cc = countries:first_intersecting(geom)
+        highways:insert({
+            hwtype = object.tags.highway,
+            country = cc,
+            colour = cc2colour[cc],
+            geom = geom,
+        })
+    end
+end

src/CMakeLists.txt

@@ -21,6 +21,7 @@ target_sources(osm2pgsql_lib PRIVATE
     flex-lua-expire-output.cpp
     flex-lua-geom.cpp
     flex-lua-index.cpp
+    flex-lua-locator.cpp
     flex-lua-table.cpp
     flex-table-column.cpp
     flex-table.cpp
@@ -33,6 +34,7 @@ target_sources(osm2pgsql_lib PRIVATE
     geom.cpp
     idlist.cpp
     input.cpp
+    locator.cpp
     logging.cpp
     lua-setup.cpp
     lua-utils.cpp