Architecture: feature-layers as DuckDB pipelines parallel to the basemap

[CommanderStorm]

Context

NavigaTUM's map has two distinct concerns:

• the basemap (planetiler-built world context — roads, landuse, water, buildings) and
• feature-layers (TUM-specific togglable overlays — WCs in feat(map): /map page with toggleable WC layer (framework + first layer) #3156, future events, vending, ...). This issue fixes the pattern every feature-layer follows.

The pattern

Each feature-layer is a self-contained pipeline running in parallel to, not inside, the basemap:

flowchart LR
    A["OSM PBF"]
    B["osmium extract"]
    C["DuckDB ST_ReadOSM"]
    D["SQL transform"]
    E["DuckDB tables"]
    F["martin (DuckDB)"]
    G["MVT"]

    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G

    P["TUM coverage polygon<br/>derived from<br/>data/sources/coordinates.csv"]
    S["Shared coverage polygon<br/>across all feature layers"]

    P -.-> B
    S -.-> D

Loading

• One coverage polygon (clustered from coordinates.csv) feeds every feature-layer's osmium extract.
• Each feature-layer owns its own DuckDB transform, its own DuckDB tables, and its own martin function. No shared schema between layers.
• The frontend layer panel toggles martin://<feature-layer> sources independently. Default off.
• Adding a feature-layer = new transform module + new martin function + new entry in the layer panel.

The planetiler basemap keeps its own pipeline and serves as the always-on substrate. Feature-layers ride on top.

Why not just bake feature-layers into the planetiler basemap?

Bundling looks like fewer pipelines but trades off badly:

1. Refresh coupling. A feature-layer schema change would force a basemap rebuild. planetiler over Germany is minutes-to-hours. Decoupled, feature-layers rebuild in seconds.
2. Cadence mismatch. The basemap regenerates when the source PBF refreshes. Feature-layers should track OSM edits closer to real-time so a corrected toilet appears quickly, without waiting for the next basemap cycle.
3. Wrong tool for togglable state. planetiler emits static tiles with baked-in attributes. Layer-visibility logic then ends up in the basemap style, polluting "what the world looks like" with "what the user toggled on".
4. Concern boundary. shortbread is opinionated about basemap context. TUM-specific concepts (ref:tum, students_have_access, indoor level) do not belong in a config that exists to describe roads and rivers — the same reason we do not fork shortbread.
5. Blast radius. A failing feature-layer transform should not be able to take down the basemap. Coupling them means any single TUM-specific edit risks the whole map.
6. Iteration loop. Feature-layer iteration is already the codebase's slowest dev loop. Coupling it to a 10 GB Germany planetiler recompile makes it strictly worse.

Implications and sequencing

• feat(map): /map page with toggleable WC layer (framework + first layer) #3156 ships frontend-only on the current osm2pgsql+PG indoor stack. The data-shape ACs of feat(map): /map page with toggleable WC layer (framework + first layer) #3156 (add category column, rename indoor_pois → pois_tiles, unified node/area in style.lua) are deferred — they fold into the migration below rather than landing as throw-away Lua/PG work.
• Once martin gains a DuckDB backend in a tagged release (~months out), a separate migration retires osm2pgsql, style.lua, and the four PG indoor_* functions in one stroke. The WC layer's basemap-style filter flips from is_*_toilet OR … to category = 'toilet' at that point.
• All future feature-layers ride the DuckDB pattern from day one — no new PG indoor tables, no new Lua.

Out of scope

• Specific feature-layer implementations (WCs, events, vending, …) — each is its own ticket.
• Migrating the basemap off planetiler — planetiler is the right tool for the basemap and stays.
• Search <-> map integration, low-zoom aggregation, attribute filters — separate follow-ups already called out in feat(map): /map page with toggleable WC layer (framework + first layer) #3156.

[CommanderStorm]

Nein