Skip to content

geocomponents

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

geocomponents

README.md

geocomponents

geocomponents turns a plain description of your geographic data into a working database and web API. You describe your data once in YAML; it creates the PostGIS tables and a standards-based (OGC) web API to read and edit that data — no SQL or API code to write by hand. Change the description, re-run, and the database and API follow.

How it works

You write a description (YAML) → run geocomponents → for each dataset you get:

  • a PostGIS database — one schema of tables, and
  • a web API at /datasets/<name>/ogc_api to read and edit the data (an OGC API — Features service).

The description is the single source of truth: the database and the API are both generated from it.

Describing a dataset

A description is a folder of YAML files. Each file is one dataset; an optional commons.yaml holds definitions shared by all of them. Point geocomponents at the folder with the GEOCOMPONENTS_DESCRIPTIONS setting.

A dataset

name: cadastre           # required — becomes the DB schema and the API path
title: Cadastre          # optional — a human-friendly name
description: Land registry data.   # optional
processes:               # optional — named operations to expose (see below)
  - hello
collections:             # the feature types in this dataset
  - ...

name becomes the PostgreSQL schema and the API mount path, e.g. /datasets/cadastre/ogc_api.

Naming: dataset and collection names become raw SQL identifiers, so use lowercase letters, digits and underscores only — no hyphens (fkb-bane → fkb_bane) and no uppercase (arealressursFlate → arealressurs_flate). This restriction will be lifted later.

A collection

A collection is one feature type (for example, parcels). Each becomes a table and an API collection:

collections:
  - name: parcels               # required — the table + collection name
    title: Parcels
    description: Land parcels.
    feature_model: simple       # 'simple' (default) = read + edit
                                # 'topology'         = read-only
    geometry:
      type: MultiPolygon        # shape type (see the list below)
      srid: 4326                # coordinate system (default 4326 = WGS84)
    fields:                     # your attributes (see below)
      - name: label
        type: string
        required: true
  • feature_model — simple collections can be read and edited (create/update/delete). topology collections share geometry between neighbouring features, so they are read-only for now; edit requests return 405 Method Not Allowed.
  • geometry — the shape type and coordinate system. The type is enforced exactly: a MultiPolygon column rejects a plain Polygon. Set has_z: true when coordinates include height; PostGIS then uses the *Z typmod (e.g. LineStringZ).

Fields (attributes)

Each field is one attribute (one column). Give it a name and exactly one way to type it — a builtin type, a type_ref (a reusable type from commons.yaml), or a codelist (a controlled vocabulary from commons.yaml):

fields:
  - name: label
    type: string          # a builtin type (table below)
    required: true        # optional, default false

  - name: municipality
    type_ref: municipality_code   # a named type defined in commons.yaml

  - name: status
    codelist: parcel_status       # a code list defined in commons.yaml

Builtin type values:

type stored as
string text
integer integer
number double precision
boolean boolean
date date
timestamp timestamp (with time zone)
uuid uuid
object jsonb (nested object — see below)

Set indexable: true on any scalar field to add a database index. On an object field, indexable: true on a sub-field adds a functional index on that key inside the JSONB column.

Nested objects

A field with type: object stores a JSON object (JSONB column). Its shape is declared with a nested fields list; sub-fields follow the same typing rules, including codelist references:

- name: kvalitet
  type: object
  required: true
  fields:
    - name: datafangstmetode
      codelist: datafangstmetode
      required: true
    - name: noyaktighet
      type: integer
    - name: synbarhet
      codelist: synbarhet
      indexable: true   # adds a functional index on (kvalitet->>'synbarhet')

Relationships

A relationship links a collection to another collection in the same dataset. It adds a <name>_id column that points at the target:

collections:
  - name: buildings
    geometry: { type: MultiPolygon, srid: 4326 }
    relationships:
      - name: parcel        # adds a 'parcel_id' column
        target: parcels     # referencing the 'parcels' collection

Fixed columns

Every collection has these without declaring them in fields::

  • id — UUID primary key, generated by the server
  • geometry — the shape column; its type, SRID and Z flag come from the collection's geometry: key
  • created_at, updated_at — write timestamps, maintained automatically

In API responses (GeoJSON), id and geometry are top-level; your attributes plus created_at/updated_at appear under properties.

Server-managed fields

Two collection keys hand ownership of field values to the server. Client-supplied values for these fields are stripped on write and replaced by the server.

outward_identifier allows a JSONB sub-field to hold the feature's id (UUID). On read the server injects id into that sub-field; on write any client value is discarded:

outward_identifier: identifikasjon.lokalid

server_managed — a map of paths to tokens:

token behaviour
timestamp_iso set to now() on every write; client value discarded
outward_identifier same as the top-level outward_identifier: key 
server_managed:
  identifikasjon.versjonid: timestamp_iso   # sub-field inside the identifikasjon JSONB column
  oppdateringsdato: timestamp_iso           # top-level scalar column

A path with two segments (identifikasjon.versjonid) targets a key inside a JSONB column. A single-segment path (oppdateringsdato) targets a top-level scalar column.

Shared definitions — commons.yaml

An optional commons.yaml in the folder holds definitions every collection inherits:

base_fields:              # extra attributes added to EVERY collection
  - name: source
    type: string
    description: Where the feature came from.

field_types:              # reusable named types, used via `type_ref`
  - name: municipality_code
    sql_type: varchar(4)

code_lists:               # controlled vocabularies, used via `codelist`
  - name: parcel_status
    values:
      - { code: active, label: Active }
      - { code: retired, label: Retired }

Codelists can also be declared directly in a dataset file under a top-level codelists: key — dataset-local codelists take precedence over commons ones with the same name. Fields that use a codelist are validated at the database level: an invalid code value returns HTTP 422.

Geometry types

Point, MultiPoint, LineString, MultiLineString, Polygon, MultiPolygon, GeometryCollection. srid defaults to 4326 (WGS84 longitude/latitude). If you omit geometry, it defaults to a Point. Optional has_z: true stores XYZ coordinates (PostGIS PointZ, LineStringZ, …).

Processes

processes lists named operations (OGC API — Processes) the dataset exposes at /processes/<id>. Each id must be registered in the process registry (src/geocomponents/processes/registry.py); the example ships a hello process.

A minimal dataset

The smallest useful dataset — a name and one collection with a geometry and a field:

name: places
collections:
  - name: points_of_interest
    geometry:
      type: Point
    fields:
      - name: name
        type: string
        required: true

A working example

The descriptions/ folder is a complete, runnable example: cadastre.yaml (a parcels collection using a code list and a shared type, a buildings collection with a relationship, and a read-only blocks topology collection), hydro.yaml, bane.yaml, bygning.yaml, and a shared commons.yaml.

Running it (local)

The whole stack runs in Docker Compose. The app reads the same discrete DB_* variables used in production (no special local config path), and your src/ is mounted into the container so code edits take effect on a restart — no rebuild:

docker compose up --build
# db  →  migrate (geocomponents apply-schema)  →  api (geocomponents serve) on :8000

docker compose up db brings up just PostGIS (used by the host-run tests below). The image's entrypoint is the geocomponents CLI (validate | apply-schema | serve); the compose services simply run those subcommands.

Using the API

# List datasets, then one dataset's collections and features
curl localhost:8000/datasets
curl "localhost:8000/datasets/cadastre/ogc_api/collections/parcels/items?f=json"

# Create a feature (simple collections only)
curl -X POST localhost:8000/datasets/cadastre/ogc_api/collections/parcels/items \
  -H 'content-type: application/geo+json' \
  -d '{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[10,55],[10,56],[11,56],[11,55],[10,55]]]]},"properties":{"label":"P1","source":"demo"}}'

# Run a process
curl -X POST localhost:8000/datasets/cadastre/ogc_api/processes/hello/execution \
  -H 'content-type: application/json' -d '{"inputs":{"name":"world"}}'

Open http://localhost:8000/datasets/cadastre/ogc_api/ in a browser for the built-in HTML view.

Testing

docker compose up -d db      # PostGIS on localhost:55432
uv run pytest                # unit tests run without Docker; DB-backed
                             # contract/integration tests use the database above

The suite is written as an executable contract: it drives the components at their real surfaces (the ogc.feature_* database functions and the HTTP OGC API), so a reimplementation in another language would pass the same tests.

Deployment

geocomponents ships as a container image (engine only — descriptions are supplied at runtime, not baked in) and is deployed to Kubernetes + CloudSQL by a separate apps repo. See DEPLOY.md for the operational contract: the DB_* connection variables and GEOCOMPONENTS_* settings, the apply-schema then serve lifecycle, and the /healthz (liveness) + /datasets (readiness) probes.

How it's built

Each dataset becomes one PostgreSQL schema; each collection becomes one table. The database shapes the data and handles create/read/update/delete; the API adds the OGC links and paging on top. They meet at the ogc.feature_* functions — see the DB ↔ API contract subsection below.

The four parts are independently swappable: descriptions/ (the format + loader), schema/ (description → PostGIS tables + functions), api/ (one dataset → one OGC API app), and gateway/ (many apps → one service).

The DB ↔ API contract

The database and the API share a standard-shaped surface for how they communicate. The database delivers a fixed ogc.* surface which callers use with OGC identifiers (dataset, collection) rather than physical table or function names. Single-feature reads and writes go through ogc.feature_*; atomic multi-feature writes go through ogc.transaction. The functions follow the naming and formatting expected by the OGC standards. The API uses these functions to read and write features, which makes either database or API substitutable as long as they adhere to the same contract.

The public ogc. contract:*

Surface Function Arguments Returns
GET /collections/{c}/items ogc.feature_items dataset, collection, bbox float8[], lim int, off int, with_matched bool jsonb — a GeoJSON FeatureCollection
GET /collections/{c}/items/{id} ogc.feature_item dataset, collection, fid uuid jsonb — a Feature, or null if the id is absent
POST /collections/{c}/items ogc.feature_create dataset, collection, feature jsonb uuid of the new feature
PUT /collections/{c}/items/{id} ogc.feature_replace dataset, collection, fid uuid, feature jsonb boolean — true when a matching feature was replaced
PATCH /collections/{c}/items/{id} ogc.feature_update dataset, collection, fid uuid, feature jsonb boolean — true when updated; only fields present in the input change
DELETE /collections/{c}/items/{id} ogc.feature_delete dataset, collection, fid uuid boolean — true when a matching feature was deleted
POST /collections/{c}/items:upsert ogc.feature_upsert dataset, collection, feature jsonb stable uuid; available when the collection resolves an upsert key
Atomic multi-feature write ogc.transaction dataset, document jsonb jsonb — a fixed-shape transaction report

Endpoints are relative to a dataset mount, e.g. /datasets/cadastre/ogc_api/collections/parcels/items. The ogc.transaction row is a database-contract entrypoint today; it is not yet routed from an HTTP transaction endpoint in this repo.

The dataset and collection arguments come from the description (cadastre, parcels) — the same names OGC puts in the URL. The dispatcher routes ogc.feature_items('cadastre', 'parcels', …) to a per-collection function cadastre._parcels_items(…) generated from the description. ogc.transaction('cadastre', …) routes item actions the same way, to the generated cadastre._<collection>_<op> functions. Change the storage layout, update the dispatcher; callers keep using the same ogc.* surface. Collections with a resolved upsert key also receive a unique index and an atomic insert-or-replace function keyed by that field. The key comes from outward_identifier, or defaults to lokalid when present.

Direct ogc.feature_* writes are for simple-feature collections. They consult per-dataset capability metadata and refuse feature_model: topology collections; ogc.transaction is the write path for those collections. Client-supplied feature ids are honored on transaction inserts, while ogc.feature_create strips them and generates ids server-side.

Transaction document shape

ogc.transaction currently accepts atomic documents of this form:

{
  "semantic": "atomic",
  "transaction": [
    {
      "action": "insert",
      "collection": "parcels",
      "feature": {
        "type": "Feature",
        "id": "…",
        "geometry": {"type": "MultiPolygon", "coordinates": [[[[10, 55], [10, 56], [11, 56], [11, 55], [10, 55]]]]},
        "properties": {"label": "P-1"}
      }
    },
    {
      "action": "update",
      "collection": "parcels",
      "id": "…",
      "feature": {"properties": {"label": "updated"}}
    }
  ]
}

The verb set is closed in this PR: insert, update, replace, delete. upsert is not part of the transaction document yet.

Transaction report shape

ogc.transaction returns a fixed-shape report:

{
  "committed": false,
  "phase": "items",
  "reason": null,
  "items": [
    {
      "index": 1,
      "action": "update",
      "collection": "parcels",
      "id": null,
      "status": "rejected",
      "sqlstate": "P0001",
      "reason": "invalid geometry"
    }
  ],
  "structure": [],
  "geometry": []
}

phase is items in this PR. reason is a document-level failure message for cases such as bad semantic or a non-array transaction; it is null for item-level failures and successful transactions. Under atomic semantics, a failed transaction reports only the rejected item, because no earlier change is visible after the rollback.

structure and geometry are always-present empty arrays today. Later PRs will fill them with verdicts, so callers must not branch on their presence. Every item key is always present too, including id when it is null.

You can call them directly:

select ogc.feature_items('cadastre', 'parcels', null, 10, 0, true);
select ogc.feature_item('cadastre', 'parcels', '…uuid…');
select ogc.transaction(
    'cadastre',
    '{
      "semantic": "atomic",
      "transaction": [
        {
          "action": "insert",
          "collection": "parcels",
          "feature": {
            "type": "Feature",
            "id": "…uuid…",
            "geometry": {"type": "MultiPolygon", "coordinates": [[[[10,55],[10,56],[11,56],[11,55],[10,55]]]]},
            "properties": {"label": "P-1"}
          }
        }
      ]
    }'::jsonb
);

Not built yet (designed for)

  • Importing descriptions from GML/UML models.
  • Emitting events when data changes.
  • Migrations when a description changes an existing table. Until then, operators must migrate manually — see the note in DEPLOY.md.