Add load_queryables function to pypgstac

.gitignore

@@ -11,3 +11,5 @@ src/pypgstac/target
 src/pypgstac/python/pypgstac/*.so
 .vscode
 .ipynb_checkpoints
+.venv
+.pytest_cache

CHANGELOG.md

@@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## [Unreleased]
+
+### Added
+
+- Add `load_queryables` function to pypgstac for loading queryables from a JSON file
+- Add support for specifying collection IDs when loading queryables
+
 ## [v0.9.5]
 
 ### Changed

docs/src/pypgstac.md

@@ -85,6 +85,80 @@ To upsert any records, adding anything new and replacing anything with the same
 pypgstac load items --method upsert
 ```
 
+### Loading Queryables
+
+Queryables are a mechanism that allows clients to discover what terms are available for use when writing filter expressions in a STAC API. The Filter Extension enables clients to filter collections and items based on their properties using the Common Query Language (CQL2).
+
+To load queryables from a JSON file:
+
+```
+pypgstac load_queryables queryables.json
+```
+
+To load queryables for specific collections:
+
+```
+pypgstac load_queryables queryables.json --collection_ids [collection1,collection2]
+```
+
+To load queryables and delete properties not present in the file:
+
+```
+pypgstac load_queryables queryables.json --delete_missing
+```
+
+To load queryables and create indexes only for specific fields:
+
+```
+pypgstac load_queryables queryables.json --index_fields [field1,field2]
+```
+
+By default, no indexes are created when loading queryables. Using the `--index_fields` parameter allows you to selectively create indexes only for fields that require them. Creating too many indexes can degrade database performance, especially for write operations, so it's recommended to only index fields that are frequently used in queries.
+
+When using `--delete_missing` with specific collections, only properties for those collections will be deleted:
+
+```
+pypgstac load_queryables queryables.json --collection_ids [collection1,collection2] --delete_missing
+```
+
+You can combine all parameters as needed:
+
+```
+pypgstac load_queryables queryables.json --collection_ids [collection1,collection2] --delete_missing --index_fields [field1,field2]
+```
+
+The JSON file should follow the queryables schema as described in the [STAC API - Filter Extension](https://github.com/stac-api-extensions/filter#queryables). Here's an example:
+
+```json
+{
+  "$schema": "https://json-schema.org/draft/2019-09/schema",
+  "$id": "https://example.com/stac/queryables",
+  "type": "object",
+  "title": "Queryables for Example STAC API",
+  "description": "Queryable names for the Example STAC API",
+  "properties": {
+    "id": {
+      "description": "Item identifier",
+      "type": "string"
+    },
+    "datetime": {
+      "description": "Datetime",
+      "type": "string",
+      "format": "date-time"
+    },
+    "eo:cloud_cover": {
+      "description": "Cloud cover percentage",
+      "type": "number",
+      "minimum": 0,
+      "maximum": 100
+    }
+  },
+  "additionalProperties": true
+}
+```
+
+The command will extract the properties from the JSON file and create queryables in the database. It will also determine the appropriate property wrapper based on the type of each property and create the necessary indexes.
+
 ### Automated Collection Extent Updates
 
 By setting `pgstac.update_collection_extent` to `true`, a trigger is enabled to automatically adjust the spatial and temporal extents in collections when new items are ingested. This feature, while helpful, may increase overhead within data load transactions. To alleviate performance impact, combining this setting with `pgstac.use_queue` is beneficial. This approach necessitates a separate process, such as a scheduled task via the `pg_cron` extension, to periodically invoke `CALL run_queued_queries();`. Such asynchronous processing ensures efficient transactional performance and updated collection extents.

src/pypgstac/examples/load_queryables_example.py

@@ -0,0 +1,65 @@
+#!/usr/bin/env python
+"""
+Example script demonstrating how to load queryables into PgSTAC.
+
+This script shows how to use the load_queryables function both from the command line
+and programmatically.
+"""
+
+import sys
+from pathlib import Path
+
+# Add the parent directory to the path so we can import pypgstac
+sys.path.append(str(Path(__file__).parent.parent))
+
+from pypgstac.pypgstac import PgstacCLI
+
+
+def load_for_specific_collections(
+    cli, sample_file, collection_ids, delete_missing=False,
+):
+    """Load queryables for specific collections.
+
+    Args:
+        cli: PgstacCLI instance
+        sample_file: Path to the queryables file
+        collection_ids: List of collection IDs to apply queryables to
+        delete_missing: If True, delete properties not present in the file
+    """
+    cli.load_queryables(
+        str(sample_file), collection_ids=collection_ids, delete_missing=delete_missing,
+    )
+
+
+def main():
+    """Demonstrate loading queryables into PgSTAC."""
+    # Get the path to the sample queryables file
+    sample_file = Path(__file__).parent / "sample_queryables.json"
+
+    # Check if the file exists
+    if not sample_file.exists():
+        return
+
+    # Create a PgstacCLI instance
+    # This will use the standard PostgreSQL environment variables for connection
+    cli = PgstacCLI()
+
+    # Load queryables for all collections
+    cli.load_queryables(str(sample_file))
+
+    # Example of loading for specific collections
+    load_for_specific_collections(cli, sample_file, ["landsat-8", "sentinel-2"])
+
+    # Example of loading queryables with delete_missing=True
+    # This will delete properties not present in the file
+    cli.load_queryables(str(sample_file), delete_missing=True)
+
+    # Example of loading for specific collections with delete_missing=True
+    # This will delete properties not present in the file, but only for the specified collections
+    load_for_specific_collections(
+        cli, sample_file, ["landsat-8", "sentinel-2"], delete_missing=True,
+    )
+
+
+if __name__ == "__main__":
+    main()

src/pypgstac/examples/sample_queryables.json

@@ -0,0 +1,79 @@
+{
+  "$schema": "https://json-schema.org/draft/2019-09/schema",
+  "$id": "https://example.com/stac/queryables",
+  "type": "object",
+  "title": "Queryables for Example STAC API",
+  "description": "Queryable names for the Example STAC API",
+  "properties": {
+    "id": {
+      "description": "Item identifier",
+      "type": "string"
+    },
+    "collection": {
+      "description": "Collection identifier",
+      "type": "string"
+    },
+    "datetime": {
+      "description": "Datetime",
+      "type": "string",
+      "format": "date-time"
+    },
+    "geometry": {
+      "description": "Geometry",
+      "type": "object"
+    },
+    "eo:cloud_cover": {
+      "description": "Cloud cover percentage",
+      "type": "number",
+      "minimum": 0,
+      "maximum": 100
+    },
+    "platform": {
+      "description": "Platform name",
+      "type": "string",
+      "enum": ["landsat-8", "sentinel-2"]
+    },
+    "instrument": {
+      "description": "Instrument name",
+      "type": "string"
+    },
+    "gsd": {
+      "description": "Ground sample distance in meters",
+      "type": "number"
+    },
+    "view:off_nadir": {
+      "description": "Off-nadir angle in degrees",
+      "type": "number"
+    },
+    "view:sun_azimuth": {
+      "description": "Sun azimuth angle in degrees",
+      "type": "number"
+    },
+    "view:sun_elevation": {
+      "description": "Sun elevation angle in degrees",
+      "type": "number"
+    },
+    "sci:doi": {
+      "description": "Digital Object Identifier",
+      "type": "string"
+    },
+    "created": {
+      "description": "Date and time the item was created",
+      "type": "string",
+      "format": "date-time"
+    },
+    "updated": {
+      "description": "Date and time the item was last updated",
+      "type": "string",
+      "format": "date-time"
+    },
+    "landcover:classes": {
+      "description": "Land cover classes",
+      "type": "array",
+      "items": {
+        "type": "string"
+      }
+    }
+  },
+  "additionalProperties": true
+}