Merge branch 'main' into CAT-1525

Author: bountx

CHANGELOG.md

@@ -17,11 +17,23 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ### Fixed
 
+- Fix unawaited coroutine in `stac_fastapi.core.core`. [#551](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/551)
+- Parse `ES_TIMEOUT` environment variable as an integer. [#556](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/556)
+- Implemented "Smart Unlink" logic in delete_catalog: when cascade=False (default), collections are unlinked from the catalog and become root-level orphans if they have no other parents, rather than being deleted. When cascade=True, collections are deleted entirely. This prevents accidental data loss and supports poly-hierarchy scenarios where collections belong to multiple catalogs. [#557](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/557)
+- Fixed delete_catalog to use reverse lookup query on parent_ids field instead of fragile link parsing. This ensures all collections are found and updated correctly, preventing ghost relationships where collections remain tagged with deleted catalogs, especially in large catalogs or pagination scenarios. [#557](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/557)
+
+### Removed
+
+### Updated
+
+## [v6.7.6] - 2025-12-04
+
+### Fixed
+
 - Fix incorrect min/max date formatting in `apply_datetime_filter` for `POST` requests. [#539](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/539)
 - Fixed datetime filtering for .0Z milliseconds to preserve precision in apply_filter_datetime, ensuring only items exactly within the specified range are returned. [#535](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/535)
 - Normalize datetime in POST /search requests to match GET /search behavior. [#543](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/543)
-
-### Removed
+- Fix optional Redis support in core.py. [#549](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/549)
 
 ## [v6.7.5] - 2025-11-25
 
@@ -663,7 +675,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - Use genexp in execute_search and get_all_collections to return results.
 - Added db_to_stac serializer to item_collection method in core.py.
 
-[Unreleased]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.5...main
+[Unreleased]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.6...main
+[v6.7.6]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.5...v6.7.6
 [v6.7.5]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.4...v6.7.5
 [v6.7.4]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.3...v6.7.4
 [v6.7.3]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.2...v6.7.3

README.md

@@ -13,7 +13,7 @@
   [![GitHub forks](https://img.shields.io/github/forks/stac-utils/stac-fastapi-elasticsearch-opensearch.svg?color=blue)](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/network/members)
    [![PyPI version](https://img.shields.io/pypi/v/stac-fastapi-elasticsearch.svg?color=blue)](https://pypi.org/project/stac-fastapi-elasticsearch/)
   [![STAC](https://img.shields.io/badge/STAC-1.1.0-blue.svg)](https://github.com/radiantearth/stac-spec/tree/v1.1.0)
-  [![stac-fastapi](https://img.shields.io/badge/stac--fastapi-6.0.0-blue.svg)](https://github.com/stac-utils/stac-fastapi)
+  [![stac-fastapi](https://img.shields.io/badge/stac--fastapi-6.1.1-blue.svg)](https://github.com/stac-utils/stac-fastapi)
 
 ## Sponsors & Supporters
 
@@ -28,6 +28,7 @@ The following organizations have contributed time and/or funding to support the
 
 ## Latest News
 
+- **12/09/2025:** Feature Merge: **Federated Catalogs**. The [`Catalogs Endpoint`](https://github.com/Healy-Hyperspatial/stac-api-extensions-catalogs-endpoint) extension is now in main! This enables a registry of catalogs and supports **poly-hierarchy** (collections belonging to multiple catalogs simultaneously). Enable it via `ENABLE_CATALOGS_EXTENSION`. _Coming next: Support for nested sub-catalogs._
 - **11/07/2025:** 🌍 The SFEOS STAC Viewer is now available at: https://healy-hyperspatial.github.io/sfeos-web. Use this site to examine your data and test your STAC API!
 - **10/24/2025:** Added `previous_token` pagination using Redis for efficient navigation. This feature allows users to navigate backwards through large result sets by storing pagination state in Redis. To use this feature, ensure Redis is configured (see [Redis for navigation](#redis-for-navigation)) and set `REDIS_ENABLE=true` in your environment.
 - **10/23/2025:** The `EXCLUDED_FROM_QUERYABLES` environment variable was added to exclude fields from the `queryables` endpoint. See [docs](#excluding-fields-from-queryables).
@@ -92,6 +93,7 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI
   - [Technologies](#technologies)
   - [Table of Contents](#table-of-contents)
   - [Collection Search Extensions](#collection-search-extensions)
+  - [Catalogs Route](#catalogs-route)
   - [Documentation & Resources](#documentation--resources)
   - [SFEOS STAC Viewer](#sfeos-stac-viewer)
   - [Package Structure](#package-structure)
@@ -168,6 +170,8 @@ SFEOS provides enhanced collection search capabilities through two primary route
 - **GET/POST `/collections`**: The standard STAC endpoint with extended query parameters
 - **GET/POST `/collections-search`**: A custom endpoint that supports the same parameters, created to avoid conflicts with the STAC Transactions extension if enabled (which uses POST `/collections` for collection creation)
 
+The `/collections-search` endpoint follows the [STAC API Collection Search Endpoint](https://github.com/Healy-Hyperspatial/stac-api-extensions-collection-search-endpoint) specification, which provides a dedicated, conflict-free mechanism for advanced collection searching.
+
 These endpoints support advanced collection discovery features including:
 
 - **Sorting**: Sort collections by sortable fields using the `sortby` parameter
@@ -227,6 +231,103 @@ These extensions make it easier to build user interfaces that display and naviga
 > **Important**: Adding keyword fields to make text fields sortable can significantly increase the index size, especially for large text fields. Consider the storage implications when deciding which fields to make sortable.
 
 
+## Catalogs Route
+
+SFEOS supports federated hierarchical catalog browsing through the `/catalogs` endpoint, enabling users to navigate through STAC catalog structures in a tree-like fashion. This extension allows for organized discovery and browsing of collections and sub-catalogs.
+
+This implementation follows the [STAC API Catalogs Extension](https://github.com/Healy-Hyperspatial/stac-api-extensions-catalogs) specification, which enables a Federated STAC API architecture with a "Hub and Spoke" structure.
+
+### Features
+
+- **Hierarchical Navigation**: Browse catalogs and sub-catalogs in a parent-child relationship structure
+- **Multi-Catalog Collections**: Collections can belong to multiple catalogs simultaneously, enabling flexible organizational hierarchies
+- **Collection Discovery**: Access collections within specific catalog contexts
+- **STAC API Compliance**: Follows STAC specification for catalog objects and linking
+- **Flexible Querying**: Support for standard STAC API query parameters when browsing collections within catalogs
+
+### Endpoints
+
+- **GET `/catalogs`**: Retrieve the root catalog and its child catalogs
+- **POST `/catalogs`**: Create a new catalog (requires appropriate permissions)
+- **GET `/catalogs/{catalog_id}`**: Retrieve a specific catalog and its children
+- **DELETE `/catalogs/{catalog_id}`**: Delete a catalog (optionally cascade delete all collections)
+- **GET `/catalogs/{catalog_id}/collections`**: Retrieve collections within a specific catalog
+- **POST `/catalogs/{catalog_id}/collections`**: Create a new collection within a specific catalog
+- **GET `/catalogs/{catalog_id}/collections/{collection_id}`**: Retrieve a specific collection within a catalog
+- **DELETE `/catalogs/{catalog_id}/collections/{collection_id}`**: Delete a collection from a catalog (removes parent_id if multiple parents exist, deletes collection if it's the only parent)
+- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items`**: Retrieve items within a collection in a catalog context
+- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items/{item_id}`**: Retrieve a specific item within a catalog context
+
+### Usage Examples
+
+```bash
+# Get root catalog
+curl "http://localhost:8081/catalogs"
+
+# Get specific catalog
+curl "http://localhost:8081/catalogs/earth-observation"
+
+# Get collections in a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections"
+
+# Create a new collection within a catalog
+curl -X POST "http://localhost:8081/catalogs/earth-observation/collections" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "id": "landsat-9",
+    "type": "Collection",
+    "stac_version": "1.0.0",
+    "description": "Landsat 9 satellite imagery collection",
+    "title": "Landsat 9",
+    "license": "MIT",
+    "extent": {
+      "spatial": {"bbox": [[-180, -90, 180, 90]]},
+      "temporal": {"interval": [["2021-09-27T00:00:00Z", null]]}
+    }
+  }'
+
+# Get specific collection within a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2"
+
+# Get items in a collection within a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items"
+
+# Get specific item within a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items/S2A_20231015_123456"
+
+# Delete a collection from a catalog
+# If the collection has multiple parent catalogs, only removes this catalog from parent_ids
+# If this is the only parent catalog, deletes the collection entirely
+curl -X DELETE "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2"
+
+# Delete a catalog (collections remain intact)
+curl -X DELETE "http://localhost:8081/catalogs/earth-observation"
+
+# Delete a catalog and all its collections (cascade delete)
+curl -X DELETE "http://localhost:8081/catalogs/earth-observation?cascade=true"
+```
+
+### Delete Catalog Parameters
+
+The DELETE endpoint supports the following query parameter:
+
+- **`cascade`** (boolean, default: `false`): 
+  - If `false`: Only deletes the catalog. Collections linked to the catalog remain in the database but lose their catalog link.
+  - If `true`: Deletes the catalog AND all collections linked to it. Use with caution as this is a destructive operation.
+
+### Response Structure
+
+Catalog responses include:
+- **Catalog metadata**: ID, title, description, and other catalog properties
+- **Child catalogs**: Links to sub-catalogs for hierarchical navigation
+- **Collections**: Links to collections contained within the catalog
+- **STAC links**: Properly formatted STAC API links for navigation
+
+This feature enables building user interfaces that provide organized, hierarchical browsing of STAC collections, making it easier for users to discover and navigate through large collections organized by theme, provider, or any other categorization scheme.
+
+> **Configuration**: The catalogs route can be enabled or disabled by setting the `ENABLE_CATALOGS_ROUTE` environment variable to `true` or `false`. By default, this endpoint is **disabled**.
+
+
 ## Package Structure
 
 This project is organized into several packages, each with a specific purpose:
@@ -360,6 +461,7 @@ You can customize additional settings in your `.env` file:
 | `ENABLE_COLLECTIONS_SEARCH`  | Enable collection search extensions (sort, fields, free text search, structured filtering, and datetime filtering) on the core `/collections` endpoint. | `true`                   | Optional                                                                                    |
 | `ENABLE_COLLECTIONS_SEARCH_ROUTE` | Enable the custom `/collections-search` endpoint (both GET and POST methods). When disabled, the custom endpoint will not be available, but collection search extensions will still be available on the core `/collections` endpoint if `ENABLE_COLLECTIONS_SEARCH` is true. | `false` | Optional |
 | `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. This is useful for deployments where mutating the catalog via the API should be prevented. If set to `true`, the POST `/collections` route for search will be unavailable in the API. | `true` | Optional |
+| `ENABLE_CATALOGS_ROUTE` | Enable the `/catalogs` endpoint for federated hierarchical catalog browsing and navigation. When enabled, provides access to federated STAC API architecture with hub-and-spoke pattern. | `false` | Optional |
 | `STAC_GLOBAL_COLLECTION_MAX_LIMIT` | Configures the maximum number of STAC collections that can be returned in a single search request. | N/A | Optional |
 | `STAC_DEFAULT_COLLECTION_LIMIT` | Configures the default number of STAC collections returned when no limit parameter is specified in the request. | `300` | Optional |
 | `STAC_GLOBAL_ITEM_MAX_LIMIT` | Configures the maximum number of STAC items that can be returned in a single search request. | N/A | Optional |

compose.yml

@@ -20,9 +20,11 @@ services:
       - ES_PORT=9200
       - ES_USE_SSL=false
       - ES_VERIFY_CERTS=false
+      - ES_TIMEOUT=30
       - BACKEND=elasticsearch
       - DATABASE_REFRESH=true
       - ENABLE_COLLECTIONS_SEARCH_ROUTE=true
+      - ENABLE_CATALOGS_ROUTE=true
       - REDIS_ENABLE=true
       - REDIS_HOST=redis
       - REDIS_PORT=6379
@@ -59,9 +61,11 @@ services:
       - ES_PORT=9202
       - ES_USE_SSL=false
       - ES_VERIFY_CERTS=false
+      - ES_TIMEOUT=30
       - BACKEND=opensearch
       - STAC_FASTAPI_RATE_LIMIT=200/minute
       - ENABLE_COLLECTIONS_SEARCH_ROUTE=true
+      - ENABLE_CATALOGS_ROUTE=true
       - REDIS_ENABLE=true
       - REDIS_HOST=redis
       - REDIS_PORT=6379

stac_fastapi/core/stac_fastapi/core/base_database_logic.py

@@ -143,3 +143,38 @@ async def delete_collection(
     async def get_queryables_mapping(self, collection_id: str = "*") -> Dict[str, Any]:
         """Retrieve mapping of Queryables for search."""
         pass
+      
+    async def get_all_catalogs(
+        self,
+        token: Optional[str],
+        limit: int,
+        request: Any = None,
+        sort: Optional[List[Dict[str, Any]]] = None,
+    ) -> Tuple[List[Dict[str, Any]], Optional[str], Optional[int]]:
+        """Retrieve a list of catalogs from the database, supporting pagination.
+
+        Args:
+            token (Optional[str]): The pagination token.
+            limit (int): The number of results to return.
+            request (Any, optional): The FastAPI request object. Defaults to None.
+            sort (Optional[List[Dict[str, Any]]], optional): Optional sort parameter. Defaults to None.
+
+        Returns:
+            A tuple of (catalogs, next pagination token if any, optional count).
+        """
+        pass
+
+    @abc.abstractmethod
+    async def create_catalog(self, catalog: Dict, refresh: bool = False) -> None:
+        """Create a catalog in the database."""
+        pass
+
+    @abc.abstractmethod
+    async def find_catalog(self, catalog_id: str) -> Dict:
+        """Find a catalog in the database."""
+        pass
+
+    @abc.abstractmethod
+    async def delete_catalog(self, catalog_id: str, refresh: bool = False) -> None:
+        """Delete a catalog from the database."""
+        pass

stac_fastapi/core/stac_fastapi/core/core.py

@@ -30,6 +30,11 @@
 )
 from stac_fastapi.core.redis_utils import redis_pagination_links
 from stac_fastapi.core.serializers import CollectionSerializer, ItemSerializer
+from stac_fastapi.core.serializers import (
+    CatalogSerializer,
+    CollectionSerializer,
+    ItemSerializer,
+)
 from stac_fastapi.core.session import Session
 from stac_fastapi.core.utilities import filter_fields, get_bool_env
 from stac_fastapi.extensions.core.transaction import AsyncBaseTransactionsClient
@@ -86,6 +91,7 @@ class CoreClient(AsyncBaseCoreClient):
     collection_serializer: Type[CollectionSerializer] = attr.ib(
         default=CollectionSerializer
     )
+    catalog_serializer: Type[CatalogSerializer] = attr.ib(default=CatalogSerializer)
     post_request_model = attr.ib(default=BaseSearchPostRequest)
     stac_version: str = attr.ib(default=STAC_VERSION)
     landing_page_id: str = attr.ib(default="stac-fastapi")
@@ -95,6 +101,16 @@ class CoreClient(AsyncBaseCoreClient):
     def __attrs_post_init__(self):
         """Initialize the queryables cache."""
         self.queryables_cache = QueryablesCache(self.database)
+    def extension_is_enabled(self, extension_name: str) -> bool:
+        """Check if an extension is enabled by checking self.extensions.
+
+        Args:
+            extension_name: Name of the extension class to check for.
+
+        Returns:
+            True if the extension is in self.extensions, False otherwise.
+        """
+        return any(ext.__class__.__name__ == extension_name for ext in self.extensions)
 
     def _landing_page(
         self,
@@ -159,6 +175,7 @@ async def landing_page(self, **kwargs) -> stac_types.LandingPage:
             API landing page, serving as an entry point to the API.
         """
         request: Request = kwargs["request"]
+
         base_url = get_base_url(request)
         landing_page = self._landing_page(
             base_url=base_url,
@@ -216,6 +233,16 @@ async def landing_page(self, **kwargs) -> stac_types.LandingPage:
                 ]
             )
 
+        if self.extension_is_enabled("CatalogsExtension"):
+            landing_page["links"].append(
+                {
+                    "rel": "catalogs",
+                    "type": "application/json",
+                    "title": "Catalogs",
+                    "href": urljoin(base_url, "catalogs"),
+                }
+            )
+
         # Add OpenAPI URL
         landing_page["links"].append(
             {
@@ -434,6 +461,8 @@ async def all_collections(
         ]
 
         if redis_enable:
+            from stac_fastapi.core.redis_utils import redis_pagination_links
+
             await redis_pagination_links(
                 current_url=str(request.url),
                 token=token,
@@ -766,7 +795,7 @@ async def post_search(
 
         body_limit = None
         try:
-            if request.method == "POST" and request.body():
+            if request.method == "POST" and await request.body():
                 body_data = await request.json()
                 body_limit = body_data.get("limit")
         except Exception:
@@ -918,6 +947,8 @@ async def post_search(
         links.extend(collection_links)
 
         if redis_enable:
+            from stac_fastapi.core.redis_utils import redis_pagination_links
+
             await redis_pagination_links(
                 current_url=str(request.url),
                 token=token_param,

stac_fastapi/core/stac_fastapi/core/extensions/__init__.py

@@ -1,5 +1,6 @@
 """elasticsearch extensions modifications."""
 
+from .catalogs import CatalogsExtension
 from .collections_search import CollectionsSearchEndpointExtension
 from .query import Operator, QueryableTypes, QueryExtension
 
@@ -8,4 +9,5 @@
     "QueryableTypes",
     "QueryExtension",
     "CollectionsSearchEndpointExtension",
+    "CatalogsExtension",
 ]