Merge branch 'main' into CAT-1622

Author: bountx

CHANGELOG.md

@@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - Added `STAC_FASTAPI_ES_MAPPINGS_FILE` environment variable to support file-based custom mappings configuration.
 - Added configuration-based support for extending Elasticsearch/OpenSearch index mappings via environment variables, allowing users to customize field mappings without code change through `STAC_FASTAPI_ES_CUSTOM_MAPPINGS` environment variable. Also added `STAC_FASTAPI_ES_DYNAMIC_MAPPING` variable to control dynamic mapping behavior. [#546](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/546)
 
+- Added catalogs route support to enable federated hierarchical catalog browsing and navigation in the STAC API. [#547](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/547)
+
 ### Changed
 
 ### Fixed

README.md

@@ -92,6 +92,7 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI
   - [Technologies](#technologies)
   - [Table of Contents](#table-of-contents)
   - [Collection Search Extensions](#collection-search-extensions)
+  - [Catalogs Route](#catalogs-route)
   - [Documentation & Resources](#documentation--resources)
   - [SFEOS STAC Viewer](#sfeos-stac-viewer)
   - [Package Structure](#package-structure)
@@ -169,6 +170,8 @@ SFEOS provides enhanced collection search capabilities through two primary route
 - **GET/POST `/collections`**: The standard STAC endpoint with extended query parameters
 - **GET/POST `/collections-search`**: A custom endpoint that supports the same parameters, created to avoid conflicts with the STAC Transactions extension if enabled (which uses POST `/collections` for collection creation)
 
+The `/collections-search` endpoint follows the [STAC API Collection Search Endpoint](https://github.com/Healy-Hyperspatial/stac-api-extensions-collection-search-endpoint) specification, which provides a dedicated, conflict-free mechanism for advanced collection searching.
+
 These endpoints support advanced collection discovery features including:
 
 - **Sorting**: Sort collections by sortable fields using the `sortby` parameter
@@ -228,6 +231,96 @@ These extensions make it easier to build user interfaces that display and naviga
 > **Important**: Adding keyword fields to make text fields sortable can significantly increase the index size, especially for large text fields. Consider the storage implications when deciding which fields to make sortable.
 
 
+## Catalogs Route
+
+SFEOS supports federated hierarchical catalog browsing through the `/catalogs` endpoint, enabling users to navigate through STAC catalog structures in a tree-like fashion. This extension allows for organized discovery and browsing of collections and sub-catalogs.
+
+This implementation follows the [STAC API Catalogs Extension](https://github.com/Healy-Hyperspatial/stac-api-extensions-catalogs) specification, which enables a Federated STAC API architecture with a "Hub and Spoke" structure.
+
+### Features
+
+- **Hierarchical Navigation**: Browse catalogs and sub-catalogs in a parent-child relationship structure
+- **Collection Discovery**: Access collections within specific catalog contexts
+- **STAC API Compliance**: Follows STAC specification for catalog objects and linking
+- **Flexible Querying**: Support for standard STAC API query parameters when browsing collections within catalogs
+
+### Endpoints
+
+- **GET `/catalogs`**: Retrieve the root catalog and its child catalogs
+- **POST `/catalogs`**: Create a new catalog (requires appropriate permissions)
+- **GET `/catalogs/{catalog_id}`**: Retrieve a specific catalog and its children
+- **DELETE `/catalogs/{catalog_id}`**: Delete a catalog (optionally cascade delete all collections)
+- **GET `/catalogs/{catalog_id}/collections`**: Retrieve collections within a specific catalog
+- **POST `/catalogs/{catalog_id}/collections`**: Create a new collection within a specific catalog
+- **GET `/catalogs/{catalog_id}/collections/{collection_id}`**: Retrieve a specific collection within a catalog
+- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items`**: Retrieve items within a collection in a catalog context
+- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items/{item_id}`**: Retrieve a specific item within a catalog context
+
+### Usage Examples
+
+```bash
+# Get root catalog
+curl "http://localhost:8081/catalogs"
+
+# Get specific catalog
+curl "http://localhost:8081/catalogs/earth-observation"
+
+# Get collections in a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections"
+
+# Create a new collection within a catalog
+curl -X POST "http://localhost:8081/catalogs/earth-observation/collections" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "id": "landsat-9",
+    "type": "Collection",
+    "stac_version": "1.0.0",
+    "description": "Landsat 9 satellite imagery collection",
+    "title": "Landsat 9",
+    "license": "MIT",
+    "extent": {
+      "spatial": {"bbox": [[-180, -90, 180, 90]]},
+      "temporal": {"interval": [["2021-09-27T00:00:00Z", null]]}
+    }
+  }'
+
+# Get specific collection within a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2"
+
+# Get items in a collection within a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items"
+
+# Get specific item within a catalog
+curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items/S2A_20231015_123456"
+
+# Delete a catalog (collections remain intact)
+curl -X DELETE "http://localhost:8081/catalogs/earth-observation"
+
+# Delete a catalog and all its collections (cascade delete)
+curl -X DELETE "http://localhost:8081/catalogs/earth-observation?cascade=true"
+```
+
+### Delete Catalog Parameters
+
+The DELETE endpoint supports the following query parameter:
+
+- **`cascade`** (boolean, default: `false`): 
+  - If `false`: Only deletes the catalog. Collections linked to the catalog remain in the database but lose their catalog link.
+  - If `true`: Deletes the catalog AND all collections linked to it. Use with caution as this is a destructive operation.
+
+### Response Structure
+
+Catalog responses include:
+- **Catalog metadata**: ID, title, description, and other catalog properties
+- **Child catalogs**: Links to sub-catalogs for hierarchical navigation
+- **Collections**: Links to collections contained within the catalog
+- **STAC links**: Properly formatted STAC API links for navigation
+
+This feature enables building user interfaces that provide organized, hierarchical browsing of STAC collections, making it easier for users to discover and navigate through large collections organized by theme, provider, or any other categorization scheme.
+
+> **Configuration**: The catalogs route can be enabled or disabled by setting the `ENABLE_CATALOGS_ROUTE` environment variable to `true` or `false`. By default, this endpoint is **disabled**.
+
+
 ## Package Structure
 
 This project is organized into several packages, each with a specific purpose:
@@ -361,6 +454,7 @@ You can customize additional settings in your `.env` file:
 | `ENABLE_COLLECTIONS_SEARCH`  | Enable collection search extensions (sort, fields, free text search, structured filtering, and datetime filtering) on the core `/collections` endpoint. | `true`                   | Optional                                                                                    |
 | `ENABLE_COLLECTIONS_SEARCH_ROUTE` | Enable the custom `/collections-search` endpoint (both GET and POST methods). When disabled, the custom endpoint will not be available, but collection search extensions will still be available on the core `/collections` endpoint if `ENABLE_COLLECTIONS_SEARCH` is true. | `false` | Optional |
 | `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. This is useful for deployments where mutating the catalog via the API should be prevented. If set to `true`, the POST `/collections` route for search will be unavailable in the API. | `true` | Optional |
+| `ENABLE_CATALOGS_ROUTE` | Enable the `/catalogs` endpoint for federated hierarchical catalog browsing and navigation. When enabled, provides access to federated STAC API architecture with hub-and-spoke pattern. | `false` | Optional |
 | `STAC_GLOBAL_COLLECTION_MAX_LIMIT` | Configures the maximum number of STAC collections that can be returned in a single search request. | N/A | Optional |
 | `STAC_DEFAULT_COLLECTION_LIMIT` | Configures the default number of STAC collections returned when no limit parameter is specified in the request. | `300` | Optional |
 | `STAC_GLOBAL_ITEM_MAX_LIMIT` | Configures the maximum number of STAC items that can be returned in a single search request. | N/A | Optional |

compose.yml

@@ -23,6 +23,7 @@ services:
       - BACKEND=elasticsearch
       - DATABASE_REFRESH=true
       - ENABLE_COLLECTIONS_SEARCH_ROUTE=true
+      - ENABLE_CATALOGS_ROUTE=true
       - REDIS_ENABLE=true
       - REDIS_HOST=redis
       - REDIS_PORT=6379
@@ -62,6 +63,7 @@ services:
       - BACKEND=opensearch
       - STAC_FASTAPI_RATE_LIMIT=200/minute
       - ENABLE_COLLECTIONS_SEARCH_ROUTE=true
+      - ENABLE_CATALOGS_ROUTE=true
       - REDIS_ENABLE=true
       - REDIS_HOST=redis
       - REDIS_PORT=6379

stac_fastapi/core/stac_fastapi/core/base_database_logic.py

@@ -138,3 +138,39 @@ async def delete_collection(
     ) -> None:
         """Delete a collection from the database."""
         pass
+
+    @abc.abstractmethod
+    async def get_all_catalogs(
+        self,
+        token: Optional[str],
+        limit: int,
+        request: Any = None,
+        sort: Optional[List[Dict[str, Any]]] = None,
+    ) -> Tuple[List[Dict[str, Any]], Optional[str], Optional[int]]:
+        """Retrieve a list of catalogs from the database, supporting pagination.
+
+        Args:
+            token (Optional[str]): The pagination token.
+            limit (int): The number of results to return.
+            request (Any, optional): The FastAPI request object. Defaults to None.
+            sort (Optional[List[Dict[str, Any]]], optional): Optional sort parameter. Defaults to None.
+
+        Returns:
+            A tuple of (catalogs, next pagination token if any, optional count).
+        """
+        pass
+
+    @abc.abstractmethod
+    async def create_catalog(self, catalog: Dict, refresh: bool = False) -> None:
+        """Create a catalog in the database."""
+        pass
+
+    @abc.abstractmethod
+    async def find_catalog(self, catalog_id: str) -> Dict:
+        """Find a catalog in the database."""
+        pass
+
+    @abc.abstractmethod
+    async def delete_catalog(self, catalog_id: str, refresh: bool = False) -> None:
+        """Delete a catalog from the database."""
+        pass

stac_fastapi/core/stac_fastapi/core/core.py

@@ -24,7 +24,11 @@
 from stac_fastapi.core.base_settings import ApiBaseSettings
 from stac_fastapi.core.datetime_utils import format_datetime_range
 from stac_fastapi.core.models.links import PagingLinks
-from stac_fastapi.core.serializers import CollectionSerializer, ItemSerializer
+from stac_fastapi.core.serializers import (
+    CatalogSerializer,
+    CollectionSerializer,
+    ItemSerializer,
+)
 from stac_fastapi.core.session import Session
 from stac_fastapi.core.utilities import filter_fields, get_bool_env
 from stac_fastapi.extensions.core.transaction import AsyncBaseTransactionsClient
@@ -81,12 +85,24 @@ class CoreClient(AsyncBaseCoreClient):
     collection_serializer: Type[CollectionSerializer] = attr.ib(
         default=CollectionSerializer
     )
+    catalog_serializer: Type[CatalogSerializer] = attr.ib(default=CatalogSerializer)
     post_request_model = attr.ib(default=BaseSearchPostRequest)
     stac_version: str = attr.ib(default=STAC_VERSION)
     landing_page_id: str = attr.ib(default="stac-fastapi")
     title: str = attr.ib(default="stac-fastapi")
     description: str = attr.ib(default="stac-fastapi")
 
+    def extension_is_enabled(self, extension_name: str) -> bool:
+        """Check if an extension is enabled by checking self.extensions.
+
+        Args:
+            extension_name: Name of the extension class to check for.
+
+        Returns:
+            True if the extension is in self.extensions, False otherwise.
+        """
+        return any(ext.__class__.__name__ == extension_name for ext in self.extensions)
+
     def _landing_page(
         self,
         base_url: str,
@@ -150,6 +166,7 @@ async def landing_page(self, **kwargs) -> stac_types.LandingPage:
             API landing page, serving as an entry point to the API.
         """
         request: Request = kwargs["request"]
+
         base_url = get_base_url(request)
         landing_page = self._landing_page(
             base_url=base_url,
@@ -207,6 +224,16 @@ async def landing_page(self, **kwargs) -> stac_types.LandingPage:
                 ]
             )
 
+        if self.extension_is_enabled("CatalogsExtension"):
+            landing_page["links"].append(
+                {
+                    "rel": "catalogs",
+                    "type": "application/json",
+                    "title": "Catalogs",
+                    "href": urljoin(base_url, "catalogs"),
+                }
+            )
+
         # Add OpenAPI URL
         landing_page["links"].append(
             {

stac_fastapi/core/stac_fastapi/core/extensions/__init__.py

@@ -1,5 +1,6 @@
 """elasticsearch extensions modifications."""
 
+from .catalogs import CatalogsExtension
 from .collections_search import CollectionsSearchEndpointExtension
 from .query import Operator, QueryableTypes, QueryExtension
 
@@ -8,4 +9,5 @@
     "QueryableTypes",
     "QueryExtension",
     "CollectionsSearchEndpointExtension",
+    "CatalogsExtension",
 ]