Merge branch 'main' into release/6.4.0

jonhealy1 · web-flow · commit 10bb44c7040b · 2025-09-25T01:25:32.000+08:00
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -18,11 +18,13 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ### Added
 
+- GET `/collections` collection search free text extension ex. `/collections?q=sentinel`. [#470](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/470)
 - Added `USE_DATETIME` environment variable to configure datetime search behavior in SFEOS. [#452](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/452)
 - GET `/collections` collection search sort extension ex. `/collections?sortby=+id`. [#456](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/456)
 - GET `/collections` collection search fields extension ex. `/collections?fields=id,title`. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
 - Improved error messages for sorting on unsortable fields in collection search, including guidance on how to make fields sortable. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
 - Added field alias for `temporal` to enable easier sorting by temporal extent, alongside `extent.temporal.interval`. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
+- Added `ENABLE_COLLECTIONS_SEARCH` environment variable to make collection search extensions optional (defaults to enabled). [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
 
 ### Changed
 
diff --git a/README.md b/README.md
@@ -126,8 +126,15 @@ SFEOS implements extended capabilities for the `/collections` endpoint, allowing
   - Example: `/collections?fields=id,title,description`
   - This helps reduce payload size when only certain fields are needed
 
+- **Free Text Search**: Search across collection text fields using the `q` parameter
+  - Example: `/collections?q=landsat`
+  - Searches across multiple text fields including title, description, and keywords
+  - Supports partial word matching and relevance-based sorting
+
 These extensions make it easier to build user interfaces that display and navigate through collections efficiently.
 
+> **Configuration**: Collection search extensions can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.
+
 > **Note**: Sorting is only available on fields that are indexed for sorting in Elasticsearch/OpenSearch. With the default mappings, you can sort on:
 > - `id` (keyword field)
 > - `extent.temporal.interval` (date field)
@@ -267,6 +274,7 @@ You can customize additional settings in your `.env` file:
 | `ENABLE_DIRECT_RESPONSE`     | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false`                  | Optional                       |
 | `RAISE_ON_BULK_ERROR`        | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` | Optional |
 | `DATABASE_REFRESH`           | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
+| `ENABLE_COLLECTIONS_SEARCH`  | Enable collection search extensions (sort, fields).                                 | `true`                   | Optional                                                                                    |
 | `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. If set to `false`, the POST `/collections` route and related transaction endpoints (including bulk transaction operations) will be unavailable in the API. This is useful for deployments where mutating the catalog via the API should be prevented. | `true` | Optional |
 | `STAC_ITEM_LIMIT` | Sets the environment variable for result limiting to SFEOS for the number of returned items and STAC collections. | `10` | Optional |
 | `STAC_INDEX_ASSETS` | Controls if Assets are indexed when added to Elasticsearch/Opensearch. This allows asset fields to be included in search queries. | `false` | Optional |
@@ -413,6 +421,10 @@ The system uses a precise naming convention:
 - **Root Path Configuration**: The application root path is the base URL by default.
   - For AWS Lambda with Gateway API: Set `STAC_FASTAPI_ROOT_PATH` to match the Gateway API stage name (e.g., `/v1`)
 
+- **Feature Configuration**: Control which features are enabled:
+  - `ENABLE_COLLECTIONS_SEARCH`: Set to `true` (default) to enable collection search extensions (sort, fields). Set to `false` to disable.
+  - `ENABLE_TRANSACTIONS_EXTENSIONS`: Set to `true` (default) to enable transaction extensions. Set to `false` to disable.
+
 
 ## Collection Pagination
 
diff --git a/stac_fastapi/core/stac_fastapi/core/core.py b/stac_fastapi/core/stac_fastapi/core/core.py
@@ -225,13 +225,18 @@ async def landing_page(self, **kwargs) -> stac_types.LandingPage:
         return landing_page
 
     async def all_collections(
-        self, fields: Optional[List[str]] = None, sortby: Optional[str] = None, **kwargs
+        self,
+        fields: Optional[List[str]] = None,
+        sortby: Optional[str] = None,
+        q: Optional[Union[str, List[str]]] = None,
+        **kwargs,
     ) -> stac_types.Collections:
         """Read all collections from the database.
 
         Args:
             fields (Optional[List[str]]): Fields to include or exclude from the results.
             sortby (Optional[str]): Sorting options for the results.
+            q (Optional[List[str]]): Free text search terms.
             **kwargs: Keyword arguments from the request.
 
         Returns:
@@ -266,8 +271,13 @@ async def all_collections(
             if parsed_sort:
                 sort = parsed_sort
 
+        # Convert q to a list if it's a string
+        q_list = None
+        if q is not None:
+            q_list = [q] if isinstance(q, str) else q
+
         collections, next_token = await self.database.get_all_collections(
-            token=token, limit=limit, request=request, sort=sort
+            token=token, limit=limit, request=request, sort=sort, q=q_list
         )
 
         # Apply field filtering if fields parameter was provided
diff --git a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py
@@ -45,8 +45,7 @@
 )
 from stac_fastapi.extensions.core.fields import FieldsConformanceClasses
 from stac_fastapi.extensions.core.filter import FilterConformanceClasses
-
-# from stac_fastapi.extensions.core.free_text import FreeTextConformanceClasses
+from stac_fastapi.extensions.core.free_text import FreeTextConformanceClasses
 from stac_fastapi.extensions.core.query import QueryConformanceClasses
 from stac_fastapi.extensions.core.sort import SortConformanceClasses
 from stac_fastapi.extensions.third_party import BulkTransactionExtension
@@ -57,7 +56,9 @@
 logger = logging.getLogger(__name__)
 
 TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
+ENABLE_COLLECTIONS_SEARCH = get_bool_env("ENABLE_COLLECTIONS_SEARCH", default=True)
 logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)
+logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH)
 
 settings = ElasticsearchSettings()
 session = Session.create_from_settings(settings)
@@ -115,25 +116,26 @@
 
 extensions = [aggregation_extension] + search_extensions
 
-# Create collection search extensions
-# Only sort extension is enabled for now
-collection_search_extensions = [
-    # QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
-    SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
-    FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
-    # CollectionSearchFilterExtension(
-    #     conformance_classes=[FilterConformanceClasses.COLLECTIONS]
-    # ),
-    # FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
-]
-
-# Initialize collection search with its extensions
-collection_search_ext = CollectionSearchExtension.from_extensions(
-    collection_search_extensions
-)
-collections_get_request_model = collection_search_ext.GET
+# Create collection search extensions if enabled
+if ENABLE_COLLECTIONS_SEARCH:
+    # Create collection search extensions
+    collection_search_extensions = [
+        # QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
+        SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
+        FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
+        # CollectionSearchFilterExtension(
+        #     conformance_classes=[FilterConformanceClasses.COLLECTIONS]
+        # ),
+        FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
+    ]
+
+    # Initialize collection search with its extensions
+    collection_search_ext = CollectionSearchExtension.from_extensions(
+        collection_search_extensions
+    )
+    collections_get_request_model = collection_search_ext.GET
 
-extensions.append(collection_search_ext)
+    extensions.append(collection_search_ext)
 
 database_logic.extensions = [type(ext).__name__ for ext in extensions]
 
@@ -170,10 +172,13 @@
     "search_get_request_model": create_get_request_model(search_extensions),
     "search_post_request_model": post_request_model,
     "items_get_request_model": items_get_request_model,
-    "collections_get_request_model": collections_get_request_model,
     "route_dependencies": get_route_dependencies(),
 }
 
+# Add collections_get_request_model if collection search is enabled
+if ENABLE_COLLECTIONS_SEARCH:
+    app_config["collections_get_request_model"] = collections_get_request_model
+
 api = StacApi(**app_config)
 
 
diff --git a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/database_logic.py b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/database_logic.py
@@ -175,6 +175,7 @@ async def get_all_collections(
         limit: int,
         request: Request,
         sort: Optional[List[Dict[str, Any]]] = None,
+        q: Optional[List[str]] = None,
     ) -> Tuple[List[Dict[str, Any]], Optional[str]]:
         """Retrieve a list of collections from Elasticsearch, supporting pagination.
 
@@ -183,6 +184,7 @@ async def get_all_collections(
             limit (int): The number of results to return.
             request (Request): The FastAPI request object.
             sort (Optional[List[Dict[str, Any]]]): Optional sort parameter from the request.
+            q (Optional[List[str]]): Free text search terms.
 
         Returns:
             A tuple of (collections, next pagination token if any).
@@ -223,6 +225,38 @@ async def get_all_collections(
         if token:
             body["search_after"] = [token]
 
+        # Apply free text query if provided
+        if q:
+            # For collections, we want to search across all relevant fields
+            should_clauses = []
+
+            # For each search term
+            for term in q:
+                # Create a multi_match query for each term
+                for field in [
+                    "id",
+                    "title",
+                    "description",
+                    "keywords",
+                    "summaries.platform",
+                    "summaries.constellation",
+                    "providers.name",
+                    "providers.url",
+                ]:
+                    should_clauses.append(
+                        {
+                            "wildcard": {
+                                field: {"value": f"*{term}*", "case_insensitive": True}
+                            }
+                        }
+                    )
+
+            # Add the query to the body using bool query with should clauses
+            body["query"] = {
+                "bool": {"should": should_clauses, "minimum_should_match": 1}
+            }
+
+        # Execute the search
         response = await self.client.search(
             index=COLLECTIONS_INDEX,
             body=body,
diff --git a/stac_fastapi/opensearch/stac_fastapi/opensearch/app.py b/stac_fastapi/opensearch/stac_fastapi/opensearch/app.py
@@ -39,8 +39,7 @@
 )
 from stac_fastapi.extensions.core.fields import FieldsConformanceClasses
 from stac_fastapi.extensions.core.filter import FilterConformanceClasses
-
-# from stac_fastapi.extensions.core.free_text import FreeTextConformanceClasses
+from stac_fastapi.extensions.core.free_text import FreeTextConformanceClasses
 from stac_fastapi.extensions.core.query import QueryConformanceClasses
 from stac_fastapi.extensions.core.sort import SortConformanceClasses
 from stac_fastapi.extensions.third_party import BulkTransactionExtension
@@ -57,7 +56,9 @@
 logger = logging.getLogger(__name__)
 
 TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
+ENABLE_COLLECTIONS_SEARCH = get_bool_env("ENABLE_COLLECTIONS_SEARCH", default=True)
 logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)
+logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH)
 
 settings = OpensearchSettings()
 session = Session.create_from_settings(settings)
@@ -115,25 +116,26 @@
 
 extensions = [aggregation_extension] + search_extensions
 
-# Create collection search extensions
-# Only sort extension is enabled for now
-collection_search_extensions = [
-    # QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
-    SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
-    FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
-    # CollectionSearchFilterExtension(
-    #     conformance_classes=[FilterConformanceClasses.COLLECTIONS]
-    # ),
-    # FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
-]
-
-# Initialize collection search with its extensions
-collection_search_ext = CollectionSearchExtension.from_extensions(
-    collection_search_extensions
-)
-collections_get_request_model = collection_search_ext.GET
+# Create collection search extensions if enabled
+if ENABLE_COLLECTIONS_SEARCH:
+    # Create collection search extensions
+    collection_search_extensions = [
+        # QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
+        SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
+        FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
+        # CollectionSearchFilterExtension(
+        #     conformance_classes=[FilterConformanceClasses.COLLECTIONS]
+        # ),
+        FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
+    ]
+
+    # Initialize collection search with its extensions
+    collection_search_ext = CollectionSearchExtension.from_extensions(
+        collection_search_extensions
+    )
+    collections_get_request_model = collection_search_ext.GET
 
-extensions.append(collection_search_ext)
+    extensions.append(collection_search_ext)
 
 database_logic.extensions = [type(ext).__name__ for ext in extensions]
 
@@ -167,13 +169,16 @@
         post_request_model=post_request_model,
         landing_page_id=os.getenv("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"),
     ),
-    "collections_get_request_model": collections_get_request_model,
     "search_get_request_model": create_get_request_model(search_extensions),
     "search_post_request_model": post_request_model,
     "items_get_request_model": items_get_request_model,
     "route_dependencies": get_route_dependencies(),
 }
 
+# Add collections_get_request_model if collection search is enabled
+if ENABLE_COLLECTIONS_SEARCH:
+    app_config["collections_get_request_model"] = collections_get_request_model
+
 api = StacApi(**app_config)
 
 
diff --git a/stac_fastapi/opensearch/stac_fastapi/opensearch/database_logic.py b/stac_fastapi/opensearch/stac_fastapi/opensearch/database_logic.py
@@ -159,6 +159,7 @@ async def get_all_collections(
         limit: int,
         request: Request,
         sort: Optional[List[Dict[str, Any]]] = None,
+        q: Optional[List[str]] = None,
     ) -> Tuple[List[Dict[str, Any]], Optional[str]]:
         """Retrieve a list of collections from Elasticsearch, supporting pagination.
 
@@ -167,6 +168,7 @@ async def get_all_collections(
             limit (int): The number of results to return.
             request (Request): The FastAPI request object.
             sort (Optional[List[Dict[str, Any]]]): Optional sort parameter from the request.
+            q (Optional[List[str]]): Free text search terms.
 
         Returns:
             A tuple of (collections, next pagination token if any).
@@ -207,6 +209,37 @@ async def get_all_collections(
         if token:
             body["search_after"] = [token]
 
+        # Apply free text query if provided
+        if q:
+            # For collections, we want to search across all relevant fields
+            should_clauses = []
+
+            # For each search term
+            for term in q:
+                # Create a multi_match query for each term
+                for field in [
+                    "id",
+                    "title",
+                    "description",
+                    "keywords",
+                    "summaries.platform",
+                    "summaries.constellation",
+                    "providers.name",
+                    "providers.url",
+                ]:
+                    should_clauses.append(
+                        {
+                            "wildcard": {
+                                field: {"value": f"*{term}*", "case_insensitive": True}
+                            }
+                        }
+                    )
+
+            # Add the query to the body using bool query with should clauses
+            body["query"] = {
+                "bool": {"should": should_clauses, "minimum_should_match": 1}
+            }
+
         response = await self.client.search(
             index=COLLECTIONS_INDEX,
             body=body,
diff --git a/stac_fastapi/tests/api/test_api_search_collections.py b/stac_fastapi/tests/api/test_api_search_collections.py
diff --git a/stac_fastapi/tests/api/test_collections_search_env.py b/stac_fastapi/tests/api/test_collections_search_env.py
