update readme, opensearch

Author: jonhealy1

CHANGELOG.md

@@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 ### Added
 
 - GET `/collections` collection search structured filter extension with support for both cql2-json and cql2-text formats. [#475](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/475)
+- GET `/collections` collections search datetime filtering support. [#476](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/476)
 
 ### Changed
 

README.md

@@ -138,11 +138,19 @@ SFEOS implements extended capabilities for the `/collections` endpoint, allowing
   - Supports both CQL2 JSON and CQL2 text formats with various operators
   - Enables precise filtering on any collection property
 
-> **Note on HTTP Methods**: All collection search extensions (sorting, field selection, free text search, and structured filtering) currently only support GET requests. POST requests with these parameters in the request body are not yet supported.
+- **Datetime Filtering**: Filter collections by their temporal extent using the `datetime` parameter
+  - Example: `/collections?datetime=2020-01-01T00:00:00Z/2020-12-31T23:59:59Z` (finds collections with temporal extents that overlap this range)
+  - Example: `/collections?datetime=2020-06-15T12:00:00Z` (finds collections whose temporal extent includes this specific time)
+  - Example: `/collections?datetime=2020-01-01T00:00:00Z/..` (finds collections with temporal extents that extend to or beyond January 1, 2020)
+  - Example: `/collections?datetime=../2020-12-31T23:59:59Z` (finds collections with temporal extents that begin on or before December 31, 2020)
+  - Collections are matched if their temporal extent overlaps with the provided datetime parameter
+  - This allows for efficient discovery of collections based on time periods
+
+> **Note on HTTP Methods**: All collection search extensions (sorting, field selection, free text search, structured filtering, and datetime filtering) currently only support GET requests. POST requests with these parameters in the request body are not yet supported.
 
 These extensions make it easier to build user interfaces that display and navigate through collections efficiently.
 
-> **Configuration**: Collection search extensions (sorting, field selection, free text search, and structured filtering) can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.
+> **Configuration**: Collection search extensions (sorting, field selection, free text search, structured filtering, and datetime filtering) can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.
 
 > **Note**: Sorting is only available on fields that are indexed for sorting in Elasticsearch/OpenSearch. With the default mappings, you can sort on:
 > - `id` (keyword field)
@@ -283,12 +291,12 @@ You can customize additional settings in your `.env` file:
 | `ENABLE_DIRECT_RESPONSE`     | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false`                  | Optional                       |
 | `RAISE_ON_BULK_ERROR`        | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` | Optional |
 | `DATABASE_REFRESH`           | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
-| `ENABLE_COLLECTIONS_SEARCH`  | Enable collection search extensions (sort, fields).                                 | `true`                   | Optional                                                                                    |
+| `ENABLE_COLLECTIONS_SEARCH`  | Enable collection search extensions (sort, fields, free text search, structured filtering, and datetime filtering).                                 | `true`                   | Optional                                                                                    |
 | `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. If set to `false`, the POST `/collections` route and related transaction endpoints (including bulk transaction operations) will be unavailable in the API. This is useful for deployments where mutating the catalog via the API should be prevented. | `true` | Optional |
 | `STAC_ITEM_LIMIT` | Sets the environment variable for result limiting to SFEOS for the number of returned items and STAC collections. | `10` | Optional |
 | `STAC_INDEX_ASSETS` | Controls if Assets are indexed when added to Elasticsearch/Opensearch. This allows asset fields to be included in search queries. | `false` | Optional |
 | `ENV_MAX_LIMIT` | Configures the environment variable in SFEOS to override the default `MAX_LIMIT`, which controls the limit parameter for returned items and STAC collections. | `10,000` | Optional |
-| `USE_DATETIME` | Configures the datetime search behavior in SFEOS. When enabled, searches both datetime field and falls back to start_datetime/end_datetime range for items with null datetime. When disabled, searches only by start_datetime/end_datetime range. | True | Optional |
+| `USE_DATETIME` | Configures the datetime search behavior in SFEOS. When enabled, searches both datetime field and falls back to start_datetime/end_datetime range for items with null datetime. When disabled, searches only by start_datetime/end_datetime range. This applies to both item searches and collection searches. | `true` | Optional |
 
 > [!NOTE]
 > The variables `ES_HOST`, `ES_PORT`, `ES_USE_SSL`, `ES_VERIFY_CERTS` and `ES_TIMEOUT` apply to both Elasticsearch and OpenSearch backends, so there is no need to rename the key names to `OS_` even if you're using OpenSearch.

stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/database_logic.py

@@ -188,6 +188,7 @@ async def get_all_collections(
             sort (Optional[List[Dict[str, Any]]]): Optional sort parameter from the request.
             q (Optional[List[str]]): Free text search terms.
             filter (Optional[Dict[str, Any]]): Structured query in CQL2 format.
+            datetime (Optional[str]): Temporal filter.
 
         Returns:
             A tuple of (collections, next pagination token if any).
@@ -271,16 +272,12 @@ async def get_all_collections(
             es_query = filter_module.to_es(await self.get_queryables_mapping(), filter)
             query_parts.append(es_query)
 
-        print("datetime: ", datetime)
-        print("type datetime, ", type(datetime))
         datetime_filter = None
         if datetime:
             datetime_filter = self._apply_collection_datetime_filter(datetime)
             if datetime_filter:
                 query_parts.append(datetime_filter)
 
-        print("datetime filter: ", datetime_filter)
-
         # Combine all query parts with AND logic
         if query_parts:
             body["query"] = (
@@ -333,18 +330,6 @@ def _apply_collection_datetime_filter(
             # If it's just a single date, use it for both start and end
             start = end = datetime_str
 
-        # For a collection with temporal extent [start_date, end_date],
-        # a datetime query should match if the datetime falls within the range.
-        # For a date range query, it should match if the ranges overlap.
-
-        # For collections, we need a different approach because the temporal extent
-        # is stored as an array of dates, not as a range field.
-        # We need to check if:
-        # 1. The collection's start date is before or equal to the query end date
-        # 2. The collection's end date is after or equal to the query start date
-
-        # This is a bit tricky with Elasticsearch's flattened arrays, but we can use
-        # a bool query to check both conditions
         return {
             "bool": {
                 "must": [

stac_fastapi/opensearch/stac_fastapi/opensearch/database_logic.py

@@ -161,6 +161,7 @@ async def get_all_collections(
         sort: Optional[List[Dict[str, Any]]] = None,
         q: Optional[List[str]] = None,
         filter: Optional[Dict[str, Any]] = None,
+        datetime: Optional[str] = None,
     ) -> Tuple[List[Dict[str, Any]], Optional[str]]:
         """Retrieve a list of collections from Opensearch, supporting pagination.
 
@@ -171,6 +172,7 @@ async def get_all_collections(
             sort (Optional[List[Dict[str, Any]]]): Optional sort parameter from the request.
             q (Optional[List[str]]): Free text search terms.
             filter (Optional[Dict[str, Any]]): Structured query in CQL2 format.
+            datetime (Optional[str]): Temporal filter.
 
         Returns:
             A tuple of (collections, next pagination token if any).
@@ -254,6 +256,12 @@ async def get_all_collections(
             es_query = filter_module.to_es(await self.get_queryables_mapping(), filter)
             query_parts.append(es_query)
 
+        datetime_filter = None
+        if datetime:
+            datetime_filter = self._apply_collection_datetime_filter(datetime)
+            if datetime_filter:
+                query_parts.append(datetime_filter)
+
         # Combine all query parts with AND logic
         if query_parts:
             body["query"] = (
@@ -370,6 +378,41 @@ def apply_free_text_filter(search: Search, free_text_queries: Optional[List[str]
             search=search, free_text_queries=free_text_queries
         )
 
+    @staticmethod
+    def _apply_collection_datetime_filter(
+        datetime_str: Optional[str],
+    ) -> Optional[Dict[str, Any]]:
+        """Create a temporal filter for collections based on their extent."""
+        if not datetime_str:
+            return None
+
+        # Parse the datetime string into start and end
+        if "/" in datetime_str:
+            start, end = datetime_str.split("/")
+            # Replace open-ended ranges with concrete dates
+            if start == "..":
+                # For open-ended start, use a very early date
+                start = "1800-01-01T00:00:00Z"
+            if end == "..":
+                # For open-ended end, use a far future date
+                end = "2999-12-31T23:59:59Z"
+        else:
+            # If it's just a single date, use it for both start and end
+            start = end = datetime_str
+
+        return {
+            "bool": {
+                "must": [
+                    # Check if any date in the array is less than or equal to the query end date
+                    # This will match if the collection's start date is before or equal to the query end date
+                    {"range": {"extent.temporal.interval": {"lte": end}}},
+                    # Check if any date in the array is greater than or equal to the query start date
+                    # This will match if the collection's end date is after or equal to the query start date
+                    {"range": {"extent.temporal.interval": {"gte": start}}},
+                ]
+            }
+        }
+
     @staticmethod
     def apply_datetime_filter(
         search: Search, datetime: Optional[str]

stac_fastapi/tests/api/test_api_search_collections.py

@@ -316,28 +316,18 @@ async def test_collections_filter_search(app_client, txn_client, load_test_data)
 
 
 @pytest.mark.asyncio
-async def test_collections_datetime_filter(app_client, load_test_data):
+async def test_collections_datetime_filter(app_client, load_test_data, txn_client):
     """Test filtering collections by datetime."""
     # Create a test collection with a specific temporal extent
-    test_collection_id = "test-collection-datetime"
-    test_collection = {
-        "id": test_collection_id,
-        "type": "Collection",
-        "stac_version": "1.0.0",
-        "description": "Test collection for datetime filtering",
-        "links": [],
-        "extent": {
-            "spatial": {"bbox": [[-180, -90, 180, 90]]},
-            "temporal": {
-                "interval": [["2020-01-01T00:00:00Z", "2020-12-31T23:59:59Z"]]
-            },
-        },
-        "license": "proprietary",
-    }
 
-    # Create the test collection
-    resp = await app_client.post("/collections", json=test_collection)
-    assert resp.status_code == 201
+    base_collection = load_test_data("test_collection.json")
+    base_collection["extent"]["temporal"]["interval"] = [
+        ["2020-01-01T00:00:00Z", "2020-12-31T23:59:59Z"]
+    ]
+    test_collection_id = base_collection["id"]
+
+    await create_collection(txn_client, base_collection)
+    await refresh_indices(txn_client)
 
     # Test 1: Datetime range that overlaps with collection's temporal extent
     resp = await app_client.get(
@@ -413,7 +403,3 @@ async def test_collections_datetime_filter(app_client, load_test_data):
     found_collections = [c for c in resp_json["collections"] if c["id"] == test_collection_id]
     assert len(found_collections) == 1, f"Expected to find collection {test_collection_id} with open-ended past range to a date within its range"
     """
-
-    # Clean up - delete the test collection
-    resp = await app_client.delete(f"/collections/{test_collection_id}")
-    assert resp.status_code == 204