|
1 | | -# stac-fastapi-elasticsearch-opensearch |
2 | | - |
3 | 1 | <!-- markdownlint-disable MD033 MD041 --> |
4 | 2 |
|
5 | 3 |
|
@@ -30,6 +28,7 @@ The following organizations have contributed time and/or funding to support the |
30 | 28 |
|
31 | 29 | ## Latest News |
32 | 30 |
|
| 31 | +- **11/07/2025:** 🌍 The SFEOS STAC Viewer is now available at: https://healy-hyperspatial.github.io/sfeos-web. Use this site to examine your data and test your STAC API! |
33 | 32 | - **10/24/2025:** Added `previous_token` pagination using Redis for efficient navigation. This feature allows users to navigate backwards through large result sets by storing pagination state in Redis. To use this feature, ensure Redis is configured (see [Redis for navigation](#redis-for-navigation)) and set `REDIS_ENABLE=true` in your environment. |
34 | 33 | - **10/23/2025:** The `EXCLUDED_FROM_QUERYABLES` environment variable was added to exclude fields from the `queryables` endpoint. See [docs](#excluding-fields-from-queryables). |
35 | 34 | - **10/15/2025:** 🚀 SFEOS Tools v0.1.0 Released! - The new `sfeos-tools` CLI is now available on [PyPI](https://pypi.org/project/sfeos-tools/) |
@@ -93,7 +92,9 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI |
93 | 92 | - [Technologies](#technologies) |
94 | 93 | - [Table of Contents](#table-of-contents) |
95 | 94 | - [Collection Search Extensions](#collection-search-extensions) |
| 95 | + - [Catalogs Route](#catalogs-route) |
96 | 96 | - [Documentation & Resources](#documentation--resources) |
| 97 | + - [SFEOS STAC Viewer](#sfeos-stac-viewer) |
97 | 98 | - [Package Structure](#package-structure) |
98 | 99 | - [Examples](#examples) |
99 | 100 | - [Performance](#performance) |
@@ -137,12 +138,39 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI |
137 | 138 | - [Gitter Chat](https://app.gitter.im/#/room/#stac-fastapi-elasticsearch_community:gitter.im) - For real-time discussions |
138 | 139 | - [GitHub Discussions](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/discussions) - For longer-form questions and answers |
139 | 140 |
|
| 141 | +## SFEOS STAC Viewer |
| 142 | + |
| 143 | +The SFEOS STAC viewer is a web-based application for examining and testing STAC APIs. It provides an interactive interface to explore geospatial data catalogs, visualize spatial extents, and test API endpoints. |
| 144 | + |
| 145 | +### Access |
| 146 | + |
| 147 | +The viewer is available at: https://healy-hyperspatial.github.io/sfeos-web/ |
| 148 | + |
| 149 | +### Features |
| 150 | + |
| 151 | +- Browse collections and items interactively |
| 152 | +- Interactive map visualization of spatial extents |
| 153 | +- Test API endpoints directly from the interface |
| 154 | +- Search and filter capabilities for exploring data |
| 155 | + |
| 156 | +### Usage |
| 157 | + |
| 158 | +Navigate to the URL above and connect to your SFEOS API instance by providing the base URL of your STAC API. This is done with the `API SERVER` button on the right side of the page. |
| 159 | + |
| 160 | +You can also override the default STAC API URL by appending the `stacApiUrl` parameter to the application URL. For example: |
| 161 | + |
| 162 | +https://healy-hyperspatial.github.io/sfeos-web?stacApiUrl=http://localhost:8080 |
| 163 | + |
| 164 | +**Note**: The parameter name `stacApiUrl` is case-sensitive. This allows you to connect to different STAC API servers without modifying the web app configuration. |
| 165 | + |
140 | 166 | ## Collection Search Extensions |
141 | 167 |
|
142 | 168 | SFEOS provides enhanced collection search capabilities through two primary routes: |
143 | 169 | - **GET/POST `/collections`**: The standard STAC endpoint with extended query parameters |
144 | 170 | - **GET/POST `/collections-search`**: A custom endpoint that supports the same parameters, created to avoid conflicts with the STAC Transactions extension if enabled (which uses POST `/collections` for collection creation) |
145 | 171 |
|
| 172 | +The `/collections-search` endpoint follows the [STAC API Collection Search Endpoint](https://github.com/Healy-Hyperspatial/stac-api-extensions-collection-search-endpoint) specification, which provides a dedicated, conflict-free mechanism for advanced collection searching. |
| 173 | + |
146 | 174 | These endpoints support advanced collection discovery features including: |
147 | 175 |
|
148 | 176 | - **Sorting**: Sort collections by sortable fields using the `sortby` parameter |
@@ -202,6 +230,96 @@ These extensions make it easier to build user interfaces that display and naviga |
202 | 230 | > **Important**: Adding keyword fields to make text fields sortable can significantly increase the index size, especially for large text fields. Consider the storage implications when deciding which fields to make sortable. |
203 | 231 |
|
204 | 232 |
|
| 233 | +## Catalogs Route |
| 234 | + |
| 235 | +SFEOS supports federated hierarchical catalog browsing through the `/catalogs` endpoint, enabling users to navigate through STAC catalog structures in a tree-like fashion. This extension allows for organized discovery and browsing of collections and sub-catalogs. |
| 236 | + |
| 237 | +This implementation follows the [STAC API Catalogs Extension](https://github.com/Healy-Hyperspatial/stac-api-extensions-catalogs) specification, which enables a Federated STAC API architecture with a "Hub and Spoke" structure. |
| 238 | + |
| 239 | +### Features |
| 240 | + |
| 241 | +- **Hierarchical Navigation**: Browse catalogs and sub-catalogs in a parent-child relationship structure |
| 242 | +- **Collection Discovery**: Access collections within specific catalog contexts |
| 243 | +- **STAC API Compliance**: Follows STAC specification for catalog objects and linking |
| 244 | +- **Flexible Querying**: Support for standard STAC API query parameters when browsing collections within catalogs |
| 245 | + |
| 246 | +### Endpoints |
| 247 | + |
| 248 | +- **GET `/catalogs`**: Retrieve the root catalog and its child catalogs |
| 249 | +- **POST `/catalogs`**: Create a new catalog (requires appropriate permissions) |
| 250 | +- **GET `/catalogs/{catalog_id}`**: Retrieve a specific catalog and its children |
| 251 | +- **DELETE `/catalogs/{catalog_id}`**: Delete a catalog (optionally cascade delete all collections) |
| 252 | +- **GET `/catalogs/{catalog_id}/collections`**: Retrieve collections within a specific catalog |
| 253 | +- **POST `/catalogs/{catalog_id}/collections`**: Create a new collection within a specific catalog |
| 254 | +- **GET `/catalogs/{catalog_id}/collections/{collection_id}`**: Retrieve a specific collection within a catalog |
| 255 | +- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items`**: Retrieve items within a collection in a catalog context |
| 256 | +- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items/{item_id}`**: Retrieve a specific item within a catalog context |
| 257 | + |
| 258 | +### Usage Examples |
| 259 | + |
| 260 | +```bash |
| 261 | +# Get root catalog |
| 262 | +curl "http://localhost:8081/catalogs" |
| 263 | + |
| 264 | +# Get specific catalog |
| 265 | +curl "http://localhost:8081/catalogs/earth-observation" |
| 266 | + |
| 267 | +# Get collections in a catalog |
| 268 | +curl "http://localhost:8081/catalogs/earth-observation/collections" |
| 269 | + |
| 270 | +# Create a new collection within a catalog |
| 271 | +curl -X POST "http://localhost:8081/catalogs/earth-observation/collections" \ |
| 272 | + -H "Content-Type: application/json" \ |
| 273 | + -d '{ |
| 274 | + "id": "landsat-9", |
| 275 | + "type": "Collection", |
| 276 | + "stac_version": "1.0.0", |
| 277 | + "description": "Landsat 9 satellite imagery collection", |
| 278 | + "title": "Landsat 9", |
| 279 | + "license": "MIT", |
| 280 | + "extent": { |
| 281 | + "spatial": {"bbox": [[-180, -90, 180, 90]]}, |
| 282 | + "temporal": {"interval": [["2021-09-27T00:00:00Z", null]]} |
| 283 | + } |
| 284 | + }' |
| 285 | + |
| 286 | +# Get specific collection within a catalog |
| 287 | +curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2" |
| 288 | + |
| 289 | +# Get items in a collection within a catalog |
| 290 | +curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items" |
| 291 | + |
| 292 | +# Get specific item within a catalog |
| 293 | +curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items/S2A_20231015_123456" |
| 294 | + |
| 295 | +# Delete a catalog (collections remain intact) |
| 296 | +curl -X DELETE "http://localhost:8081/catalogs/earth-observation" |
| 297 | + |
| 298 | +# Delete a catalog and all its collections (cascade delete) |
| 299 | +curl -X DELETE "http://localhost:8081/catalogs/earth-observation?cascade=true" |
| 300 | +``` |
| 301 | + |
| 302 | +### Delete Catalog Parameters |
| 303 | + |
| 304 | +The DELETE endpoint supports the following query parameter: |
| 305 | + |
| 306 | +- **`cascade`** (boolean, default: `false`): |
| 307 | + - If `false`: Only deletes the catalog. Collections linked to the catalog remain in the database but lose their catalog link. |
| 308 | + - If `true`: Deletes the catalog AND all collections linked to it. Use with caution as this is a destructive operation. |
| 309 | + |
| 310 | +### Response Structure |
| 311 | + |
| 312 | +Catalog responses include: |
| 313 | +- **Catalog metadata**: ID, title, description, and other catalog properties |
| 314 | +- **Child catalogs**: Links to sub-catalogs for hierarchical navigation |
| 315 | +- **Collections**: Links to collections contained within the catalog |
| 316 | +- **STAC links**: Properly formatted STAC API links for navigation |
| 317 | + |
| 318 | +This feature enables building user interfaces that provide organized, hierarchical browsing of STAC collections, making it easier for users to discover and navigate through large collections organized by theme, provider, or any other categorization scheme. |
| 319 | + |
| 320 | +> **Configuration**: The catalogs route can be enabled or disabled by setting the `ENABLE_CATALOGS_ROUTE` environment variable to `true` or `false`. By default, this endpoint is **disabled**. |
| 321 | +
|
| 322 | + |
205 | 323 | ## Package Structure |
206 | 324 |
|
207 | 325 | This project is organized into several packages, each with a specific purpose: |
@@ -335,13 +453,17 @@ You can customize additional settings in your `.env` file: |
335 | 453 | | `ENABLE_COLLECTIONS_SEARCH` | Enable collection search extensions (sort, fields, free text search, structured filtering, and datetime filtering) on the core `/collections` endpoint. | `true` | Optional | |
336 | 454 | | `ENABLE_COLLECTIONS_SEARCH_ROUTE` | Enable the custom `/collections-search` endpoint (both GET and POST methods). When disabled, the custom endpoint will not be available, but collection search extensions will still be available on the core `/collections` endpoint if `ENABLE_COLLECTIONS_SEARCH` is true. | `false` | Optional | |
337 | 455 | | `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. This is useful for deployments where mutating the catalog via the API should be prevented. If set to `true`, the POST `/collections` route for search will be unavailable in the API. | `true` | Optional | |
| 456 | +| `ENABLE_CATALOGS_ROUTE` | Enable the `/catalogs` endpoint for federated hierarchical catalog browsing and navigation. When enabled, provides access to federated STAC API architecture with hub-and-spoke pattern. | `false` | Optional | |
338 | 457 | | `STAC_GLOBAL_COLLECTION_MAX_LIMIT` | Configures the maximum number of STAC collections that can be returned in a single search request. | N/A | Optional | |
339 | 458 | | `STAC_DEFAULT_COLLECTION_LIMIT` | Configures the default number of STAC collections returned when no limit parameter is specified in the request. | `300` | Optional | |
340 | 459 | | `STAC_GLOBAL_ITEM_MAX_LIMIT` | Configures the maximum number of STAC items that can be returned in a single search request. | N/A | Optional | |
341 | 460 | | `STAC_DEFAULT_ITEM_LIMIT` | Configures the default number of STAC items returned when no limit parameter is specified in the request. | `10` | Optional | |
342 | 461 | | `STAC_INDEX_ASSETS` | Controls if Assets are indexed when added to Elasticsearch/Opensearch. This allows asset fields to be included in search queries. | `false` | Optional | |
343 | 462 | | `USE_DATETIME` | Configures the datetime search behavior in SFEOS. When enabled, searches both datetime field and falls back to start_datetime/end_datetime range for items with null datetime. When disabled, searches only by start_datetime/end_datetime range. | `true` | Optional | |
| 463 | +| `USE_DATETIME_NANOS` | Enables nanosecond precision handling for `datetime` field searches as per the `date_nanos` type. When `False`, it uses 3 millisecond precision as per the type `date`. | `true` | Optional | |
344 | 464 | | `EXCLUDED_FROM_QUERYABLES` | Comma-separated list of fully qualified field names to exclude from the queryables endpoint and filtering. Use full paths like `properties.auth:schemes,properties.storage:schemes`. Excluded fields and their nested children will not be exposed in queryables. | None | Optional | |
| 465 | +| `EXCLUDED_FROM_ITEMS` | Specifies fields to exclude from STAC item responses. Supports comma-separated field names and dot notation for nested fields (e.g., `private_data,properties.confidential,assets.internal`). | `None` | Optional | |
| 466 | + |
345 | 467 |
|
346 | 468 | > [!NOTE] |
347 | 469 | > The variables `ES_HOST`, `ES_PORT`, `ES_USE_SSL`, `ES_VERIFY_CERTS` and `ES_TIMEOUT` apply to both Elasticsearch and OpenSearch backends, so there is no need to rename the key names to `OS_` even if you're using OpenSearch. |
|
0 commit comments