feat: add STAC_FASTAPI_ES_MAPPINGS_FILE for file-based custom mappings configuration

Author: Andrzej Pijanowski

CHANGELOG.md

@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 ## [Unreleased]
 
 ### Added
+- Added `STAC_FASTAPI_ES_MAPPINGS_FILE` environment variable to support file-based custom mappings configuration.
 - Added configuration-based support for extending Elasticsearch/OpenSearch index mappings via environment variables, allowing users to customize field mappings without code change through `STAC_FASTAPI_ES_CUSTOM_MAPPINGS` environment variable. Also added `STAC_FASTAPI_ES_DYNAMIC_MAPPING` variable to control dynamic mapping behavior. [#546](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/546)
 
 ### Changed

README.md

@@ -371,6 +371,7 @@ You can customize additional settings in your `.env` file:
 | `EXCLUDED_FROM_QUERYABLES` | Comma-separated list of fully qualified field names to exclude from the queryables endpoint and filtering. Use full paths like `properties.auth:schemes,properties.storage:schemes`. Excluded fields and their nested children will not be exposed in queryables. | None | Optional |
 | `EXCLUDED_FROM_ITEMS` | Specifies fields to exclude from STAC item responses. Supports comma-separated field names and dot notation for nested fields (e.g., `private_data,properties.confidential,assets.internal`). | `None` | Optional |
 | `STAC_FASTAPI_ES_CUSTOM_MAPPINGS` | JSON string of custom Elasticsearch/OpenSearch property mappings to merge with defaults. See [Custom Index Mappings](#custom-index-mappings). | `None` | Optional |
+| `STAC_FASTAPI_ES_MAPPINGS_FILE` | Path to a JSON file containing custom Elasticsearch/OpenSearch property mappings to merge with defaults. See [Custom Index Mappings](#custom-index-mappings). | `None` | Optional |
 | `STAC_FASTAPI_ES_DYNAMIC_MAPPING` | Controls dynamic mapping behavior for item indices. Values: `true` (default), `false`, or `strict`. See [Custom Index Mappings](#custom-index-mappings). | `true` | Optional |
 
 
@@ -709,11 +710,17 @@ SFEOS provides environment variables to customize Elasticsearch/OpenSearch index
 | Variable | Description | Default |
 |----------|-------------|---------|
 | `STAC_FASTAPI_ES_CUSTOM_MAPPINGS` | JSON string of property mappings to merge with defaults | None |
+| `STAC_FASTAPI_ES_MAPPINGS_FILE` | Path to a JSON file containing property mappings to merge with defaults | None |
 | `STAC_FASTAPI_ES_DYNAMIC_MAPPING` | Controls dynamic mapping: `true`, `false`, or `strict` | `true` |
 
-### Custom Mappings (`STAC_FASTAPI_ES_CUSTOM_MAPPINGS`)
+### Custom Mappings
 
-Accepts a JSON string with the same structure as the default ES mappings. The custom mappings are **recursively merged** with the defaults at the root level.
+You can customize the Elasticsearch/OpenSearch mappings by providing a JSON configuration. This can be done via:
+
+1. `STAC_FASTAPI_ES_CUSTOM_MAPPINGS` environment variable (takes precedence)
+2. `STAC_FASTAPI_ES_MAPPINGS_FILE` environment variable (file path)
+
+The configuration should have the same structure as the default ES mappings. The custom mappings are **recursively merged** with the defaults at the root level.
 
 #### Merge Behavior
 
@@ -806,6 +813,86 @@ export STAC_FASTAPI_ES_CUSTOM_MAPPINGS='{
 }'
 ```
 
+**Example - Using a mappings file (recommended for complex configurations):**
+
+Instead of passing large JSON blobs via environment variables, you can use a file:
+
+```bash
+# Create a mappings file
+cat > custom-mappings.json <<EOF
+{
+  "properties": {
+    "properties": {
+      "properties": {
+        "sar:frequency_band": {"type": "keyword"},
+        "sar:center_frequency": {"type": "float"},
+        "sar:polarizations": {"type": "keyword"},
+        "sar:product_type": {"type": "keyword"},
+        "eo:cloud_cover": {"type": "float"},
+        "platform": {"type": "keyword"}
+      }
+    }
+  }
+}
+EOF
+
+# Reference the file
+export STAC_FASTAPI_ES_MAPPINGS_FILE=/path/to/custom-mappings.json
+```
+
+In Docker Compose, you can mount the file:
+
+```yaml
+services:
+  app-elasticsearch:
+    volumes:
+      - ./custom-mappings.json:/app/mappings.json:ro
+    environment:
+      - STAC_FASTAPI_ES_MAPPINGS_FILE=/app/mappings.json
+```
+
+In Kubernetes, use a ConfigMap:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: stac-mappings
+data:
+  mappings.json: |
+    {
+      "properties": {
+        "properties": {
+          "properties": {
+            "platform": {"type": "keyword"},
+            "eo:cloud_cover": {"type": "float"}
+          }
+        }
+      }
+    }
+---
+apiVersion: apps/v1
+kind: Deployment
+spec:
+  template:
+    spec:
+      containers:
+      - name: stac-fastapi
+        env:
+        - name: STAC_FASTAPI_ES_MAPPINGS_FILE
+          value: /etc/stac/mappings.json
+        volumeMounts:
+        - name: mappings
+          mountPath: /etc/stac
+      volumes:
+      - name: mappings
+        configMap:
+          name: stac-mappings
+```
+
+> [!TIP]
+> If both `STAC_FASTAPI_ES_CUSTOM_MAPPINGS` and `STAC_FASTAPI_ES_MAPPINGS_FILE` are set, the environment variable takes precedence, allowing quick overrides during testing or troubleshooting.
+
 ### Dynamic Mapping Control (`STAC_FASTAPI_ES_DYNAMIC_MAPPING`)
 
 Controls how Elasticsearch/OpenSearch handles fields not defined in the mapping:

stac_fastapi/sfeos_helpers/stac_fastapi/sfeos_helpers/mappings.py

@@ -138,6 +138,18 @@ def get_items_mappings(
         if custom_mappings is not None
         else os.getenv("STAC_FASTAPI_ES_CUSTOM_MAPPINGS")
     )
+
+    if custom_config is None:
+        mappings_file = os.getenv("STAC_FASTAPI_ES_MAPPINGS_FILE")
+        if mappings_file:
+            try:
+                with open(mappings_file, "r") as f:
+                    custom_config = f.read()
+            except Exception as e:
+                logger.error(
+                    f"Failed to read STAC_FASTAPI_ES_MAPPINGS_FILE at {mappings_file}: {e}"
+                )
+
     apply_custom_mappings(mappings, custom_config)
 
     return mappings

stac_fastapi/tests/sfeos_helpers/test_mappings_file.py

@@ -0,0 +1,106 @@
+import json
+
+from stac_fastapi.sfeos_helpers.mappings import get_items_mappings
+
+
+class TestMappingsFile:
+    def test_mappings_file_applied(self, monkeypatch, tmp_path):
+        """Test that mappings are read from file when env var is set."""
+        custom_mappings = {
+            "properties": {"properties": {"file_field": {"type": "keyword"}}}
+        }
+        mappings_file = tmp_path / "mappings.json"
+        mappings_file.write_text(json.dumps(custom_mappings))
+
+        monkeypatch.setenv("STAC_FASTAPI_ES_MAPPINGS_FILE", str(mappings_file))
+        monkeypatch.delenv("STAC_FASTAPI_ES_CUSTOM_MAPPINGS", raising=False)
+
+        mappings = get_items_mappings()
+
+        assert mappings["properties"]["properties"]["file_field"] == {"type": "keyword"}
+
+    def test_env_var_precedence(self, monkeypatch, tmp_path):
+        """Test that STAC_FASTAPI_ES_CUSTOM_MAPPINGS takes precedence over file."""
+        file_mappings = {
+            "properties": {"properties": {"shared_field": {"type": "keyword"}}}
+        }
+        mappings_file = tmp_path / "mappings.json"
+        mappings_file.write_text(json.dumps(file_mappings))
+
+        env_mappings = {
+            "properties": {"properties": {"shared_field": {"type": "text"}}}
+        }
+
+        monkeypatch.setenv("STAC_FASTAPI_ES_MAPPINGS_FILE", str(mappings_file))
+        monkeypatch.setenv("STAC_FASTAPI_ES_CUSTOM_MAPPINGS", json.dumps(env_mappings))
+
+        mappings = get_items_mappings()
+
+        assert mappings["properties"]["properties"]["shared_field"] == {"type": "text"}
+
+    def test_missing_file_handled_gracefully(self, monkeypatch, caplog):
+        """Test that missing file is logged and ignored."""
+        monkeypatch.setenv("STAC_FASTAPI_ES_MAPPINGS_FILE", "/non/existent/file.json")
+        monkeypatch.delenv("STAC_FASTAPI_ES_CUSTOM_MAPPINGS", raising=False)
+
+        get_items_mappings()
+
+        assert "Failed to read STAC_FASTAPI_ES_MAPPINGS_FILE" in caplog.text
+
+    def test_invalid_json_in_file(self, monkeypatch, tmp_path, caplog):
+        """Test that invalid JSON in file is logged and ignored."""
+        mappings_file = tmp_path / "invalid.json"
+        mappings_file.write_text("{this is not valid json}")
+
+        monkeypatch.setenv("STAC_FASTAPI_ES_MAPPINGS_FILE", str(mappings_file))
+        monkeypatch.delenv("STAC_FASTAPI_ES_CUSTOM_MAPPINGS", raising=False)
+
+        get_items_mappings()
+
+        assert "Failed to parse STAC_FASTAPI_ES_CUSTOM_MAPPINGS JSON" in caplog.text
+
+    def test_file_and_env_var_both_set(self, monkeypatch, tmp_path):
+        """Test that env var completely overrides file when both are set."""
+        file_mappings = {
+            "properties": {
+                "properties": {
+                    "file_only_field": {"type": "keyword"},
+                    "shared_field": {"type": "text"},
+                }
+            }
+        }
+        mappings_file = tmp_path / "mappings.json"
+        mappings_file.write_text(json.dumps(file_mappings))
+
+        env_mappings = {
+            "properties": {
+                "properties": {
+                    "env_only_field": {"type": "keyword"},
+                    "shared_field": {"type": "integer"},
+                }
+            }
+        }
+
+        monkeypatch.setenv("STAC_FASTAPI_ES_MAPPINGS_FILE", str(mappings_file))
+        monkeypatch.setenv("STAC_FASTAPI_ES_CUSTOM_MAPPINGS", json.dumps(env_mappings))
+
+        mappings = get_items_mappings()
+
+        # Only env var fields should be present
+        assert "env_only_field" in mappings["properties"]["properties"]
+        assert "file_only_field" not in mappings["properties"]["properties"]
+        assert mappings["properties"]["properties"]["shared_field"] == {
+            "type": "integer"
+        }
+
+    def test_empty_file_handled_gracefully(self, monkeypatch, tmp_path):
+        """Test that empty file is handled without error."""
+        mappings_file = tmp_path / "empty.json"
+        mappings_file.write_text("")
+
+        monkeypatch.setenv("STAC_FASTAPI_ES_MAPPINGS_FILE", str(mappings_file))
+        monkeypatch.delenv("STAC_FASTAPI_ES_CUSTOM_MAPPINGS", raising=False)
+
+        # Should not raise, just use default mappings
+        mappings = get_items_mappings()
+        assert "properties" in mappings