Skip to content

Commit 1cfe56a

Browse files
Teddy-1000lukas-phaf
Teddy-1000
and
lukas-phaf
authored
Add option for custom openapi metadata and dataset template file (#277)
* Update readme with new env variables * Add configurable openapi metadata and j2 dataset metadata template * Explain how and where to mount new files * Move default openapi_metadata to a file, and just load the file as default * Explain how and where to mount new files * Add absolute path as default * Update API README * Fix formatting * Update default j2 dataset template * Fix line length * Update api/templates/dataset_metadata_template.j2 Co-authored-by: Lukas Phaf <lukas.phaf@knmi.nl> * Update api/templates/dataset_metadata_template.j2 Co-authored-by: Lukas Phaf <lukas.phaf@knmi.nl> * Remove print * Clearify that the openapi metadata is also used for the landing page * Use fastAPI openapi wrapper to mitigate passing arbitrary arguemnts to fastAPI constructor * Update readme about custom openapi contents * Revert "Use fastAPI openapi wrapper to mitigate passing arbitrary arguemnts to fastAPI constructor" This reverts commit dba41cb. * Make list of acceptable parameters to take from openapi_metadata.json --------- Co-authored-by: Lukas Phaf <lukas.phaf@knmi.nl>
1 parent d6cca8d commit 1cfe56a

File tree

5 files changed

+198
-121
lines changed

5 files changed

+198
-121
lines changed

‎api/README.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,36 @@ Environment variables that can be used to configure the container or the environ
1212
| GUNICORN_CMD_ARGS | Command-line arguments for configuring Gunicorn, a Python WSGI HTTP Server. | ☐ |
1313
| CORS_ORIGINS | Indicates whether the response can be shared with requesting code from the given origins (passed as a comma separated string) | ☐ |
1414
| CORS_HEADERS | Indicates what headers should be supported with cross-origin requests (passed as a comma separated string) | ☐ |
15+
| JINJA2_TEMPLATES | Path to a folder with jinja2 templates to override the default templates used by the API. See template section for details | ☐ |
16+
| OPENAPI_METADATA_PATH | Path to an alternative OpenAPI metadata json file. It need to have the same fields as the openapi/openapi_metadata.py file. | ☐ |
17+
18+
## OpenAPI and EDR landing page metadata
19+
20+
To load your own metadata file, mount your new openapi metadata json file, with the same structure and key as the default one, found in openapi_default_files, and point the environment variable `OPENAPI_METADATA_PATH` to the new file. If you mount your folder to the same as the default one, make sure to not overwrite files you have not replaced.
21+
22+
This Metdata is used both for the OpenAPI specification and the EDR landing page.
23+
24+
The available fields are: title, version, summary, description, terms_of_service, contact, license_info, tags, external_docs.
25+
26+
## JINJA2_TEMPLATES
27+
28+
The jinja2 folder must container the following files:
29+
30+
- dataset_metadata_template.j2: this is the metadata template for the observation collection.
31+
32+
### dataset_metadata_template.j2
33+
34+
The current jinja2 filters will be replaced. It's important to use the json j2 filter when inserting these strings.
35+
36+
- spatial_extent
37+
- temporal_extent
38+
- url_base
39+
- url_conformance
40+
- url_docs
41+
42+
All url fields are dynamically generated based on the request url base.
43+
44+
To load a custom template, mount a folder with your new template in to the container and set the `JINJA2_TEMPLATES` environment variable to point your new folder.
1545

1646
## Prerequisites of running locally
1747

‎api/openapi/openapi_metadata.py‎

Lines changed: 21 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,21 @@
1-
openapi_metadata = {
2-
"title": "EDR Observations API Europe EUMETNET",
3-
"description": (
4-
"OGC EDR API data service for European meteorological observations from EUMETNET,"
5-
" co-funded by the European Union."
6-
),
7-
"contact": {
8-
"name": "EUMETNET",
9-
"url": "https://www.eumetnet.eu/about-us/",
10-
"email": "eucos@metoffice.gov.uk",
11-
},
12-
"license_info": {
13-
"name": "CC-BY-4.0",
14-
"url": "https://creativecommons.org/licenses/by/4.0/",
15-
},
16-
}
1+
import os
2+
import json
3+
4+
with open(
5+
os.getenv("OPENAPI_METADATA_PATH", "/app/openapi_default_files/openapi_metadata.json"),
6+
"r",
7+
) as file:
8+
openapi_metadata = json.load(file)
9+
valid_keys = [
10+
"title",
11+
"version",
12+
"summary",
13+
"description",
14+
"terms_of_service",
15+
"contact",
16+
"license_info",
17+
"openapi_tags",
18+
]
19+
unwanted = set(openapi_metadata) - set(valid_keys)
20+
for unwanted_key in unwanted:
21+
del openapi_metadata[unwanted_key]

‎api/openapi_default_files/openapi_metadata.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"title": "EDR Observations API Europe EUMETNET",
3+
"description": "OGC EDR API data service for European meteorological observations from EUMETNET, co-funded by the European Union.",
4+
"contact": {
5+
"name": "EUMETNET",
6+
"url": "https://www.eumetnet.eu/about-us/",
7+
"email": "eucos@metoffice.gov.uk"
8+
},
9+
"license_info": {
10+
"name": "CC-BY-4.0",
11+
"url": "https://creativecommons.org/licenses/by/4.0/"
12+
}
13+
}

‎api/routers/feature.py‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import json
2+
import os
23
from typing import Annotated
34

45
import datastore_pb2 as dstore
@@ -23,7 +24,7 @@
2324

2425
router = APIRouter(prefix="/collections/observations")
2526

26-
env = Environment(loader=FileSystemLoader("templates"), autoescape=select_autoescape())
27+
env = Environment(loader=FileSystemLoader(os.getenv("JINJA2_TEMPLATES", "templates")), autoescape=select_autoescape())
2728

2829

2930
@router.get(
@@ -174,10 +175,7 @@ async def get_dataset_metadata(request: Request):
174175
]
175176
],
176177
"temporal_extents": [
177-
[
178-
f"{extent.temporal_extent.start.ToDatetime().strftime('%Y-%m-%dT%H:%M:%SZ')}",
179-
f"{extent.temporal_extent.end.ToDatetime().strftime('%Y-%m-%dT%H:%M:%SZ')}",
180-
],
178+
[f"{extent.temporal_extent.start.ToDatetime().strftime('%Y-%m-%dT%H:%M:%SZ')}", ".."],
181179
],
182180
"url_base": base_url,
183181
"url_conformance": base_url + "conformance",

‎api/templates/dataset_metadata_template.j2‎

Lines changed: 131 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
{
2-
"id": "urn:wmo:md:eu-eumetnet-observations:swob-realtime",
2+
"id": "urn:wmo:md:eu-eumetnet-surface-observations:land-station-observations",
33
"conformsTo": [
4-
"http://wis.wmo.int/spec/wcmp/2/conf/core"
5-
],
6-
"type": "Feature",
4+
"http://wis.wmo.int/spec/wcmp/2/conf/core"
5+
],
6+
"type": "Feature",
77
"geometry": {
88
"type": "Polygon",
99
"coordinates": {{ spatial_extents }}
@@ -12,105 +12,136 @@
1212
"interval": {{ temporal_extents|tojson }},
1313
"resolution": "PT10M"
1414
},
15-
"properties": {
16-
"title": "Land surface weather observations",
17-
"description": "Land surface observations measured at automatic and manual stations of EUMETNET Members (last 24 hours)",
18-
"themes": [
19-
{
20-
"concepts": [
21-
{
22-
"id": "weather"
23-
}
24-
],
25-
"scheme": "https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline"
26-
},
27-
{
28-
"concepts": [
29-
{
30-
"id": "surface-based-observations"
31-
}
32-
],
33-
"scheme": "https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/_weather"
34-
}
15+
"properties": {
16+
"title": "Land surface weather observations",
17+
"description":"Land surface observations measured at automatic and manual weather stations of EUMETNET Members and their trusted partners (last 24 hours only)",
18+
"contacts": [
19+
{
20+
"name": "Met Norway",
21+
"organization": "National Meteorological service of Norway, Met Norway",
22+
"phones": [
23+
{
24+
"value": "+4722963000"
25+
}
3526
],
36-
"language": "en",
37-
"type": "dataset",
38-
"created": "2023-01-01T00:00:00Z",
39-
"updated": "2024-09-19T00:00:00Z",
40-
"contacts": [
41-
{
42-
"organization": "EUMETNET",
43-
"addresses": [
44-
{
45-
"deliveryPoint": [
46-
"Avenue Circulaire 3"
47-
],
48-
"city": "Bruxelles",
49-
"postalCode": "1180",
50-
"country": "Belgique"
51-
}
52-
],
53-
"links": [
54-
{
55-
"href": "https://www.eumetnet.eu/about-us/",
56-
"rel": "about",
57-
"type": "text/html"
58-
}
59-
],
60-
"roles": [
61-
"host"
62-
]
63-
}
27+
"emails": [
28+
{
29+
"value": "post@met.no"
30+
}
6431
],
65-
"keywords": [
66-
"surface weather",
67-
"observations",
68-
"meteorology"
32+
"addresses": [
33+
{
34+
"deliveryPoint": [
35+
"https://www.met.no/en/contact-us"
36+
],
37+
"name": "Met Norway",
38+
"city": "Oslo",
39+
"postalCode": "0313",
40+
"country": "Norway"
41+
}
6942
],
70-
"wmo:dataPolicy": "core"
71-
},
72-
"links": [
73-
{
74-
"href": "E-SOH dataset mqtt stream",
75-
"rel": "items",
76-
"title": "E-SOH dataset data notifications",
77-
"type": "application/json"
78-
},
79-
{
80-
"href": "E-SOH time series mqtt stream",
81-
"rel": "items",
82-
"title": "E-SOH time series data notifications",
83-
"type": "application/json"
84-
},
85-
{
86-
"href": {{ url_base|tojson }},
87-
"rel": "data",
88-
"title": "E-SOH EDR API landing page",
89-
"type": "application/json"
90-
},
91-
{
92-
"href": {{ url_docs|tojson }},
93-
"rel": "related",
94-
"title": "E-SOH API documentation",
95-
"type": "application/json"
96-
},
97-
{
98-
"href": {{ url_conformance|tojson }},
99-
"rel": "conformance",
100-
"title": "E-SOH Conformance Declaration",
101-
"type": "application/json"
102-
},
103-
{
104-
"href": "https://www.eumetnet.eu/wp-content/uploads/2018/03/List-of-current-Members-as-pdf.pdf",
43+
"links": [
44+
{
10545
"rel": "about",
106-
"title": "EUMETNET Members",
107-
"type": "text/html"
46+
"type": "text/html",
47+
"href": "https://www.eumetnet.eu/about-us"
48+
}
49+
],
50+
"roles": ["host"]
51+
}
52+
],
53+
"keywords": [
54+
"weather",
55+
"surface-based observations",
56+
"observations",
57+
"meteorology",
58+
"surface weather",
59+
"Norway",
60+
"Finland",
61+
"The Netherlands"
62+
],
63+
"themes": [
64+
{ "concepts": [
65+
{
66+
"id":"weather",
67+
"title": "Weather",
68+
"url": "https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/weather"
69+
}
70+
],
71+
"scheme":"https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline"
72+
},
73+
{ "concepts": [
74+
{
75+
"id":"surface-based-observations"
76+
}
77+
],
78+
"scheme":"https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/weather/surface-based-observations"
10879
},
109-
{
110-
"href": "https://creativecommons.org/licenses/by/4.0/",
111-
"rel": "license",
112-
"title": "Creative Commons BY 4.0 licence",
113-
"type": "text/html"
114-
}
115-
]
80+
{
81+
"concepts": [
82+
{
83+
"id": "air_temperature",
84+
"title": "Air temperature",
85+
"url": "http://vocab.nerc.ac.uk/standard_name/air_temperature/"
86+
},
87+
{
88+
"id": "wind_speed",
89+
"title": "Wind Speed",
90+
"url": "http://vocab.nerc.ac.uk/standard_name/wind_speed/"
91+
},
92+
{
93+
"id": "wind_to_direction",
94+
"title": "Wind to diection",
95+
"url": "http://vocab.nerc.ac.uk/standard_name/wind_to_direction/"
96+
}
97+
],
98+
"scheme": "https://vocab.nerc.ac.uk/standard_name"
99+
}
100+
],
101+
"language":"en",
102+
"created": "2025-06-04T14:00:00Z",
103+
"updated": "2025-06-04T14:00:00Z",
104+
"rights": "Users are granted free and unrestricted access to this data, without charge and with no conditions on use. Users are requested to attribute the producer of this data. WMO Unified Data Policy (Resolution 1 (Cg-Ext 2021))",
105+
"licence": "CC BY 4.0 Creative Commons license",
106+
"type": "dataset",
107+
"wmo:dataPolicy": "recommended"
108+
},
109+
"links": [
110+
{
111+
"href": {{ url_base|tojson }},
112+
"rel": "data",
113+
"title": "E-SOH EDR API landing page",
114+
"type": "application/json"
115+
},
116+
{
117+
"href": {{ url_docs|tojson }},
118+
"rel": "related",
119+
"title": "E-SOH API documentation",
120+
"type": "application/json"
121+
},
122+
{
123+
"href": {{ url_conformance|tojson }},
124+
"rel": "conformance",
125+
"title": "E-SOH Conformance Declaration",
126+
"type": "application/json"
127+
},
128+
{
129+
"rel": "related",
130+
"href": "https://www.eumetnet.eu/observations/observations-data-sharing-2/",
131+
"type": "text/html",
132+
"title": "Documentation related to EUMETNET data sharing"
133+
},
134+
{
135+
"rel": "related",
136+
"href": "https://www.eumetnet.eu/about-us-2/members-partners/",
137+
"type": "text/html",
138+
"title": "List of EUMETNET Members"
139+
},
140+
{
141+
"rel": "license",
142+
"href": "https://creativecommons.org/licenses/by/4.0/",
143+
"title": "Creative Commons BY 4.0 licence",
144+
"type": "text/html"
145+
}
146+
]
116147
}

0 commit comments

Comments
 (0)