Skip to content

Commit d7f319c

Browse files
j08luej08lue
authored
Merge branch 'main' into fix/eoapi-api-docs
2 parents acd5011 + 7e3fc28 commit d7f319c

15 files changed

+389
-15
lines changed

‎.github/workflows/build-image.yaml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Build and publish Docker image
2+
on:
3+
push:
4+
branches: [ main ]
5+
tags:
6+
- "*"
7+
pull_request:
8+
branches: [ main ]
9+
jobs:
10+
push_to_registry:
11+
name: Build and push Docker image to Docker Hub
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Check out the repo
15+
uses: actions/checkout@v2
16+
- name: Login to DockerHub
17+
uses: docker/login-action@v1
18+
with:
19+
username: ${{ secrets.DOCKER_USERNAME }}
20+
password: ${{ secrets.DOCKER_PASSWORD }}
21+
- name: Push latest to Docker Hub
22+
uses: docker/build-push-action@v2
23+
with:
24+
tags: eoepca/stac-manager:latest
25+
push: true
26+
build-args: |
27+
PRIMARY_COLOR='#005ACD',SECONDARY_COLOR='#048A81',STAC_API_URL='https://eoapi.develop.eoepca.org/stac'
28+
- name: Push tagged to Docker Hub
29+
uses: docker/build-push-action@v2
30+
if: github.ref_type == 'tag'
31+
with:
32+
tags: eoepca/stac-manager:${{ github.ref_name }}
33+
push: true
34+
secrets: |
35+
id=keycloak_url,src=${{ secrets.KEYCLOAK_URL }}
36+
id=keycloak_realm,src=${{ secrets.KEYCLOAK_REALM }}
37+
id=keycloak_client_id,src=${{ secrets.KEYCLOAK_CLIENT_ID }}
38+
build-args: |
39+
PRIMARY_COLOR='#005ACD',SECONDARY_COLOR='#048A81',STAC_API_URL='https://eoapi.develop.eoepca.org/stac'

‎Dockerfile‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
FROM node:20-slim
2+
3+
RUN apt-get update && apt-get install -y git
4+
5+
WORKDIR /app
6+
7+
ARG PUBLIC_URL="/"
8+
ARG STAC_API_URL
9+
ARG PRIMARY_COLOR='#005ACD'
10+
ARG SECONDARY_COLOR='#048A81'
11+
12+
RUN git clone https://github.com/developmentseed/stac-manager .
13+
14+
ENV PUBLIC_URL=${PUBLIC_URL}
15+
ENV REACT_APP_STAC_API=${STAC_API_URL}
16+
ENV REACT_APP_THEME_PRIMARY_COLOR=${PRIMARY_COLOR}
17+
ENV REACT_APP_THEME_SECONDARY_COLOR=${SECONDARY_COLOR}
18+
19+
RUN npm i -g http-server
20+
RUN --mount=type=secret,id=keycloak_url \
21+
--mount=type=secret,id=keycloak_realm \
22+
--mount=type=secret,id=keycloak_client_id \
23+
export REACT_APP_KEYCLOAK_URL=$(cat /run/secrets/keycloak_url) && \
24+
export REACT_APP_KEYCLOAK_REALM=$(cat /run/secrets/keycloak_realm) && \
25+
export REACT_APP_KEYCLOAK_CLIENT_ID=$(cat /run/secrets/keycloak_client_id) && \
26+
npm i && npm run all:build
27+
28+
EXPOSE 80
29+
30+
ENTRYPOINT ["http-server"]
31+
CMD ["-p", "80", "packages/client/dist"]

‎docs/design/overview.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,15 +21,15 @@ As described in the [System Architecture document](https://eoepca.readthedocs.io
2121

2222
The components design of the Resource Discovery Building Block is shown in the following component figure.
2323

24-
The software pycsw is deployed as Resource Catalogue for all kinds of resources providing OGC API Records, STAC API (as well as OGC CSW, OpenSearch Geo/Time) and connection to the external automation & notification building block supporting the publish-subscribe (PubSub) mechanism.
24+
The software pycsw is deployed as Resource Catalogue for all kinds of resources providing OGC API - Records, STAC API (as well as OGC CSW, OpenSearch Geo/Time) and connection to the external automation & notification building block supporting the publish-subscribe (PubSub) mechanism.
2525

2626
The Resource Catalogue is focusing on collection metadata as well as resource types, such as workflows, algorithms, job results, services, web applications, Jupyter notebooks, and documentation.
2727

2828
eoAPI is used as Data Catalogue in addition to pycsw providing STAC API and connection to the external automation & notification building block supporting the PubSub mechanism.
2929

3030
Both components are deployed by default with PostgreSQL as a backend. In the roadmap of EOEPCA, support for Elasticsearch as a backend will be implemented. In this setup, collection metadata from the eoAPI-based data catalogue are registered into pycsw (e.g., using the PubSub mechanism) so that the resource catalogue provide a high-level overview of all resources. Operators of the platform may choose to implement one or both of the catalogue components, depending on their requirements.
3131

32-
The web-based administration interface is based on STAC Admin, EOxElements, and STAC Browser allowing for editing STAC and OGC API Records metadata amongst other functionalities.
32+
The web-based administration interface is based on STAC Admin, EOxElements, and STAC Browser allowing for editing STAC and OGC API - Records metadata amongst other functionalities.
3333

3434
All APIs are protected with the Tyk-based API Gateway, which connects to the Identity Management building block for authentication and authorization. In addition, Tyk is able to cache requests and conduct rate-limiting.
3535

‎docs/design/resource-admin-ui/design.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Design
2+
3+
The web-based administration user interface allows operators and users of a resource catalogue to
4+
maintain metadata and configuration.
5+
6+
Our vision is a unified interface for the platform operators where the different building block (BB) capabilities (in particular,
7+
Resource Discovery and Data Access) can be administrated seamlessly and thus is split between the different building blocks.
8+
9+
## STAC and OGC API - Records
10+
11+
The admin interface supports both [STAC API](https://stacspec.org/) and [OGC API - Records](https://ogcapi.ogc.org/records/). The two standards are converging and, with some small deviations,
12+
STAC is basically a rendtition of OGC API - Records with more required properties tailored to spatio-temporal data assets, while OGC API - Records
13+
is more generic and therefore a more natural choice for non-spatio-temporal data such as documents and workflow definitions.
14+
For a comprehensive comparison, see this [STAC Spec document](https://github.com/radiantearth/stac-api-spec/blob/bab9bdf22b54cd119e931859b0da13f42091abb8/PRINCIPLES.md).
15+
16+
The administration user interface will feature a branch between the two standards, giving users some context to make the choice between them,
17+
based on the type of asset they want to edit / add.
18+
19+
The below figures are preliminary design wireframes.
20+
21+
![Preliminary wireframe showing STAC collections](../../img/resource-admin-ui-wireframe-collections.png)
22+
23+
![Preliminary wireframe showing STAC collection overview](../../img/resource-admin-ui-wireframe-collection-overview.png)
24+
25+
26+
27+
## Version 1 - transactions and STAC extensions
28+
29+
In its first version, the interface will support transactional / atomic metadata changes, i.e. create, read, update, and delete (CRUD)
30+
operations, on individual metadata items (STAC / OGC API - Records collections or STAC items).
31+
32+
The web application makes use of the Transactions API of both eoAPI and pycsw, a REST API interface that is already available in both software packages.
33+
34+
All APIs are protected with the Tyk-based API Gateway, which connects to the Identity Management building block for authentication and authorization.
35+
36+
The user interface will support all required STAC fields (for now v1.0.0) and be extenable through React-based frontend code plugins to support additional
37+
metadata as well, such as fields added by STAC extensions. A priority case of extensibility to support is the [Render](https://github.com/stac-extensions/render) extension.
38+
39+
40+
![Service connectivity using Transactions](../../img/resource-admin-ui-transactions.png)
41+
42+
43+
## Version 2 - bulk ingestion and configuration
44+
45+
In its second version, the interface will gain support for mass import/export using external GeoParquet files etc. While numeric data file processing
46+
is left to file managers like the Workspace BB and dedicated data ingestion services like the [Resource Registration](https://eoepca.readthedocs.io/projects/resource-registration) building block.
47+
48+
Also CRUD operations will be routed through the [Resource Registration](https://eoepca.readthedocs.io/projects/resource-registration) building block.
49+
50+
![Service connectivity using Resource Registration](../../img/resource-admin-ui-registration.png)
51+
52+
Furthermore, features of resource collections should become configurable through the admin UI, such as (de-) activating data services for collections.

‎docs/design/resource-admin-ui/plugins.md‎

Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
# Plugins
2+
3+
The Resource Administration UI can be extended with a set of plugins that can be loaded to generate the editor.
4+
The below documentation describes the preliminary concepts and examples.
5+
6+
Each plugin would be responsible for a section of the editor and would be able to define the fields that should be shown.
7+
This allows for a more modular approach to the editor. Each instance could use a different set of plugins to define the editor structure.
8+
These could be custom plugins or plugins from the community.
9+
10+
Drawing inspiration from the JSON schema spec, each plugin would define a schema to create the editor.
11+
For each field type there would be a corresponding default widget to render it,
12+
but plugins could define their own widgets (in the form of a React component) to be used by the fields.
13+
14+
Below are some examples.
15+
16+
!!! NOTE
17+
The provided code examples are preliminary illustrations of the concepts and not necessarily complete.
18+
19+
## Simple example plugin
20+
21+
```javascript
22+
export class PluginMeta extends Plugin {
23+
editSchema() {
24+
return {
25+
type: 'root',
26+
properties: {
27+
title: {
28+
type: 'string'
29+
},
30+
description: {
31+
type: 'string'
32+
},
33+
provider: {
34+
type: 'string',
35+
'ui:widget': 'select',
36+
enum: [
37+
['ESA', 'esa'],
38+
['NASA', 'nasa'],
39+
['JAXA', 'jaxa'],
40+
['CNSA', 'cnsa']
41+
]
42+
}
43+
}
44+
};
45+
}
46+
47+
enterData(data) {
48+
// Transform the input data to the format expected by the editor.
49+
// A transformation may be needed in cases where an object key is being edited.
50+
}
51+
52+
exitData(data) {
53+
// Transform the data from the editor to the format expected by STAC.
54+
}
55+
}
56+
```
57+
58+
## More complicated example
59+
60+
Something more complicated would include an async init function to get values needed for the plugin.
61+
62+
```javascript
63+
export class PluginRender extends Plugin {
64+
async init(data) {
65+
this.colorMaps = [];
66+
67+
try {
68+
const response = await fetch(
69+
'https://dev.openveda.cloud/api/raster/colorMaps'
70+
);
71+
const result = await response.json();
72+
this.colorMaps = result.colorMaps;
73+
} catch (error) {
74+
// oops!
75+
}
76+
}
77+
78+
// ... more code
79+
}
80+
```
81+
82+
## Configuration
83+
84+
All this would be set up in a configuration file that would define the plugins to be used by the editor.
85+
The plugins could even be loaded dynamically depending on some condition.
86+
87+
```javascript
88+
export const config = {
89+
// Collection level plugins.
90+
collectionPlugins: [
91+
new PluginMeta()
92+
],
93+
94+
// Item level plugins.
95+
itemPlugins: [
96+
new PluginMeta(),
97+
new PluginExtension(),
98+
(data) => data.stac_extensions?.some((e) => e.includes('/render/'))
99+
? new PluginRender()
100+
: null
101+
],
102+
103+
// Custom widgets.
104+
'widgets': {
105+
keyname: KeynameWidget,
106+
select: SelectWidget
107+
}
108+
};
109+
```
110+
111+
## Widgets
112+
113+
```javascript
114+
export class PluginRender extends Plugin {
115+
editSchema() {
116+
return {
117+
type: 'root',
118+
properties: {
119+
renders: {
120+
type: 'array',
121+
items: {
122+
type: 'object',
123+
'ui:widget': 'renderExtensionItemWidget',
124+
properties: {
125+
key: {
126+
label: 'Key name',
127+
'ui:widget': 'keyname',
128+
type: 'string'
129+
},
130+
assets: {
131+
label: 'Assets',
132+
type: 'array',
133+
items: {
134+
type: 'string'
135+
}
136+
},
137+
nodata: {
138+
label: 'No Data',
139+
type: 'string'
140+
},
141+
colormap_name: {
142+
label: 'Colormap Name',
143+
type: 'string',
144+
}
145+
// ... more fields
146+
}
147+
}
148+
}
149+
}
150+
};
151+
}
152+
}
153+
```
154+
155+
Then the Widget:
156+
157+
```javascript
158+
export function RenderExtensionItemWidget(props) {
159+
const [tilejson, setTilejson] = useState();
160+
return (
161+
<Box>
162+
<Box position='relative' aspectRatio='4/2'>
163+
<RMap
164+
mapboxAccessToken={process.env.MAPBOX_TOKEN}
165+
mapStyle='mapbox://styles/mapbox/satellite-v9'
166+
>
167+
{tilejson && (
168+
<Source tiles={tilejson.tiles} type='raster'>
169+
<Layer id='data' type='raster' />
170+
</Source>
171+
)}
172+
</RMap>
173+
</Box>
174+
<Flex gap={4} direction='column' mt={4}>
175+
<ObjectField pointer={props.pointer} field={props.field} />
176+
</Flex>
177+
</Box>
178+
);
179+
}
180+
```
181+
182+
`ObjectField` is the default widget for objects so basically we're adding a wrapper around it to show the map.
183+
184+
Lastly, the configuration file would be used to generate the editor:
185+
186+
```javascript
187+
export const config = {
188+
collectionPlugins: [
189+
new PluginMeta(),
190+
new PluginRender()
191+
],
192+
193+
'ui:widget': {
194+
renderExtensionItemWidget: RenderExtensionItemWidget
195+
}
196+
};
197+
```

‎docs/design/resource-and-data-catalogue/api/endpoint-specification.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Specification
22

3+
# OpenAPI/Swagger description
4+
<swagger-ui src="https://resource-catalogue.develop.eoepca.org/openapi"/>
5+
36
# Endpoints
47

58
## OGC API - Records

‎docs/design/resource-and-data-catalogue/api/usage.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,8 @@ records will be specified in additional parts.
2626

2727
> Versions
2828
29-
**OGC API - Records - Part 1: Core** is currently in draft and is planned to be
30-
submitted to the OGC Architecture Board (OAB) in December 2023 for public review
31-
in Q1 2024.
29+
**OGC API - Records - Part 1: Core** has been submitted to the OGC Architecture Board (OAB)
30+
and has completed public review stage. It is expected to be finalized in Q4 2024.
3231

3332
> Test suite
3433
@@ -326,6 +325,10 @@ queryables. An example query based on a "search engine" style search using the
326325
Given OGC API - Records uses OGC API - Common and OGC API - Features as building blocks, please see the [OGC API - Features](https://ogcapi-workshop.ogc.org/api-deep-dive/features/#feature) deep dive
327326
for a detailed explanation.
328327

328+
### GeoJSON
329+
330+
The OGC API - Records GeoJSON Requirements Class specifies a GeoJSON based encoding for Record core, based on RFC7946. Given the ubiquity of GeoJSON, numerous tools exist to validate process and decode/encode GeoJSON, making OGC API - Records GeoJSON easy to include in metadata processing pipelines. OGC API - Records includes the [JSON Schema](https://github.com/opengeospatial/ogcapi-records/blob/master/core/openapi/schemas/recordGeoJSON.yaml) for the GeoJSON representation and thus can be used for runtime or offline validation of metadata payloads. Applications based on OGC API - Reocrds GeoJSON can extend and constrain the schema accordingly for domain specific workflows.
331+
329332
## Summary
330333

331334
OGC API - Records provides functionality for working with metadata data on the Web. This deep dive provided an overview of the standard and the various Resources and endpoints that are supported.

‎docs/img/resource-admin-ui-registration.png‎

36.6 KB
Loading

‎docs/img/resource-admin-ui-transactions.png‎

23.9 KB
Loading

‎docs/img/resource-admin-ui-wireframe-collection-overview.png‎

60.4 KB
Loading

0 commit comments

Comments
 (0)