Skip to content
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 63 additions & 54 deletions extensions/federation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,71 +106,25 @@ schema:
}
```

## Lists of resources

Clients will assume that all lists of resources are the a combination of all back-ends listed in `GET /`.
Federated APIs can expose if any of the back-ends is not available and thus is not part of the response.

Applies to:

- `GET /collections`
- `GET /processes`
- `GET /file_formats`
- `GET /process_graphs`
- `GET /files`
- `GET /jobs`
- `GET /jobs/{job_id}`
- `GET /jobs/{job_id}/results`
- `GET /jobs/{job_id}/logs`
- `GET /services`

The following endpoints define the resources (UDF runtimes / service types) at the top level of their response as key-value pairs.
Consequently, they are not extensible with additional properties for federation purposes.

- `GET /udf_runtimes`
- `GET /service_types`

### OpenAPI fragment

```yaml
schema:
type: object
properties:
'federation:missing':
description: >-
Lists all back-ends that were not considered in the response (e.g. because they were not accessible).
If not given or empty, all back-ends were considered for creating the response.
Back-ends that were listed as offline in the capabilities still need to be listed here.
type: array
items:
type: string
description: The ID of a back-end.
```

### Example

```json
{
"federation:missing": ["wwu"],
"collections": [...],
"links": [...]
}
```

## Resources supported only by a subset of back-ends

Every discoverable resource that is defined as an object and allows to contain additional properties, can list the subset of back-ends that support or host the exposed resource/functionality.
Every discoverable resource that is defined as an object and allows to contain additional properties, can list the subset of back-ends that permanently support or host the exposed resource/functionality.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... back-ends that permanently support ...

from the standpoint of the aggregator or another federation component, I find it hard to implement this label "permanent".

I understand that you want to ignore temporary unavailability or other unintended glitches, but it still seems to imply that there is some kind of contract (outside the scope of the basic openEO API between the aggregator and each backend that expresses some kind of SLA about the resources.

I'm fine with this contract being out-of-band of this specification, but that means that the value of the "permanent" label is very fuzzy and might cause confusion.

Copy link
Member Author

@m-mohr m-mohr May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Permanently means that it's listed in the endpoints list or is returned by another list that's relevant in the context of the endpoint (e.g. the collection list).

Given the clarification in #564 (comment), maybe we can write something that doesn't depend on "temporary"/"permanently"?

I don't really think about much magic here ;-)

Examples of where this could apply to (**not** comprehensive):

- `GET /collctions`
- `GET /collections/{id}`
- `GET /processes` (per process, per parameter)
- `GET /file_formats` (per file format)
- `GET /processes` (global, per process, per parameter)
- `GET /file_formats` (global, per file format)
- `GET /service_types` (per service)
- `GET /udf_runtimes` (per UDF runtime, per version)
- `POST /validation` (the back-ends that can run the process, see below)
- `GET /files`
- `GET /process_graphs`
- `GET /process_graphs/{id}`
- `GET /jobs`
- `GET /jobs/{job_id}` (the back-ends that generated the result)
- `GET /jobs/{job_id}/results` (the back-ends that generated the result)
- `GET /services`
- `GET /services/{id}` (the back-ends that host the service)

This can also be embedded deeply into a hierarchical structure, e.g. for process or file format parameters.
Expand Down Expand Up @@ -240,3 +194,58 @@ This also covers the case where the federation supports splitting a process into
...
}
```

## Temporarily or unintentionally unavailable resources

Resources and back-ends can be temporarily or unintentionally unavailable.
It is especially important to communicate to users missing resources when compiling lists of resources across multiple back-ends.
Clients will assume that all lists of resources are the a combination of all back-ends listed in `GET /`.
Federated APIs can expose if any of the back-ends is temporarily not available and thus is not part of the response.

Examples of where this could apply to (**not** comprehensive):

- `GET /collections`
- `GET /processes`
- `GET /file_formats`
- `GET /process_graphs`
- `GET /files`
- `GET /jobs`
- `GET /jobs/{job_id}`
- `GET /jobs/{job_id}/results`
- `GET /jobs/{job_id}/logs`
- `GET /services`

### OpenAPI fragment

```yaml
schema:
type: object
properties:
'federation:missing':
description: >-
Lists all back-ends that were temporarily not considered in the response (e.g. because they were not accessible).
If not given or empty, all back-ends supporting this endpoint were considered for creating the response.
Back-ends that were listed as offline in the capabilities still need to be listed here.
type: array
items:
type: string
description: The ID of a back-end.
```

### Example

```json
{
"federation:missing": ["wwu"],
"collections": [...],
"links": [...]
}
```

## Endpoints that can't list federation details

The following endpoints define the resources (UDF runtimes / service types) at the top level of their response as key-value pairs.
Consequently, they are not extensible with additional properties for federation purposes.

- `GET /udf_runtimes`
- `GET /service_types`