General framework

m-mohr · m-mohr · commit b3a21af4ddf0 · 2024-08-12T11:21:56.000+02:00
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -27,6 +27,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- The extension is a framework for storage providers, it doesn't strictly define the individual providers.
 - The storage providers are grouped in `storage:schemes` and located in the Item Properties, Collections or Catalog metadata
 - Assets and Links reference the storage schemes by key in `storage:refs` 
 
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,33 @@
+# Contributing
+
+All contributions are subject to the
+[STAC Specification Code of Conduct](https://github.com/radiantearth/stac-spec/blob/master/CODE_OF_CONDUCT.md).
+For contributions, please follow the
+[STAC specification contributing guide](https://github.com/radiantearth/stac-spec/blob/master/CONTRIBUTING.md) Instructions
+for running tests are copied here for convenience.
+
+## Running tests
+
+The same checks that run as checks on PR's are part of the repository and can be run locally to verify that changes are valid.
+To run tests locally, you'll need `npm`, which is a standard part of any [node.js installation](https://nodejs.org/en/download/).
+
+First you'll need to install everything with npm once. Just navigate to the root of this repository and on
+your command line run:
+
+```bash
+npm install
+```
+
+Then to check markdown formatting and test the examples against the JSON schema, you can run:
+
+```bash
+npm test
+```
+
+This will spit out the same texts that you see online, and you can then go and fix your markdown or examples.
+
+If the tests reveal formatting problems with the examples, you can fix them with:
+
+```bash
+npm run format-examples
+```
diff --git a/README.md b/README.md
@@ -54,74 +54,47 @@ The fields in the table below can be used in these parts of STAC documents:
 
 | Field Name     | Type    | Description |
 | -------------- | ------- | ----------- |
-| platform       | string  | **REQUIRED.** The [cloud provider](#platforms) where data is stored. |
-| region         | string  | The region where the data is stored. Relevant to speed of access and inter region egress costs (as defined by PaaS provider) |
-| requester_pays | boolean | Is the data requester pays or is it data manager/cloud provider pays. Defaults to `false` |
-| tier           | string  | The title for the tier type (as defined by PaaS provider) |
+| platform       | string  | **REQUIRED.** The cloud provider where data is stored as URI or URI template to the API. |
+| region         | string  | The region where the data is stored. Relevant to speed of access and inter region egress costs (as defined by PaaS provider). |
+| requester_pays | boolean | Is the data "requester pays" (`true`) or is it "data manager/cloud provider pays" (`false`). Defaults to `false`. |
+| ...            | ...     | Additional properties as defined in the URL template or in the platform specific documents. |
 
-The properties `title` and `description` as defined in Common Metadata can be used as well.
+The properties `title` and `description` as defined in Common Metadata should be used as well.
 
-#### Platforms
+#### platform
 
-The `platform` field identifies the cloud provider where the data is stored.
+The `platform` field identifies the cloud provider where the data is stored as URI or URI template to the API of the service.
 
-There are a couple of pre-defined values for common providers:
+If a URI template is provided, all variables must be defined in the Storage Scheme Object as a property with the same name.
+For example, the URI template `https://{bucket}.{region}.example.com` must have at least the properties 
+`bucket` and `region` defined:
 
-- Alibaba Cloud (Aliyun): `ALIBABA`
-- Amazon AWS: `AWS`
-- Microsoft Azure: `AZURE`
-- Google Cloud Platform: `GCP`
-- IBM Cloud: `IBM`
-- Oracle Cloud: `ORACLE`
-
-All other PaaS solutions must use a unique URL to the service.
+```json
+{
+  "platform": "https://{bucket}.{region}.example.com",
+  "region": "eu-fr",
+  "bucket": "john-doe-stac",
+  "requester_pays": true
+}
+```
 
 In case an `href` contains a non-HTTP URL that is not directly resolvable,
 the `platform` property must identify the host so that the URL can be resolved without further information.
-This is especially useful to provide the endpoint URL for custom S3 providers.
-In this case the `platform` is effectively the endpoint URL.
-
-#### Tiers
-
-Recommended values for the `tier` field:
-
-| Minimum Duration | [Google Cloud Platform](https://cloud.google.com/storage/docs/storage-classes) | [Amazon AWS](https://aws.amazon.com/s3/storage-classes/) | [Microsoft Azure](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-storage-tiers) | [IBM Cloud](https://cloud.ibm.com/objectstorage/create#pricing)  | [Oracle Cloud](https://www.oracle.com/cloud/storage/pricing.html) | [Alibaba Cloud](https://www.alibabacloud.com/product/oss/pricing) |
-| ------------- | --------- | ------------------------ | ------- |----------  | ----------------- | ----------------- |
-| 0 (Auto-Tier) |           | Intelligent-Tiering      |         | Smart Tier |
-| 0 days        | STANDARD  | Standard                 | hot     | Standard   | Standard          | Standard          |
-| 30 days       | NEARLINE  | Standard-IA, One Zone-IA | cool    | Vault      | Infrequent Access | Infrequent Access |
-| 60 days       |           |                          |         |            |                   | Archive           |
-| 90 days       | COLDLINE  | Glacier                  |         | Cold Vault | Archive           | |
-| 180 days      |           | Glacier Deep Archive     | archive |            |                   | Cold Archive |
-| 365 days      | ARCHIVE   |                          |         |            |                   | |
-
-## Contributing
-
-All contributions are subject to the
-[STAC Specification Code of Conduct](https://github.com/radiantearth/stac-spec/blob/master/CODE_OF_CONDUCT.md).
-For contributions, please follow the
-[STAC specification contributing guide](https://github.com/radiantearth/stac-spec/blob/master/CONTRIBUTING.md) Instructions
-for running tests are copied here for convenience.
+For example, this is especially useful to provide the endpoint URL for custom S3 providers.
+In this case the `platform` could effectively provide the endpoint URL.
 
-### Running tests
+We try to collect pre-defined templates and specifications for as many providers as possible
+in this repository, but be aware that these are not part of the official extension releases
+and are not validated. This extension just provides the framework, the provider specifications
+may change at any time.
 
-The same checks that run as checks on PR's are part of the repository and can be run locally to verify that changes are valid. 
-To run tests locally, you'll need `npm`, which is a standard part of any [node.js installation](https://nodejs.org/en/download/).
+The following providers have pre-defined specifications at this point:
 
-First you'll need to install everything with npm once. Just navigate to the root of this repository and on 
-your command line run:
-```bash
-npm install
-```
+- [AWS S3](platforms/aws.md)
+- [Generic S3 (non-AWS)](platforms/s3.md)
 
-Then to check markdown formatting and test the examples against the JSON schema, you can run:
-```bash
-npm test
-```
+Feel encouraged to submit additional platform specifications via Pull Requests.
 
-This will spit out the same texts that you see online, and you can then go and fix your markdown or examples.
+## Contributing
 
-If the tests reveal formatting problems with the examples, you can fix them with:
-```bash
-npm run format-examples
-```
+See the [Contributor documentation](CONTRIBUTING.md) for details.
diff --git a/json-schema/schema.json b/json-schema/schema.json
@@ -101,24 +101,9 @@
               "properties": {
                 "platform": {
                   "title": "Platform",
-                  "oneOf": [
-                    {
-                      "type": "string",
-                      "enum": [
-                        "AWS",
-                        "GCP",
-                        "AZURE",
-                        "IBM",
-                        "ALIBABA",
-                        "ORACLE"
-                      ]
-                    },
-                    {
-                      "type": "string",
-                      "format": "iri",
-                      "pattern": "^[\\w\\+.-]+://"
-                    }
-                  ]
+                  "type": "string",
+                  "format": "uri-template",
+                  "pattern": "^[\\w\\+.-]+://"
                 },
                 "region": {
                   "title": "Region",
@@ -128,12 +113,9 @@
                   "type": "boolean",
                   "title": "Requester pays",
                   "default": false
-                },
-                "tier": {
-                  "title": "Tier",
-                  "type": "string"
                 }
-              }
+              },
+              "additionalProperties": true
             }
           },
           "additionalProperties": false
diff --git a/platforms/aws.md b/platforms/aws.md
@@ -0,0 +1,7 @@
+# AWS S3
+
+This defines the Amazon Web Services (AWS) S3 interface.
+
+- `platform`: `https://{bucket}.s3.{region}.amazonaws.com`
+- `bucket`: The bucket name
+- `region`: One of the S3 regions (lowercase)
diff --git a/platforms/s3.md b/platforms/s3.md
@@ -0,0 +1,5 @@
+# S3 (non-AWS)
+
+This defines the S3 interface for providers other than AWS (e.g. minio-based).
+
+- `platform`: The API URL, must be the endpoint URL that can be used for the AWS CLI for example.
