Skip to content

Commit 3ed3c6e

Browse files
m-mohrm-mohr
committed
Propose version 2 of this extension. Fixes #16, #21, #22, #23 
1 parent 2948d11 commit 3ed3c6e

File tree

9 files changed

+368
-152
lines changed

9 files changed

+368
-152
lines changed

‎.github/workflows/publish.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@ jobs:
77
runs-on: ubuntu-latest
88
steps:
99
- name: Inject env variables
10-
uses: rlespinasse/github-slug-action@v3.x
11-
- uses: actions/checkout@v2
10+
uses: rlespinasse/github-slug-action@v4
11+
- uses: actions/checkout@v4
1212
- name: deploy JSON Schema for version ${{ env.GITHUB_REF_SLUG }}
13-
uses: peaceiris/actions-gh-pages@v3
13+
uses: peaceiris/actions-gh-pages@v4
1414
with:
1515
github_token: ${{ secrets.GITHUB_TOKEN }}
1616
publish_dir: json-schema

‎.github/workflows/test.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ jobs:
44
deploy:
55
runs-on: ubuntu-latest
66
steps:
7-
- uses: actions/setup-node@v2
7+
- uses: actions/setup-node@v4
88
with:
99
node-version: 'lts/*'
10-
- uses: actions/checkout@v2
10+
- uses: actions/checkout@v4
1111
- run: |
1212
npm install
1313
npm test

‎CHANGELOG.md‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1616

1717
### Fixed
1818

19+
## [v2.0.0] - 2021-06-23
20+
21+
### Added
22+
23+
- `storage:schemes`, `storage:refs` and Storage Scheme Object
24+
- Support the storage extension in Links
25+
- Support for the Alternate Assets Extension
26+
- Support for other storage providers, including custom S3 hosts
27+
28+
### Changed
29+
30+
- The storage providers are grouped in `storage:schemes` and located in the Item Properties, Collections or Catalog metadata
31+
- Assets and Links reference the storage schemes by key in `storage:refs`
32+
33+
### Removed
34+
35+
- `storage:platform`, `storage:region`, `storage:requester_pays` and `storage:tier`
36+
1937
## [v1.0.0] - 2021-06-23
2038

2139
Initial release
2240

23-
[Unreleased]: <https://github.com/stac-extensions/storage/compare/v1.0.0...HEAD>
41+
[Unreleased]: <https://github.com/stac-extensions/storage/compare/v2.0.0...HEAD>
42+
[v2.0.0]: <https://github.com/stac-extensions/storage/compare/v1.0.0...v2.0.0>
2443
[v1.0.0]: <https://github.com/stac-extensions/storage/tree/v1.0.0>

‎README.md‎

Lines changed: 49 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,58 +1,86 @@
11
# Storage Extension Specification
22

33
- **Title:** Storage
4-
- **Identifier:** <https://stac-extensions.github.io/storage/v1.0.0/schema.json>
4+
- **Identifier:** <https://stac-extensions.github.io/storage/v2.0.0/schema.json>
55
- **Field Name Prefix:** storage
6-
- **Scope:** Item, Collection
6+
- **Scope:** Item, Catalog, Collection
77
- **Extension [Maturity Classification](https://github.com/radiantearth/stac-spec/tree/master/extensions/README.md#extension-maturity):** Pilot
88
- **Owner**: @davidraleigh @matthewhanson
99

1010
This document explains the Storage Extension to the [SpatioTemporal Asset Catalog](https://github.com/radiantearth/stac-spec) (STAC) specification.
11-
It allows adding details related to cloud storage access and costs to be associated with STAC Assets.
11+
It allows adding details related to cloud object storage access and costs to be associated with STAC Assets.
1212
This extension does not cover NFS solutions provided by PaaS cloud companies.
1313

1414
- Examples:
15-
- [Item example 1](examples/item-naip.json): Shows the basic usage of the extension in a STAC Item.
16-
- [Item example 2](examples/item-nsl.json): Another example of basic usage.
15+
- [NAIP Item](examples/item-naip.json): Shows the usage of the extension in combination with the alternate asset extension.
16+
- [NSL Item](examples/item-nsl.json): Shows a mixture of storage providers, including custom S3 hosts.
17+
- [Catalog with Link](examples/catalog-link.json): Shows the usage of the extension on a link in a STAC Catalog.
1718
- [JSON Schema](json-schema/schema.json)
1819
- [Changelog](./CHANGELOG.md)
1920

2021
## Fields
2122

2223
The fields in the table below can be used in these parts of STAC documents:
2324

25+
- [x] Catalogs
26+
- [x] Collections
27+
- [x] Item Properties (incl. Summaries in Collections)
28+
- [ ] Assets (for both Collections and Items, incl. Item Asset Definitions in Collections)
29+
- [ ] Links
30+
31+
| Field Name | Type | Description |
32+
| ----------------- | ------------------------------------------------------------ | ----------- |
33+
| `storage:schemes` | Map<string, [Storage Scheme Object](#storage-scheme-object)> | **REQUIRED.** A property that contains all of the storage schemes used by Assets and Links in the STAC Item, Catalog or Collection. |
34+
35+
---
36+
37+
The fields in the table below can be used in these parts of STAC documents:
38+
2439
- [ ] Catalogs
2540
- [ ] Collections
26-
- [x] Item Properties (incl. Summaries in Collections)
41+
- [ ] Item Properties (incl. Summaries in Collections)
2742
- [x] Assets (for both Collections and Items, incl. Item Asset Definitions in Collections)
28-
- [ ] Links
43+
- [x] Links
44+
- [x] [Alternate Assets Object](https://github.com/stac-extensions/alternate-assets?tab=readme-ov-file#alternate-asset-object)
45+
46+
| Field Name | Type | Description |
47+
| -------------- | ---------- | ----------- |
48+
| `storage:refs` | \[string\] | A property that specifies which schemes in `storage:schemes` may be used to access an Asset or Link. Each value must be one of the keys defined in `storage:schemes`. |
49+
50+
### Storage Scheme Object
2951

30-
| Field Name | Type | Description |
31-
| ---------------------- | --------- | ----------- |
32-
| storage:platform | string | The [cloud provider](#providers) where data is stored |
33-
| storage:region | string | The region where the data is stored. Relevant to speed of access and inter region egress costs (as defined by PaaS provider) |
34-
| storage:requester_pays | boolean | Is the data requester pays or is it data manager/cloud provider pays. *Defaults to false* |
35-
| storage:tier | string | The title for the tier type (as defined by PaaS provider) |
52+
| Field Name | Type | Description |
53+
| -------------- | ------- | ----------- |
54+
| platform | string | **REQUIRED.** The [cloud provider](#platforms) where data is stored. |
55+
| region | string | The region where the data is stored. Relevant to speed of access and inter region egress costs (as defined by PaaS provider) |
56+
| requester_pays | boolean | Is the data requester pays or is it data manager/cloud provider pays. Defaults to `false` |
57+
| tier | string | The title for the tier type (as defined by PaaS provider) |
3658

37-
While these are all valid properties on an Item, they will typically be defined per-asset. If a field applies equally
38-
to all assets (e.g., storage:platform=AWS if all assets are on AWS), then it should be specified in Item properties.
59+
The properties `title` and `description` as defined in Common Metadata can be used as well.
3960

40-
### Additional Field Information
61+
#### Platforms
4162

42-
#### Providers
43-
Currently this document is arranged to support object storage users of the following PaaS solutions:
63+
The `platform` field identifies the cloud provider where the data is stored.
64+
65+
There are a couple of pre-defined values for common providers:
4466

4567
- Alibaba Cloud (Aliyun): `ALIBABA`
4668
- Amazon AWS: `AWS`
4769
- Microsoft Azure: `AZURE`
4870
- Google Cloud Platform: `GCP`
4971
- IBM Cloud: `IBM`
5072
- Oracle Cloud: `ORACLE`
51-
- All other PaaS solutions: `OTHER`
5273

53-
The upper-cased values are meant to be used for `storage:platform`.
74+
All other PaaS solutions must use a unique URL to the service.
75+
76+
In case an `href` contains a non-HTTP URL that is not directly resolvable,
77+
the `platform` property must identify the host so that the URL can be resolved without further information.
78+
This is especially useful to provide the endpoint URL for custom S3 providers.
79+
In this case the `platform` is effectively the endpoint URL.
80+
81+
#### Tiers
5482

55-
#### Cloud Provider Storage Tiers
83+
Recommended values for the `tier` field:
5684

5785
| Minimum Duration | [Google Cloud Platform](https://cloud.google.com/storage/docs/storage-classes) | [Amazon AWS](https://aws.amazon.com/s3/storage-classes/) | [Microsoft Azure](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-storage-tiers) | [IBM Cloud](https://cloud.ibm.com/objectstorage/create#pricing) | [Oracle Cloud](https://www.oracle.com/cloud/storage/pricing.html) | [Alibaba Cloud](https://www.alibabacloud.com/product/oss/pricing) |
5886
| ------------- | --------- | ------------------------ | ------- |---------- | ----------------- | ----------------- |

‎examples/catalog-link.json‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
{
2+
"stac_version": "1.0.0",
3+
"stac_extensions": [
4+
"https://stac-extensions.github.io/storage/v2.0.0/schema.json"
5+
],
6+
"type": "Catalog",
7+
"id": "20190822T183518Z_746_POM1_ST2_P",
8+
"title": "Example Catalog",
9+
"description": "An example catalog with a link to documentation on object storage.",
10+
"storage:schemes": {
11+
"aws": {
12+
"platform": "AWS",
13+
"region": "us-west-2",
14+
"requester_pays": true,
15+
"tier": "Standard"
16+
}
17+
},
18+
"links": [
19+
{
20+
"href": "https://example.com/examples/catalog-link.json",
21+
"rel": "self"
22+
},
23+
{
24+
"title": "Documentation",
25+
"href": "s3://mybucket/project/documentation.pdf",
26+
"type": "application/pdf",
27+
"rel": "about",
28+
"storage:refs": [
29+
"aws"
30+
]
31+
}
32+
]
33+
}

‎examples/item-naip.json‎

Lines changed: 81 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
{
22
"stac_version": "1.0.0",
33
"stac_extensions": [
4-
"https://stac-extensions.github.io/storage/v1.0.0/schema.json"
4+
"https://stac-extensions.github.io/storage/v2.0.0/schema.json",
5+
"https://stac-extensions.github.io/version/v1.2.0/schema.json"
56
],
67
"id": "m_3009743_sw_14_1_20160928_20161129",
7-
"collection": "NAIP_MOSAIC",
88
"bbox": [
99
-97.75,
1010
30.25,
@@ -43,65 +43,107 @@
4343
"datetime": "2016-09-28T00:00:00+00:00",
4444
"mission": "NAIP",
4545
"platform": "UNKNOWN_PLATFORM",
46-
"gsd": 1
46+
"gsd": 1,
47+
"storage:schemes": {
48+
"az-wus2-arc": {
49+
"platform": "AZURE",
50+
"region": "westus2",
51+
"tier": "archive"
52+
},
53+
"gs-cld": {
54+
"platform": "GCP",
55+
"region": "us-central1",
56+
"requester_pays": true,
57+
"tier": "COLDLINE"
58+
},
59+
"aws-std": {
60+
"platform": "AWS",
61+
"region": "us-west-2",
62+
"requester_pays": true,
63+
"tier": "Standard"
64+
},
65+
"az-weu-hot": {
66+
"platform": "AZURE",
67+
"region": "westeurope",
68+
"requester_pays": false,
69+
"tier": "hot"
70+
},
71+
"az-eus-hot": {
72+
"platform": "AZURE",
73+
"region": "eastus",
74+
"requester_pays": false,
75+
"tier": "hot",
76+
"deprecated": true
77+
},
78+
"minio": {
79+
"platform": "https://play.min.io:9000"
80+
}
81+
}
4782
},
4883
"assets": {
84+
"CO_GEOTIFF_RGB": {
85+
"href": "s3://mybucket/tx/2016/100cm/rgb/30097/m_3009743_sw_14_1_20160928.tif",
86+
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
87+
"storage:refs": [
88+
"minio"
89+
]
90+
},
91+
"CO_GEOTIFF_AWS_RGB": {
92+
"href": "s3://naip-visualization/tx/2016/100cm/rgb/30097/m_3009743_sw_14_1_20160928.tif",
93+
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
94+
"storage:refs": [
95+
"gs-std"
96+
]
97+
},
4998
"GEOTIFF_AZURE_RGBIR": {
5099
"href": "https://naip-nsl.blob.core.windows.net/tx/2016/100cm/rgb/30097/m_3009743_sw_14_1_20160928.tif",
51-
"type": "image/vnd.stac.geotiff",
52-
"storage:platform": "AZURE",
53-
"storage:region": "westus2",
54-
"storage:tier": "archive"
100+
"type": "image/tiff; application=geotiff",
101+
"storage:refs": [
102+
"az-wus2-ar"
103+
]
55104
},
56105
"CO_GEOTIFF_GCP_RGB": {
57106
"href": "gs://naip-data/tx/2016/100cm/rgb/30097/m_3009743_sw_14_1_20160928.tif",
58-
"type": "image/vnd.stac.geotiff; cloud-optimized=true",
59-
"storage:platform": "GCP",
60-
"storage:region": "us-central1",
61-
"storage:requester_pays": true,
62-
"storage:tier": "COLDLINE"
63-
},
64-
"CO_GEOTIFF_AWS_RGB": {
65-
"href": "s3://naip-visualization/tx/2016/100cm/rgb/30097/m_3009743_sw_14_1_20160928.tif",
66-
"type": "image/vnd.stac.geotiff; cloud-optimized=true",
67-
"storage:platform": "AWS",
68-
"storage:region": "us-west-2",
69-
"storage:requester_pays": true,
70-
"storage:tier": "Standard"
107+
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
108+
"storage:refs": [
109+
"gs-cld"
110+
]
71111
},
72112
"CO_GEOTIFF_AZURE_RGB": {
73113
"href": "https://naipeuwest.blob.core.windows.net/naip/v002/tx/2016/tx_100cm_2016/30097/m_3009743_sw_14_1_20160928.tif",
74-
"type": "image/vnd.stac.geotiff; cloud-optimized=true",
75-
"storage:platform": "AZURE",
76-
"storage:region": "westeurope",
77-
"storage:requester_pays": false,
78-
"storage:tier": "hot"
114+
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
115+
"storage:refs": [
116+
"az-weu-hot"
117+
]
79118
},
80119
"CO_GEOTIFF_AZURE_RGB_DEPRECATED": {
81120
"href": "https://naipblobs.blob.core.windows.net/naip/v002/tx/2016/tx_100cm_2016/30097/m_3009743_sw_14_1_20160928.tif",
82-
"type": "image/vnd.stac.geotiff; cloud-optimized=true",
83-
"storage:platform": "AZURE",
84-
"storage:region": "eastus",
85-
"storage:requester_pays": false,
86-
"storage:tier": "hot"
121+
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
122+
"storage:refs": [
123+
"az-eus-hot"
124+
],
125+
"deprecated": true
126+
},
127+
"THUMBNAIL": {
128+
"href": "https://naipblobs.blob.core.windows.net/naip/v002/tx/2016/tx_100cm_2016/30097/m_3009743_sw_14_1_20160928.200.jpg",
129+
"type": "image/jpeg",
130+
"storage:refs": [
131+
"minio"
132+
]
87133
},
88134
"THUMBNAIL_AZURE_DEPRECATED": {
89135
"href": "https://naipblobs.blob.core.windows.net/naip/v002/tx/2016/tx_100cm_2016/30097/m_3009743_sw_14_1_20160928.200.jpg",
90136
"type": "image/jpeg",
91-
"storage:platform": "AZURE",
92-
"storage:region": "eastus",
93-
"storage:requester_pays": false,
94-
"storage:tier": "hot"
137+
"storage:refs": [
138+
"az-eus-hot"
139+
],
140+
"deprecated": true
95141
}
96142
},
97143
"links": [
98144
{
99-
"href": "https://example.com/examples/item.json",
145+
"href": "https://example.com/examples/item-naip.json",
100146
"rel": "self"
101-
},
102-
{
103-
"href": "https://example.com/examples/item.json",
104-
"rel": "collection"
105147
}
106148
]
107149
}

0 commit comments

Comments
 (0)