Skip to content

Commit 355b19d

Browse files
committed
Merge pull request #111 from stevvooe/descriptors-specification
spec: describe descriptors and digests
2 parents 361950c + 0b2b3fd commit 355b19d

File tree

8 files changed

+189
-50
lines changed

8 files changed

+189
-50
lines changed

descriptor.md

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# OpenContainers Content Descriptors
2+
3+
OCI have several components that come to together to describe an image.
4+
References between components form a [Merkle Directed Acyclic Graph (DAG)](https://en.wikipedia.org/wiki/Merkle_tree).
5+
The references in the _Merkle DAG_ are expressed through _Content Descriptors_.
6+
A _Content Descriptor_ or _Descriptor_, describes the disposition of targeted content.
7+
A _Descriptor_ includes the type of content, an independently-verifiable content identifier, known as a "digest" and the byte-size of the raw content.
8+
9+
Descriptors SHOULD be embedded in other formats to securely reference external content.
10+
11+
Other formats SHOULD use descriptors to securely reference external content.
12+
13+
## Properties
14+
15+
The following describe the primary set of properties that make up a _Descriptor_.
16+
17+
- **`mediaType`** *string*
18+
19+
This REQUIRED property contains the MIME type of the referenced object.
20+
21+
- **`digest`** *string*
22+
23+
This REQUIRED property is the _digest_ of the targeted content, meeting the requirements outlined in [Digests and Verification](#digests-and—verification).
24+
Retrieved content SHOULD be verified against this digest when consumed via untrusted sources.
25+
26+
- **`size`** *int*
27+
This REQUIRED property specifies the size in bytes of the blob.
28+
This property exists so that a client will have an expected size for the content before validating.
29+
If the length of the retrieved content does not match the specified length, the content SHOULD NOT be trusted.
30+
31+
### Reserved
32+
33+
The following are field keys that MUST NOT be used in descriptors specified in other OCI specifications:
34+
35+
- **`urls`** *array*
36+
37+
This key is RESERVED for future versions of the specification.
38+
39+
- **`data`** *string*
40+
41+
This key is RESERVED for futures versions of the specification.
42+
43+
All other fields may be included in other OCI specifications.
44+
Extended _Descriptor_ field additions proposed in other OCI specifications SHOULD first be considered for addition into this specification.
45+
46+
## Digests and Verification
47+
48+
The _digest_ component of a _Descriptor_ acts as a content identifier, employing [content addressability](http://en.wikipedia.org/wiki/Content-addressable_storage) for the OCI image format.
49+
It uniquely identifies content by taking a collision-resistant hash of the bytes.
50+
Such an identifier can be independently calculated and verified by selection of a common _algorithm_.
51+
If such an identifier can be communicated in a secure manner, one can retrieve the content from an insecure source, calculate it independently and be certain that the correct content was obtained.
52+
Put simply, the identifier is a property of the content.
53+
54+
To disambiguate from other concepts, we call this identifier a _digest_.
55+
A _digest_ is a serialized hash result, consisting of a _algorithm_ and _hex_ portion.
56+
The _algorithm_ identifies the methodology used to calculate the digest, which are shared by implementations.
57+
The _hex_ portion is the hex-encoded result of the hash.
58+
59+
We define a _digest_ string to match the following grammar:
60+
61+
```
62+
digest := algorithm ":" hex
63+
algorithm := /[A-Fa-f0-9_+.-]+/
64+
hex := /[A-Fa-f0-9]+/
65+
```
66+
67+
Some examples of _digests_ include the following:
68+
69+
digest | description |
70+
----------------------------------------------------------------------------------|------------------------------------------------
71+
sha256:6c3c624b58dbbcd3c0dd82b4c53f04194d1247c6eebdaab7c610cf7d66709b3b | Common sha256 based digest |
72+
73+
Before consuming content targeted by a descriptor from untrusted sources, the byte content SHOULD be verified against the _digest_.
74+
The size of the content SHOULD be verified to reduce hash collision space.
75+
Heavy processing of before calculating a hash SHOULD be avoided.
76+
Implementations MAY employ some canonicalization to ensure stable content identifiers.
77+
78+
### Algorithms
79+
80+
While the _algorithm_ does allow one to implement a wide variety of algorithms, compliant implementations SHOULD use [SHA-256](#SHA-256).
81+
82+
Let's use a simple example in pseudo-code to demonstrate a digest calculation:
83+
A _digest_ is calculated by the following pseudo-code, where `H` is the selected hash algorithm, identified by string `<alg>`:
84+
```
85+
let ID(C) = Descriptor.digest
86+
let C = <bytes>
87+
let D = '<alg>:' + EncodeHex(H(C))
88+
let verified = ID(C) == D
89+
```
90+
Above, we define the content identifier as `ID(C)`, extracted from the `Descriptor.digest` field.
91+
Content `C` is a string of bytes.
92+
Function `H` returns a the hashs of `C` in bytes and is passed to function `EncodeHex` to obtain the _digest_.
93+
The result `verified` is true if `ID(C)` is equal to `D`, confirming that `C` is the content identified by `D`.
94+
After verification, the following is true:
95+
96+
```
97+
D == ID(C) == '<alg>:' + EncodeHex(H(C))
98+
```
99+
100+
The _digest_ is confirmed as the content identifier by independently calculating the _digest_.
101+
102+
#### SHA-256
103+
104+
[SHA-256](https://tools.ietf.org/html/rfc4634#page-7) is a collision-resistant hash function, chosen for ubiquity, reasonable size and secure characteristics.
105+
Implementations MUST implement SHA-256 digest verification for use in descriptors.
106+
107+
## Examples
108+
109+
The following example describes a [_Manifest_](manifest.md#image-manifest) with a content identifier of "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270", of size 7682 bytes:
110+
111+
```json,title=Content%20Descriptor&mediatype=application/vnd.oci.descriptor.v1%2Bjson
112+
{
113+
"mediaType": "application/vnd.oci.image.manifest.v1+json",
114+
"size": 7682,
115+
"digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270"
116+
}
117+
```

media-types.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
The following `mediaType` MIME types are used by the formats described here, and the resources they reference:
44

5+
- `application/vnd.oci.descriptor.v1+json`: [Content Descriptor](descriptor.md)
56
- `application/vnd.oci.image.manifest.list.v1+json`: [Manifest list](manifest.md#manifest-list)
67
- `application/vnd.oci.image.manifest.v1+json`: [Image manifest format](manifest.md#image-manifest)
78
- `application/vnd.oci.image.serialization.rootfs.tar.gzip`: ["Layer", as a gzipped tar archive](serialization.md#creating-an-image-filesystem-changeset)

schema/content-descriptor.json

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"description": "OpenContainer Content Descriptor Specification",
3+
"$schema": "http://json-schema.org/draft-04/schema#",
4+
"id": "https://opencontainers.org/schema/descriptor",
5+
"type": "object",
6+
"properties": {
7+
"mediaType": {
8+
"description": "the mediatype of the referenced object",
9+
"$ref": "defs-image.json#definitions/mediaType"
10+
},
11+
"size": {
12+
"description": "the size in bytes of the referenced object",
13+
"type": "integer"
14+
},
15+
"digest": {
16+
"$ref": "defs-image.json#definitions/digest"
17+
}
18+
},
19+
"required": [
20+
"mediaType",
21+
"size",
22+
"digest"
23+
]
24+
}

schema/defs-image.json

Lines changed: 0 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -11,28 +11,6 @@
1111
"type": "string",
1212
"pattern": "^[a-z0-9]+:[a-fA-F0-9]+$"
1313
},
14-
"descriptor": {
15-
"id": "https://opencontainers.org/schema/image/descriptor",
16-
"type": "object",
17-
"required": [
18-
"mediaType",
19-
"size",
20-
"digest"
21-
],
22-
"properties": {
23-
"mediaType": {
24-
"description": "the mediatype of the referenced object",
25-
"$ref": "#definitions/mediaType"
26-
},
27-
"size": {
28-
"description": "the size in bytes of the referenced object",
29-
"type": "integer"
30-
},
31-
"digest": {
32-
"$ref": "#definitions/digest"
33-
}
34-
}
35-
},
3614
"manifestDescriptor": {
3715
"id": "https://opencontainers.org/schema/image/manifestDescriptor",
3816
"type": "object",

schema/fs.go

Lines changed: 39 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,7 @@ var _escData = map[string]*_escFile{
203203
"/config-schema.json": {
204204
local: "config-schema.json",
205205
size: 707,
206-
modtime: 1463700693,
206+
modtime: 1464139547,
207207
compressed: `
208208
H4sIAAAJbogA/5SRPW7DMAyF5/oUhpOxjjp0ytoDdOgJVJmKGcCiQDJDUPju1U/c2kvhLobx+L73JOqr
209209
adtuAHGMUZFCd2679wjhjYJaDMBt+vN4aT8iOPTobHE9Z+woboTJZmRUjWdjrkKhr+qJ+GIGtl77l1dT
@@ -213,10 +213,23 @@ T4aE9IoTdGU2V0tnbzoS/xG1dbMbUdPhbgx7GZK9zscuVu4jgy+HBy99HZ/yKxxMUjBgfi1ZdrjJYiL1
213213
`,
214214
},
215215

216+
"/content-descriptor.json": {
217+
local: "content-descriptor.json",
218+
size: 616,
219+
modtime: 1464914956,
220+
compressed: `
221+
H4sIAAAJbogA/4yRMVPDMAyF9/wKXdqR1gxMXWFngI1jcG0lVe9iG1kMhet/x4oTSGGgW/L8vvck+7MB
222+
aD1mx5SEYmh30D4mDPcxiKWADPqFQeBhMkWGp4SOOnJ2JG40Yp3dAQer+EEk7Yw55hg2Vd1G7o1n28nm
223+
9s5UbVU58jOSCxNLs5ub84hVt/Hf7ZWTU0Il4/6ITqqWuPAshLmc6GJFG9CTfa7mKv3dVw4Io09DIXag
224+
AmOHXKZBD4uOEV+XM+U8dnlDg+1xq8uuyj8F0tRsfnpH6lzhNtPHf5OoBSjA/iSYr5hmvgkqz9QjX/Z5
225+
6jHLsvGa4SeqJjVTWsv49k6M+mAvvy93ud5ldfl5bc7NVwAAAP//Zc2MR2gCAAA=
226+
`,
227+
},
228+
216229
"/defs-config.json": {
217230
local: "defs-config.json",
218231
size: 1755,
219-
modtime: 1463700704,
232+
modtime: 1464139547,
220233
compressed: `
221234
H4sIAAAJbogA/7xUzc7TMBA8N09hBY6BXhAHri1HVKQIOFZusm63xF5rvQEi1HfHSauS/qRKKV8PVeOx
222235
Z2bXY/t3olRaQigYvSC59INK52DQYTsKymsWLOpKsxJSCw9uRk40OmAVvwyuVe6hQIOF7vjZXvCoEAVb
@@ -230,27 +243,27 @@ W1SwXppll8oQfUVUgXaDqSTtb5f8CQAA//8Cok052wYAAA==
230243

231244
"/defs-image.json": {
232245
local: "defs-image.json",
233-
size: 3100,
234-
modtime: 1462965360,
246+
size: 2528,
247+
modtime: 1464914958,
235248
compressed: `
236-
H4sIAAAJbogA/+xWy27bMBC86ysIJUAOfqjXGkGAoEGBnlIgPdVQgY20kphKpLqkgzqB/r3U+x3Xidte
237-
erK55A5nhmPSzxZjto/KI55qLoW9YfYNBlzwfKRYCqS5t4uBmJbsNkXxQQoNXCCxTwmEyO5S9HjAPSja
238-
lyVeA2Dw8i1MMUGfw5d9ik3JFLmfbxhpnaqN40gD79Xwai0pdJQXYQIOz7dyWohlDaBLQFtp4iJs6ylo
239-
jVTI+baF1ZO7cLbvVu/Nt+vV1/XCXZzbxdKs7LB9HqLSXWoDU3SEzKN9qmVIkEbcY4aZ913tElb2Mhmw
240-
fJG8f0BPLxkXxbAiwi4uI1DR1eYywp/gG8sSiKvOq4vj9Rgt7mJjvgXXq4/FYCiooi/p9X53MEYES5lt
241-
nfDHjhPm+Nuq1jv0ZVtU/Kk3rryvCm6rmQxBEz9UHQkzSZo7smJtzrk+HsIAychGnw0kFBDnZj7vPetk
242-
uJO7Zmk21HOYSr4sT8X9XqP6TTq121xoDJGm9x9ld15J32oDY3U/6+wkIHhg1t2cIEMTWH8hS51KGoMO
243-
JCX/8/Uv8jV1EAOg4/LUoEzqmNI4maZiBsiLuDYNO8Jej5mTyu4U3B7iTHDGmMPZV6t1XqA6fjV609lY
244-
2OloGbA3klk/mh3KFJ+OVAP6VnIBQu74aS1rUWfpARHsx9MmAckUlwPC2nt+WugjEAcx/IUf7ddLZv0x
245-
YSMo8P3iMoL4c/dnGkCs0JrzJDvwIoIQUkP/H+3REeiCNI+QFHgb9K6myVvWXLJq/aCkOHN6Lwekd4Uv
246-
dwN3Or48T12UYhfH478BrlWPMiuzfgUAAP//VjUNyBwMAAA=
249+
H4sIAAAJbogA/7RWy27bMBC8+ysIJUAOfqjXGkGAoEGBnlIgPTVQgY20kphKpLqkgzqB/r2k3k+nrt0b
250+
ueQOZ4Zjym8LxpwAlU8801wKZ8ucOwy54HamWAakub9LgJiW7D5D8UkKDVwgsS8pRMgeMvR5yH0o2lcl
251+
XgNg8OwRpphiwOHbPsOmZIo8sAfGWmdq67rSwPs1vNpIilzlx5iCy+1RbguxqgF0CegoTVxEbT0DrZEK
252+
OT8eYf3qLd3HD+uPZnS7/r5ZestLp9ialx1OwCNUukttYIqOkfm0z7SMCLKY+8ww83+qXcrKXiZDZjfJ
253+
p2f09YpxUUwrIuzqOgYV32yvY/wNgbEshaTqvLk6Xo/R4i23ZhTerj8Xk4GgFAQPDfhdJUPSCb6PsUaE
254+
S9ltnfDXjhPacx6rWi8Eq7ao+GtvXt1Fp5IloENJqVOVvNYXMuRNRFF15M2kbe5ai71WR32FhCGSsQQD
255+
NpBVQFyaddt70cl5J5vN1nyo8X0qdptNztNeo/pLOvUNcKExQpo+f5TveSXV1kmY5iIGQMflqUGZ1DGl
256+
cTJNxQqQH3NtGnaEvR6zJpXTKXg9xJngjDGHq/+s1j1AdfzL7y3nY2Hno2XATiSzeTEnlCk+H6kG9FRy
257+
IYJ1/LyWtaiz9IAI9uNlk4B0iss7woy0g0JfgDiI4S/8aL8OmfXfhI2gIAiKxwiSr92faQiJwsWcJ+24
258+
HuW9LyIIITX0/5UcHYEuSPMRkgLvw97TNPnKmkdWbZ6VFBdu78sB2UPhy8PAnY4vb1MPpdgliTMS7S3q
259+
Wb7IF38CAAD//wKthPngCQAA
247260
`,
248261
},
249262

250263
"/defs.json": {
251264
local: "defs.json",
252265
size: 3193,
253-
modtime: 1463700693,
266+
modtime: 1464139547,
254267
compressed: `
255268
H4sIAAAJbogA/7RWTXPaMBC98ys8tEfa2PIX9NYp/cghAzOZnjo9uGYBtSCpstxpmuG/VzLGWPZiMKWH
256269
JPau9r23T6tYzwPHGS4gSyUVinI2fOMMp7CkjJq3zMkzWDhqLXm+WvNc6UdwZgLYO85UQhlI51FASpc0
@@ -268,23 +281,23 @@ MrVJbn8cB+ZnN/gbAAD//0JyEpx5DAAA
268281

269282
"/image-manifest-schema.json": {
270283
local: "image-manifest-schema.json",
271-
size: 1064,
272-
modtime: 1462965360,
284+
size: 1032,
285+
modtime: 1464914959,
273286
compressed: `
274-
H4sIAAAJbogA/6RTvVLjMBDu/RQ7TspzdMVVaa+64oaCDA1DIeyVvZlYMlrBTCaTd0c/UZAJBSSlV/v9
275-
Sj5UAHWH3FqaHBldr6G+m1D/NdpJ0mjh3yh7hP9Sk0J2cD9hS4paGbd/BfiS2wFHGaCDc9NaiC0b3aTp
276-
ythedFYq1/z+I9JskXDUZQh7jPGqbVblCEvbgoIDMZ4cJKzbTxjQ5nmL7Wk2Wc9hHSH7kxDMzxLFg2dM
277-
4dL4MvNmIAZFuOuAU0JkcANCFIcsDokP3hIhSAapgbTDHm10EcmvSybmZs9sOWuWifNjOq5H7Ehu0sbh
278-
Rv0PrrP20qIKXB0qbuL6KlzuQvgBaQr1cYGbWfOaivrS17fY8s2YT0l3cu/tl3S5GGmt3BftOxzLvWuF
279-
vfTMgNTauPju+faymx35xkvKn3VeIqvsNTqtLb68ksVg6/Grv+Di5czva163/3iqjtV7AAAA///++ypf
280-
KAQAAA==
287+
H4sIAAAJbogA/6RSPU/zMBDe8ytOacc39TswdWViQAxULIjBJOfkqsYOPoNUVf3v+KMujsoAdMyTez7u
288+
8R0qgLpDbi1Njoyu11A/TKhvjXaSNFq4G2WPcC81KWQHjxO2pKiVcfpfoC+5HXCUgTo4N62F2LLRTUJX
289+
xvais1K55v+NSNgi8ajLFPYc413b7MqRlqYFhQRiPCVIXLefMLDN6xbbEzZZr2EdIfs/YTGPJYknr5iW
290+
S/DlzpuBGBThrgNOGyKDGxCiOWRzSHrwkQRBMkgNpB32aGOKKP63zcQ87Fkt75ptIn5Mv+sRO5KbNHG4
291+
0v9L6+y9tKiCVoeKmzi+Co+7EB4gTaE+LnizaN5TUV/mymohDWrX5EcwNqrO6Tu593FLei5CWiv3RdsO
292+
x3Lup0beamYotTYu3jVfX2azI99oKfm7TktmlbPGpLXFt3eyGGI9f3flF5cxf495vf7jpTpWnwEAAP//
293+
X3p8DwgEAAA=
281294
`,
282295
},
283296

284297
"/manifest-list-schema.json": {
285298
local: "manifest-list-schema.json",
286299
size: 1010,
287-
modtime: 1462965360,
300+
modtime: 1464139547,
288301
compressed: `
289302
H4sIAAAJbogA/6ySMU/7MBDF93yKU9rxn/o/MHWFBQnEQMWCGExyaa5q7OAzSFXV747tS0qiMIDoUqkv
290303
fu9+7+xjBpBXyKWjzpM1+Rryhw7NtTVek0EHt63eItxrQzWyhzsKP48dllRTqZPlX8xYctlgq6O/8b5b

schema/image-manifest-schema.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,12 @@
1414
"$ref": "defs-image.json#/definitions/mediaType"
1515
},
1616
"config": {
17-
"$ref": "defs-image.json#/definitions/descriptor"
17+
"$ref": "content-descriptor.json"
1818
},
1919
"layers": {
2020
"type": "array",
2121
"items": {
22-
"$ref": "defs-image.json#/definitions/descriptor"
22+
"$ref": "content-descriptor.json"
2323
}
2424
},
2525
"annotations": {

schema/schema.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ import "net/http"
1818

1919
// Media types for the OCI image formats
2020
const (
21+
MediaTypeDescriptor Validator = `application/vnd.oci.descriptor.v1+json`
2122
MediaTypeManifest Validator = `application/vnd.oci.image.manifest.v1+json`
2223
MediaTypeManifestList Validator = `application/vnd.oci.image.manifest.list.v1+json`
2324
MediaTypeImageSerialization unimplemented = `application/vnd.oci.image.serialization.rootfs.tar.gzip`
@@ -32,6 +33,7 @@ var (
3233

3334
// specs maps OCI schema media types to schema files.
3435
specs = map[Validator]string{
36+
MediaTypeDescriptor: "content-descriptor.json",
3537
MediaTypeManifest: "image-manifest-schema.json",
3638
MediaTypeManifestList: "manifest-list-schema.json",
3739
MediaTypeImageSerializationConfig: "config-schema.json",

schema/spec_test.go

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@ var (
3333
errFormatInvalid = errors.New("format: invalid")
3434
)
3535

36+
func TestValidateDescriptor(t *testing.T) {
37+
validate(t, "../descriptor.md")
38+
}
39+
3640
func TestValidateManifest(t *testing.T) {
3741
validate(t, "../manifest.md")
3842
}

0 commit comments

Comments
 (0)