Add distribution spec project proposal #35
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| # Abstract | ||
|
|
||
| The Docker registry protocol has become the defacto standard across the container registry world. | ||
|
|
||
| In the OCI, having a solid, common distribution specification with conformance testing will ensure long lasting security and interoperability throughout the container ecosystem. | ||
|
|
||
| ## Proposal | ||
|
|
||
| TL;DR; Move [`api.md`][api.md] to a new [distribution-spec project](https://github.com/opencontainers/distribution-spec). | ||
|
|
||
| This proposal covers the distribution API spec, and while it does not cover the code for the docker-registry, that implementation is considered the reference implementation. There are other implementations of this protocol, not all are open-source though (Google gcr.io, Amazon ECR, CoreOS Quay, Gitlab registry, JFrog Artifactory registry, Huawei Dockyard, etc). | ||
|
|
||
| In the past when the topic of having an OCI specification around the distribution of container images was discussed, it was deferred as "let’s get the image format defined, meanwhile the industry will settle on a distribution standard". Fast forward, OCI image format is out and adopted, and the Registry v2 is the defacto standard. There is and will be use-cases for alternate methods and the future will likely hold creative ways to push, fetch and share container images, but right now this promotion serves to acknowledge by the OCI the current industry standard of distributing container images. | ||
| This proposal also provides the container ecosystem with a means to discuss and schedule extensions to the distribution specification. | ||
|
|
||
| There is polish that is needed e.g. broken links to storage-driver docs, as well as making sections more generic regarding the OCI descriptors and media-types, but on the whole this is a lateral move. | ||
|
|
||
| ## Initial Maintainers | ||
|
|
||
| * Stephen Day <[email protected]> (@stevvooe) | ||
| * Vincent Batts <[email protected]> (@vbatts) | ||
| * Derek McGowan <[email protected]> (@dmcgowan) | ||
|
|
||
| Additional Maintainers to consider: | ||
|
Contributor
There was a problem hiding this comment. The TOB needs to pick an initial maintainer set. I think these suggestions are for the TOB, and should be reviewed and either added to the initial maintainer list or dropped before the vote. A three-person initial maintainer set can add other maintainers on their own, so the TOB can drop tge whole consideration list and punt to the initial maintainers if it wants.
Member
There was a problem hiding this comment. Same. How are TOB members supposed to interpret this list? Maintainers are picked based on current and past contributions to a codebase or area of expertise and usually their maintainer vote comes with justification of why they should be a maintainer in a project.
Contributor
There was a problem hiding this comment. @caniszczyk, were we just dropping this list, and leaving it up to the above-three maintainers to add additional maintainers? |
||
|
|
||
| * Ahmet Alp Balkan (Google) | ||
| * Matt Moore (Google) | ||
| * Yuwa (MSFT) | ||
| * Clayton Coleman (Red Hat) | ||
| * Antonio Murdaca (@runcom) (Red Hat) | ||
| * Samuel Karp (@samuelkarp) (AWS) | ||
| * Mike Brown (IBM) | ||
| * Jimmy Zelinskie [email protected] (@jzelinskie) | ||
| * Liu Genping <[[email protected]](mailto:[email protected])> | ||
| * Vanessa Sochat (@vsoch) (Stanford) <[email protected]> | ||
| * Eduardo Arango (@ArangoGutierrez) (Sylabs) <[email protected]> | ||
|
|
||
| ## Code of Conduct | ||
|
|
||
| This project would incorporate (by reference) the OCI Code of Conduct ([https://github.com/opencontainers/tob/blob/master/code-of-conduct.md](https://github.com/opencontainers/tob/blob/master/code-of-conduct.md)). | ||
|
|
||
| ## Governance and Releases | ||
|
|
||
| This project would incorporate the Governance and Releases processes from the OCI project template: [https://github.com/opencontainers/project-template](https://github.com/opencontainers/project-template). | ||
|
|
||
| ## Project Communications | ||
|
|
||
| Both of the proposed projects would continue to use existing channels in use by the OCI developer community for communication including: | ||
|
|
||
| * GitHub for issues and pull requests | ||
| * The [email protected] email list | ||
| * The monthly OCI developer community conference call | ||
|
Contributor
There was a problem hiding this comment. Does starting at v2 (as suggested below) get us out of the pre-1.0 weekly meeting recommendation? I'm not sure if the "we're starting at v2" approach was considered when writing those docs (I certainly hadn't considered it).
Member
There was a problem hiding this comment. It was discussed in the google doc
Contributor
There was a problem hiding this comment. hey @wking here you go! If you follow the link at the top, click on "Comments" in the top right, you can explore some of the discussion.
Contributor
There was a problem hiding this comment. I'm fine with starting at v2. I'm just not clear on whether monthly meetings are the right target level for a new spec (regardless of the number we put on our initial release). The discussion in this PR suggests folks are going to have lots of ideas. Project-template suggests weekly meetings during pre-1.0 development (presumably because the roadmap is less obvious/established then). And there's no reason that roadmap discussions and such couldn't happen asynchronously (but then maybe we want to drop the meeting recommendation from project-template?). Anyhow, I'd just like to make sure that, if this project starts out with monthly meetings, it was a conscious decision that we didn't expect to need weekly meetings.
Contributor
There was a problem hiding this comment.
The specification is not new. It was already discussed in the open and 100+ comments.
Member
There was a problem hiding this comment. @wking sounds like something to discuss on the next monthly meeting :-)
Member
There was a problem hiding this comment. While there is plenty of discussion here, not such an overhaul needed as to change increase from monthly. I am sure we'll adjust the meeting schedule as needed |
||
| * The #OpenContainers freenode IRC channel | ||
|
|
||
| ## Versioning / Roadmap | ||
|
|
||
| The API spec is currently considered v2 and we will start the specification at v2.0. Fewer places to change and compare, and it would keep with it being a lateral move. | ||
|
Member
There was a problem hiding this comment. I'm not sure we should start with How about starting with
Contributor
There was a problem hiding this comment.
Somewhat complicating this approach is that the fact that Docker's v2 API has been receiving lettered minor/patch bumps. Presumably those will all be considered part of the 2.0.x series? And if the goal is to start using SemVer (which sounds like a useful goal to me), we'll need to specify that and wait for some deprecation period before cutting v2.1, or we'll have all the 2.0 clients choke and die when the version-check endpoint sets
Contributor
There was a problem hiding this comment. @wking In general, since the subversions are backwards compatible, we do not expose the subversions. In general, we should move away from path-level versioning and favor type-level versioning.
Member
There was a problem hiding this comment. @AkihiroSuda are you saying for that header, or just tagged releases or both?
Member
There was a problem hiding this comment. FWIW, Docker v2-2 is the most recent protocol -- I hope everyone here knows that and has just been using 2.1 as an example. |
||
|
|
||
| ## Frequently Asked Questions (FAQ) | ||
|
|
||
| **Q: Does this include the code of the docker-registry?** | ||
|
|
||
| A: No. This is an API specification discussion. | ||
|
|
||
| **Q: Does this change the OCI Charter or Scope Table?** | ||
|
|
||
| A: Not the charter, but it does change the scope table. | ||
| This project is scoped to specifying per-image client ↔ registry interaction with an [HTTP][rfc7230]-based protocol. | ||
| The following scope entries should be removed from the [scope table][scope]: | ||
|
|
||
| * “Use of Hash as Content Addressable name for immutable containers”. | ||
| This entry is in scope for this project, and a more detailed entry will be added as described below. | ||
| * “Creating Reference spec for optional DNS based naming & distribution”. | ||
| This entry conflates naming and distribution, which will be separated by this proposal. | ||
| * “Standardizing on a particular Distribution method”. | ||
| This proposal will provide one (of possibly many) distribution specifications, so the old “There is no current agreement on how to distribute content” no longer applies. | ||
|
|
||
| The following entries should be added to the [scope table][scope]: | ||
|
|
||
| * “Specifying authentication and authorization schemes”. | ||
| Docker's current registry uses an [extension][token] of the [`Bearer`][rfc6750] [auth scheme][rfc7235-s2.1]. | ||
| Work on specifying Docker's scheme will continue independently, and is orthogonal to the registry API. | ||
|
|
||
| * What: Specifying authentication and authorization schemes | ||
| * In/Out/Future: Out of scope | ||
| * Status: N/A | ||
| * Description: Defining protocols for authenticating and authorizing distribution access. | ||
| * Why: As an HTTP-based protocol, clients and servers can negotiate authentication via HTTP's [challenge-response authentication framework][rfc7235-s2.1]. | ||
|
Contributor
There was a problem hiding this comment. This should be specified in such a way that the bearer can be opaque to the registry implementation.
Contributor
There was a problem hiding this comment.
Can you file a PR against this branch or suggest alternative wording? I think making negotiated authentication out of scope (as I've tried to do with this scope entry) is even stronger than “opaque to the registry implementation”. More on this specific issue in #37, where I initially tried to make the
Member
There was a problem hiding this comment.
The mentioned rfc is broad enough to cover this case. This section is just saying the registry can return a response requesting a specific authentication type (could be bearer, basic, or whatever). The comment below this could probably be more generalized or removed completely as the rfc already describes this behavior in detail. |
||
|
|
||
| * “Creating a reference spec for optional DNS based naming and discovery”. | ||
| Discovery and registry protocols are completely separate and do not need to be added together. | ||
| This entry replaces part of the previous “Creating Reference spec for optional DNS based naming & distribution” entry. | ||
|
|
||
| * What: Creating a reference spec for optional DNS based naming and discovery | ||
| * In/Out/Future: In scope for future specification | ||
| * Status: Work not yet started | ||
| * Description: Define a protocol for resolving an image name to retrieval information. | ||
| When we address this, we will also allow for alternative name-to-image discovery protocols in parallel with the OCI-specified protocol. | ||
| * Why: It is reasonable to provide a standardized way to use DNS based distribution in conjunction with OCI without requiring its use. | ||
| There are many good use cases for DNS based distribution, but not all use cases support this. | ||
| Furthermore, encoding the location of a bundle into the bundle can cause issues with downloads from alternate locations other than the origin specified in the name. | ||
|
|
||
| * “Specifying a distribution method”. | ||
| This entry replaces part of the previous “Creating Reference spec for optional DNS based naming & distribution” and “Standardizing on a particular Distribution method” entries. | ||
|
|
||
| Tag-listing (e.g. “what named manifests are in `library/busybox`?”) endpoints are in scope and required for backwards compatibility with clients. | ||
| An endpoint for listing image repository names within a registry is considered out of scope for this specification. | ||
|
|
||
| Managing the grouping of image repository names is considered part of distribution policy or content management, which are out of scope. | ||
| For example, “which image repositories are under `library/`?” is out of scope for this project. | ||
|
|
||
| * What: Specifying a distribution method | ||
| * In/Out/Future: In scope | ||
| * Status: In progress (see opencontainers/distribution-spec) | ||
| * Description: Define a protocol for creating, retrieving, updating, and deleting content-addressable objects, such as those defined in the [image specification][image-spec]. | ||
| * Why: This specification will provide one (of possibly many) distribution specifications. | ||
| Alternative distribution specifications may be developed for uses cases not covered by this specification, but defining them is currently out of scope for the OCI. | ||
|
|
||
| * “Retrieving image content by its content-addressable hash”. | ||
| Docker's registry API already provides [endpoints for fetching manifest objects by digest][get-manifest]. | ||
| Docker's registry API does not currently provide endpoints for fetching [image-index][] objects by digest, but this is the project where that will happen. | ||
|
Contributor
There was a problem hiding this comment.
This is the “implicit link name shortcut”, and GitHub renders it fine.
Member
There was a problem hiding this comment. I think you can forgo the |
||
|
|
||
| * What: Retrieving image content by its content-addressable hash | ||
| * In/Out/Future: In scope | ||
| * Status: In progress (see opencontainers/distribution-spec) | ||
| * Description: Specify a protocol for retrieving an [image index][image-index], [manifest][], or other [image specification][image-spec] object from a distribution engine by its content-addressable hash. | ||
| * Why: Using a hash as a name is a way to ensure a unique image name without relying on a particular naming authority or system. | ||
| Using hashing for name is an acceptable addition as it does not encode any centralized namespace. | ||
|
|
||
| The following entries should remain in the [scope table][scope] but not be addressed by this project: | ||
|
|
||
| * “Specifying way to attach signatures”. | ||
| We don't need to address this as part of distribution, because all resources are content-addressable and can be signed in external systems. | ||
|
|
||
| ## Related GitHub Issues | ||
|
|
||
| * Simplifies tag listing: docker/distribution#2169 | ||
|
Contributor
There was a problem hiding this comment. Let's add a statement here about what these PRs mean. Are these additions required by the proposal? They have been vetted and implemented, so I would think they are a good candidate. |
||
| * Allows listing of manifests: docker/distribution#2199 | ||
|
|
||
| [api.md]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/api.md | ||
| [catalog]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/api.md#catalog | ||
| [get-manifest]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/api.md#pulling-an-image-manifest | ||
| [image-spec]: https://github.com/opencontainers/image-spec/ | ||
| [image-index]: https://github.com/opencontainers/image-spec/blob/v1.0.1/image-index.md | ||
| [manifest]: https://github.com/opencontainers/image-spec/blob/v1.0.1/manifest.md | ||
| [manifests]: https://github.com/opencontainers/image-spec/blame/v1.0.1/image-index.md#L23 | ||
| [rfc6750]: https://tools.ietf.org/html/rfc6750 | ||
| [rfc7230]: https://tools.ietf.org/html/rfc7230 | ||
| [rfc7235-s2.1]: https://tools.ietf.org/html/rfc7235#section-2.1 | ||
| [scope]: https://www.opencontainers.org/about/oci-scope-table | ||
| [token]: https://github.com/docker/distribution/blob/5cb406d511b7b9163bff9b6439072e4892e5ae3b/docs/spec/auth/token.md | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this proposal referencing project-template for the project's initial rules, we'll need a way to deal with project-template's Chief Maintainer role. Possibilities include:
My recommendation would be to remove the role (slightly more discussion here and here), but that PR has been dangling for 1.4 years, so I don't know how likely it is to land in the next month. Perhaps parallel work can land the Chief Maintainer removal, or opencontainers/project-template#41 may end up with the TOB in charge of project-template. With project-template (currently) outside of direct TOB control, the simplest approach would be to appoint a Chief Maintainer here, as that entirely within the TOB's control.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This happened, so nothing left to do for this specific point.