Proposal: Working Group for Reference Types

[SteveLasker]

Working Group Proposal: Reference Types

Proposal created from OCI WG template.

Reference Types OCI Working Group - Governance Charter

This document describes the basic governance principles for the Reference Types Working Group (the “WG”).

The WG operates as an OCI Working Group under the Open Container Initiative (OCI) Charter, which describes the responsibilities of the OCI Technical Oversight Board (the "TOB”). The WG is established by the TOB as an OCI Working Group pursuant to the OCI Charter. Accordingly, the WG will operate in accordance with the OCI Charter and OCI's other policies and procedures, supplemented by the details below.

Purpose

As cloud native development continues to grow, there is increased interest in evolving registries to natively store, discover, and pull a graph of content associated with specific container images in a registry. Use cases for said associated artifacts include but are not limited to Software Bill of Materials (SBoM), security scan results, and signing. Having a native way to store and discover artifacts associated with other artifacts enables end-users to answer the question of: “What SBOMs or signatures are associated with this container image?”

Scope

• Define and deliver the capability to store, discover, and pull a graph of artifacts associated with a specific artifacts to OCI distribution compliant registries. These set of capabilities has been commonly known as "reference types" or "references".
  • Define supported use cases
  • Document impact to existing user-facing tools and registries
  • Define the method for creating, distributing, and discovering referenced objects
  • Document user expectations for promoting an artifact between registries with its references
  • Document onboaring process for registies and user-facing tools to adopt reference types
  • Defined expectations for artifact reference lifecycle management
  • Deliver a reference implementation of the reference types proposal

Out of Scope

• Identified out of scope items will be listed here as WG progresses

Intended work product

• Referrers API specification that provides the ability to discover references to existing container images. These include listing signatures, SBoMs, security scan results that refer to the digest of a manifest. The referrers API specification will sit within or along side the Distribution specification.
• Identify, and document the pros and cons for versioning the existing manifests, compared with creating a new manifest to support reference types.

Proposed Owners

• Lachlan Evenson (@lachie83)
• Justin Cormack (@justincormack)
• Michael Brown (@michaelb990)
• Derek McGowan (@dmcgowan)
• Jon Johnson (@jonjohnsonjr)

Sponsors

• Microsoft
• AWS
• Docker

Related issues/PRs

• Proposal: Working Group for Reference Types #96
• OCI Artifact Manifest - with weak reference support artifacts#27
• OCI artifact manifest, Phase 1-Reference Types artifacts#29
• WIP generic object spec artifacts#37
• Proposal: Add References image-spec#827
• Add reference field image-spec#828
• https://github.com/oras-project/artifacts-spec/

Governance

• Working Group:
  • The TOB is establishing the WG as an OCI Working Group, pursuant to section 6(p) of the OCI Charter.
• Owners:
  • The WG proposal to the TOB will specify one or more initial "owners" of the WG.
  • The current owners will be listed in the OCI Working Group documentation.
  • The owners shall be responsible for:
    • scheduling regular meetings of the WG community;
    • facilitating open discussion among WG community participants;
    • coordinating and managing the development of the WG work product and outputs;
    • recording decisions that are reached by the WG community; and
    • keeping the TOB regularly informed about the status of the WG’s efforts, including when the WG has readied the work product and outputs for TOB approval.
• Maintainers:
  • If the WG owners request the TOB to approve a draft specification as a released OCI Specification, the request shall include a list of proposed "maintainers" of the OCI Specification.
  • The current maintainers will be listed in the OCI Working Group documentation.
  • The maintainers shall be responsible for continuing the work of overseeing updates, improvements and changes to a released OCI Specification on an ongoing basis.
• Meetings:
  • Meetings of the WG shall be open to the public.
  • Participants in the meetings shall comply with the OCI Code of Conduct and all other policies of OCI and The Linux Foundation.
• TOB Approval:
  • The WG shall operate pursuant to the procedures set forth in section 6(p) of the OCI Charter, with regards to obtaining TOB approval for initial release of the work product and outputs as an OCI Specification or other OCI Project, and for subsequent maintenance activities thereafter.
• Amendments:
  • The owners of the WG may from time to time propose to the TOB (1) amendments to this WG Governance Document, and/or (2) changes to the composition of the owners or maintainers of the WG.
  • As set forth in the OCI Charter, the TOB may, in its discretion by a two-thirds vote, approve or reject the requested amendments or changes.
  • As set forth in the OCI Charter, the TOB may also disband the WG by a two-thirds vote.

[dlorenc]

Is this group meant to supersede, replace or take over any of the existing proposals?

opencontainers/artifacts#37
opencontainers/artifacts#29
opencontainers/image-spec#828

[SteveLasker]

In accordance with last weeks working group discussion, this is a working group proposal to define the goals, objectives and, validations which should outline which of these we might pursue, or how we might pursue them in sequence to meet short-term and long term goals.

[dlorenc]

Maybe this is something that needs to be cleared up in the working group proposal itself as it turns into an actual proposed charter change, but my understanding from reading it is that it's intended for the development and proposal of entirely new specifications (like the distribution example).

I think there's still some fundamental disagreement on whether or not reference types actually needs a new specification, or whether they can be accomplished via changes to the existing specifications. Proposing a WG here feels premature, especially before the WG model itself has been sorted out.

[SteveLasker]

Please see the scope section as we called out the specific concern:

Scope
Define enhancements to an existing spec or a new spec for how artifacts may support artifact reference types

[dlorenc]

Right, the question is whether a WG is needed for minor additions to an existing spec.

[dmcgowan]

Agreed that it is something that needs to be clarified. I think with this working group, there needs to be a new specification or major revision which introduces such a manifest. For minor releases in a spec, there does not need to be working group. We still need to figure out in the distribution spec how new endpoints would be handled in minor releases.

The sponsors in this case should include the registry clients which will support this new feature. It is not intended to just be the companies which have engineers working on it, but the projects which will implement it.

What I am hoping the working group proposal method can do is unblock the decision process of what is being delivered since establishing the working group will be voted on. With that in mind, the scope should be explicit about what the expected result should be. For example, is this intending to deliver an artifacts-spec 1.0 or an image spec 2.0. If it has dependency on delivering a distribution-spec 1.1 (or 1.next) with a new endpoint, that should be called out. The distribution spec changes would not be released by this working group though, it could be part of the implementation process though.

[dlorenc]

What I am hoping the working group proposal method can do is unblock the decision process of what is being delivered since establishing the working group will be voted on. With that in mind, the scope should be explicit about what the expected result should be. For example, is this intending to deliver an artifacts-spec 1.0 or an image spec 2.0. If it has dependency on delivering a distribution-spec 1.1 (or 1.next) with a new endpoint, that should be called out. The distribution spec changes would not be released by this working group though, it could be part of the implementation process though.

Well said!

To be extra clear - if minor, backwards compatible changes can be made to image-spec 1.0, I would expect that to still go through the standard image-spec proposal process defined and approved by the image-spec maintainers.

The image-spec group and maintainers could always decide or ask for a working group to help out with the implementation if they think it would help.

If we're not sure whether or not a major revision is needed, IMO the best thing to do would be to figure that out definitively then propose the next course of action, or even to try both paths simultaneously.

[SteveLasker]

With PR #99 merged, what are the next steps for the Reference Types working group?
@estesp, @vbatts,, @dmcgowan

[dlorenc]

My guess as to process, from the charter:

iii. Approval of a new OCI Working Group requires a two-thirds vote of the TOB. The TOB will maintain a public list of all approved OCI Working Groups.

So this would need to be voted on by the TOB.

I think @dmcgowan's questions around exact scope and deliverables of this working group still need to be addressed in the proposal:

With that in mind, the scope should be explicit about what the expected result should be. For example, is this intending to deliver an artifacts-spec 1.0 or an image spec 2.0. If it has dependency on delivering a distribution-spec 1.1 (or 1.next) with a new endpoint, that should be called out. The distribution spec changes would not be released by this working group though, it could be part of the implementation process though.

[vbatts]

big agree with Derek.

To be clear, I want to see some form of reference types and the relevant APIs in distribution-spec. It seems to me this could be minimal, and therefore a point release (not a major version bump).

The extent to which there are new manifests needing to be defined, and possibly untangled from what is currently in image-spec, is related but slightly separate thread.