index.json: make org.opencontainers.image.ref.name unique?

[cyphar]

While trying to implement index.json parsing in umoci I've come to a bit of a worrying realisation. There isn't really any explanation of how cases where org.opencontainers.ref.name is not unique should be handled. While I understand the case where you have multiple application/vnd.oci.image.manifest.v1+json descriptors with different platform entries (though I think that doesn't make sense in light of the fact that application/vnd.oci.image.index.v1+json has platform entries too), it doesn't really make sense to me in the general case to allow org.opencontainers.ref.names to be duplicated in index.json.

So here's two things that would be nice if we could make org.opencontainers.ref.names unique. This includes the multiple-platform case I mention above because it starts to get a bit hairy -- and honestly you should just be using application/vnd.oci.image.index.v1+json for that.

Sorry to stir this pot again, I've only just hit this issue trying to fix up the oci/cas implementation in umoci.

#533

[qianzhangxa]

While I understand the case where you have multiple application/vnd.oci.image.manifest.v1+json descriptors with different platform entries (though I think that doesn't make sense in light of the fact that application/vnd.oci.image.index.v1+json has platform entries too).

Do we have platform in application/vnd.oci.image.index.v1+json itself? I think we only have it in application/vnd.oci.image.manifest.v1+json descriptor.

[wking]

On Wed, Feb 22, 2017 at 11:36:15PM -0800, Aleksa Sarai wrote: While I understand the case where you have multiple `application/vnd.oci.image.manifest.v1+json` descriptors with different `platform` entries (though I think that doesn't make sense in light of the fact that `application/vnd.oci.image.index.v1+json` has `platform` entries too), it doesn't really make sense to me in the general case to allow `org.opencontainers.ref.name`s to be duplicated in `index.json`.

In #533, @stevvooe and @vbatts explicitly discussed multiple entries with one name but different platform information [1], so you can have: index.json → platform-appropriate-manifest With unique names, you'd need: index.json → multi-platform-index → platform-appropriate-manifest Of course, without clear platform matching (or other metadata-based matching) to break ties, things get a bit awkward. Consumers putting a new platform-agnostic descriptor should have options for “clobber any pre-existing, platform-agnostic refs” or “add a new platform-agnostic ref, even if there is a pre-existing, platform-agnostic ref”. Consumers getting a descriptor can error out with “you asked for $NAME, but there are multiple descriptors matching that name with equivalently appropriate platform information”. To @qianzhangxa's question, here's platform in the index JSON [2], so you can declare a platform for everything your reference from the index. [1]: #533 (comment) [2]: https://github.com/opencontainers/image-spec/blame/v1.0.0-rc5/image-index.md#L41

[stevvooe]

There should be no validation of annotation data.

Either way, the platform dispatch case is why you would want manifests with different names.

@cyphar I may be misunderstanding something. Could you provide an example of the data structures you think should be invalid? application/vnd.oci.image.manifest.v1+json should not have multiple manifests.

[cyphar]

@stevvooe Can you explain how a tool like umoci should handle the following index.json?

{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "size": 7143,
      "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
      "annotations": {
        "org.opencontainers.ref.name": "some-tag"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "size": 8119,
      "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
      "annotations": {
        "org.opencontainers.ref.name": "some-tag"
      }
    }
  ]
}

When a user asks to do something with the some-tag tag?

Even if you say "well, that's not defined by the spec" then how is something like this meant to be handled -- which is something I'm quite concerned about because it starts combining the index.json layer with the ManifestList (now ImageIndex) layer -- combining tagging and multi-platform manifests:

{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.index.v1+json",
      "size": 7143,
      "digest": "sha256:0228f90e926ba6b96e4f39cf294b2586d38fbb5a1e385c05cd1ee40ea54fe7fd",
      "annotations": {
        "org.opencontainers.ref.name": "stable-release"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "size": 8119,
      "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "annotations": {
        "org.opencontainers.ref.name": "stable-release"
      }
    }
  ]
}

And even if you have multiple tags that are all unique platforms (once you parse the tree) -- you're now violating some of the layer separation -- how can you sanely separate the reference evaluation from the ManifestList/Manifest parsing?

[qianzhangxa]

What about we just remove image index from image layout and only have manifest descriptors in index.json? I think the main use case of image index is multi-platform support, but essentially index.json is an image index which already gives us the ability to support multi-platform, so I do not think two-levels image index is necessary. And then I think in an index.json, each org.opencontainers.ref.name + platform must be unique, if there is a manifest descriptor has no platform, it can be just ignored.

[cyphar]

@qianzhangxa Two levels of indirection give you many benefits -- the most obvious of which is that you can have multiple tags that reference the same multi-architecture image. And the multi-architecture image is part of the CAS, which gives you a lot of other benefits.

In my opinion, index.json should be as dumb of a store as is physically possible. While I understand the need for ManifestDescriptors in ManifestList, it doesn't make sense for index.json -- there's no one clear way for how multi-platform images should be implemented.

Not to mention that you've added all of this ambiguity and complicated logic to the reference resolver -- which doesn't make sense from my point of view. It seems to me to be quite a rampant layer violation, and from the umoci side it will make the code much more ugly and frustrating to implement (not to mention that the oci/cas interface and implementation will get pretty ugly too).

The current scheme means that GetReference and PutReference now need to pass a bunch of other information to the CAS store than just the "tag name".

[qianzhangxa]

@cyphar Can you please elaborate a bit more about "multiple tags that reference the same multi-architecture image."? Did you mean the case that two tags (e.g., latest and v2.0) reference the same image index?

[cyphar]

@qianzhangxa Sorry, I misspoke. What I meant was (and if you look at my example you'll see what I mean) was

the most obvious of which is that you can have multiple identical tags with the same name that reference different multi-architecture images.

Basically, the problem with the current state of org.opencontainers.ref.name is that there is no single unambiguous way of deciding what manifest is being referenced by a given tag. Which means that as it stands, tagging is effectively broken.

[stevvooe]

@cyphar In general, the proposal to validate the contents of annotation field is a non-starter. Annotations are functionally meaningless from the point of the view of the specification. It is up to the consumer decide how to present the data. If the tool doesn't reflect the properties of the data structure, then the tool is broken.

umoci can make its own determination about how to handle this case. One approach is to linearize the descriptors that match some-tag with an in-order traversal and then run the standard platform matching algorithm on that ordered set. On duplicates, you take the first entry that matches. Another approach would be to error out on duplicates.

You'll actually find that the algorithm proposed above is both sufficient and deterministic in resolving multi-platform images, even cases in where duplicate or ambiguous tags are present.

[cyphar]

@stevvooe But we do define what are valid values for annotations. Annotations in the org.opencontainers.* namespace are reserved by us and we have the right to specify how they should be used. Nothing is stopping a third party from making their own tagging system with annotations.

One approach is to linearize the descriptors that match some-tag with an in-order traversal and then run the standard platform matching algorithm on that ordered set.

What is the "standard platform matching algorithm" -- and how am I meant to run said algorithm if umoci is not running on the platform which I'm extracting (do I have to copy the entire contents of /proc/cpuinfo in order for it to be made clear)? The current scheme combines two very separate components of CAS operations -- reference resolution and blob/manifest parsing.

But all of this is besides the point -- why are we allowing multiple tags in this instance when you can just create a single tag which points to an ImageIndex that then shards up the multi-platform stuff. Is there some benefit I'm missing, because the current system strikes me to be significantly less elegant than if we'd just taken the refs directory and mapped it directly to json (as opposed to overloading the use of ManifestList to become ImageIndex).

You'll actually find that the algorithm proposed above is both sufficient and deterministic in resolving multi-platform images, even cases in where duplicate or ambiguous tags are present.

It might be both sufficient and deterministic, but that doesn't make it sensible when you could just drop this ambiguity in the first place.

Also, the algorithm you propose is not defined in the spec. So how is a user meant to know how their image will be understood by an image-spec implementation?

[cyphar]

And to make the point more clear, the current CAS interface for umoci is this:

// Engine is an interface that provides methods for accessing and modifying an
// OCI image, namely allowing access to reference descriptors and blobs.
type Engine interface {
	// PutReference adds a new reference descriptor blob to the image. This is
	// idempotent; a nil error means that "the descriptor is stored at NAME"
	// without implying "because of this PutReference() call". ErrClobber is
	// returned if there is already a descriptor stored at NAME, but does not
	// match the descriptor requested to be stored.
	PutReference(ctx context.Context, name string, descriptor ispec.Descriptor) (err error)

	// GetReference returns a reference from the image. Returns os.ErrNotExist
	// if the name was not found.
	GetReference(ctx context.Context, name string) (descriptor ispec.Descriptor, err error)

	// DeleteReference removes a reference from the image. This is idempotent;
	// a nil error means "the content is not in the store" without implying
	// "because of this DeleteReference() call".
	DeleteReference(ctx context.Context, name string) (err error)

	// ListReferences returns the set of reference names stored in the image.
	ListReferences(ctx context.Context) (names []string, err error)

	// ... other methods omitted for brevity ...

How do you propose that I change the ListReferences, PutReference, DeleteReference and GetReference interfaces to facilitate this ambiguity and platform matching code? I have no problem with switching away from Descriptors to ManifestDescriptors (though I also don't agree with the need for that change either, but let's just side-step that issue for now), but the required changes to now pass around some opaque concept of "what platform I'm using" doesn't make sense to me.

[qianzhangxa]

the most obvious of which is that you can have multiple identical tags with the same name that reference different multi-architecture images.

@cyphar But why is this a benefit? Won't cause ambiguity? E.g., if there are two same tags in the index.json reference two different image index, then for the runtime to pull the image based on that tag, which one should it pull?

[cyphar]

@qianzhangxa It isn't a benefit, my point is that the current system allows this and it shouldn't. That's what this issue is about, removing this ambiguity.