index.json: improve implementors note to better represent algorithms necessary

[cyphar]

As discussed in the OCI call, currently the implementors note for index.json doesn't fully encapsulate the reason for the more complex Index data structure and the ways in which implementations should handle the n-tall tree that it can represent.

The main points that need to be clarified are:

• That the UX for selecting what manifest is being operated on is not defined by the spec, and that implementations are free to either fail when they can't uniquely resolve a manifest, or provide some sort of indication to the user that

• The platform matching / annotation matching algorithm necessary to match and flatten the tree structure of descriptors and then reduce the set of descriptors to those that match a particular query. The SQL analogy would probably be something we should mention.

@stevvooe Stepped up to work on this one.

[stevvooe]

Based on my research over the last week, I think we can address this by pulling up the platform descriptions into the descriptor to get rid of the type hierarchy of descriptors which is awkward. This is a Go type change and a slight reorganization of the types definition which is completely compatible, but it would be nice to have that correct for 1.0.

Let me know what you think about this or if more details are needed.

[cyphar]

I agree with moving platform to descriptor annotations (now that we have them). Though we still would need to add some implementor notes about how they are expected to handle the hierarchy of annotations that you could run into and that the recommended referencing implementation would involve effectively tag matching (more of a search engine and less of a simple dereferencer).

I've been working on this on the umoci side and I'll admit it's quite difficult coming up with an interface that is symmetric when it comes to referencing "from" and "to" an image (you are effectively binding the interface quite tightly with the concept of annotations which is a spec detail that I'm not sure users should care about). But we'll see how it goes.

[stevvooe]

I agree with moving platform to descriptor annotations (now that we have them).

I think we'd have to support both platform field and annotations, in practice, but the simplification seems straightforward.

I've been working on this on the umoci side and I'll admit it's quite difficult coming up with an interface that is symmetric when it comes to referencing "from" and "to" an image (you are effectively binding the interface quite tightly with the concept of annotations which is a spec detail that I'm not sure users should care about). But we'll see how it goes.

I'll see if I can look deeper into platform qualified pull to give some better guidance here, as promised.

[qianzhangxa]

@stevvooe Did you mean adding Platform into Descriptor as its field and then we do not need ManifestDescriptor anymore?

And could you please provide more details about why it can address this issue (e.g., how to handle duplicated org.opencontainers.ref.name in index.json)?

[xiekeyang]

@cyphar

more complex Index data structure

Could you please put up some possible examples of complex index data struct? Actually I'm interest the experimental usage of index.

That the UX for selecting what manifest is being operated on is not defined by the spec, and that implementations are free to either fail when they can't uniquely resolve a manifest, or provide some sort of indication to the user that

If it means the contents(config+layers) in one manifest might be not operable? Instead, manifests in one index can be combined and run a container according to customized order?

[stevvooe]

@stevvooe Did you mean adding Platform into Descriptor as its field and then we do not need ManifestDescriptor anymore?

Yes.

And could you please provide more details about why it can address this issue (e.g., how to handle duplicated org.opencontainers.ref.name in index.json)?

This can help with the issue by avoid having to deal with different descriptor types, which was never intended by this specification.

As far as an example of a ux that deals with multiple reference, let's say we have a tool that lists the references in a image layout:

$ oci image references foo.tar.gz
REF    PLATFORM     TYPE                                                 DIGEST      
foo    arm64/linux  application/vnd.oci.image.manifest.v1+json           sha256:ffff
foo    amd64/linux  application/vnd.oci.image.manifest.v1+json           sha256:aa

Then we can unpack with something like this:

$ oci image unpack target/ foo foo.tar.gz
error: reference foo is ambiguous

Now, just qualify the reference:

$ oci image unpack --platform.arch arm64  target/ foo foo.tar.gz

Or:

$ oci image unpack target/ foo@sha256:ffff foo.tar.gz

[qianzhangxa]

Thanks @stevvooe for the example!

And I am thinking another case:

$ oci image references foo.tar.gz
REF    PLATFORM     TYPE                                                 DIGEST      
foo    N/A          application/vnd.oci.image.index.v1+json              sha256:ffff
foo    amd64/linux  application/vnd.oci.image.manifest.v1+json           sha256:aa

In this case, what will be the expected behavior of oci image unpack target/ foo foo.tar.gz? Should it fail with error: reference foo is ambiguous too? Or it should go with the first foo whose media type is application/vnd.oci.image.index.v1+json.

[stevvooe]

In this case, what will be the expected behavior of oci image unpack target/ foo foo.tar.gz? Should it fail with error: reference foo is ambiguous too? Or it should go with the first foo whose media type is application/vnd.oci.image.index.v1+json.

I think you would need to unroll the references in the targeted index. Perhaps, we could print that as a tree?

[qianzhangxa]

Do we really need to allow multiple descriptors with the name but with different media types in an image layout? I think it is definitely OK if there are multiple manifest descriptors with the same name but for different platforms. But if there is already an index descriptor which is for multiple platforms, why should there be a manifest descriptor with the same name for a specific platform? I think the later should be part of the former rather than part of index.json.