spec: define charset for "org.opencontainers.ref.name"

[stevvooe]

Currently, there is no restriction on the character set for reference names. We'll need something open but define some common bounds to make this.

I suggested restricting this to the URI character set, in #591 (comment).

Problem called out in #591.

[stevvooe]

cc @q384566678

[vbatts]

cc @alexlarsson, would the URI character set work for you as well?

[jonboulle]

To be very explicit, is this a change in direction from the position expressed here and here?

[stevvooe]

@jonboulle Not exactly and there is some subtlety here.

From the perspective of the specification, annotations are opaque. They will be read and propagated without modification or verification.

This does not mean that the specifier of the annotation can't make requirements that a given annotation has a structure that is internal to that annotation. Practically, this means that when an annotation producer or consumer reads an annotation, it may be generated or validated in a specified format.

Let me know if this distinction is unclear. It seems like splitting hairs, but it is to avoid having the annotations "leak" into the first-class field space.

[zhouhao3]

I think that as long as the string type of standard can be, do not need more norms.

[erikh]

https://en.m.wikipedia.org/wiki/Punycode Is a method of encoding non-latin encodings into urls in a way that's easy to both parse and display in all instances. Maybe that's a good solution since it has a limited input set?

[erikh]

It's used in many internationalized urls, so it's not just a spec for spec's sake. :D

[erikh]

https://godoc.org/golang.org/x/net/idna Is a golang library to manage the conversion.

[AkihiroSuda]

How to tell punycode string from ASCII string?
(Is is valid to use "xn--" prefix for non-domain strings?)

Why not use percent-encoded UTF-8?

[erikh]

no argument from me re: percent encoded utf-8

[erikh]

So I did some reading tonight and discovered these. With the punycode suggestion I was attempting to find a word-safe set of characters to use and realized later that the unicode standard already does this for us elsewhere. :)

http://unicode.org/reports/tr18/#Categories

Which are supported by most regex implementations these days, since unicode classes are well-defined.

See also: https://golang.org/pkg/unicode/

We could use subsets of these supported regex classes to define our character set; the reason I suggest this is that it'd be an insanely painful exercise to satisfy all needs here by defining it manually.

So, presuming this is a suitable solution, we really just need to cherry-pick classes from the listings. I would start the discussion with digits+alpha+marks, maybe. No strong opinions myself, and would love to hear from people actually storing/indexing by name.

[erikh]

while I'm here, I think it might be useful to have annotations (either as struct tags in code that are managed or just in the spec) that define this clearly for all named string fields in the spec, and also for general unnamed sections (like annotations in the spec).

e.g., the digests should have a charset, the chainIDs, the tags, etc.