Disallow whitespace as the first character of a reserved-body in a reserved-statement.

[bhaible]

In the 'reserved-statement' nonterminal, there is an ambiguity if there is more than one whitespace character between the 'reserved-keyword' and the first non-whitespace character of the 'reserved-body', because these whitespace characters can be seen as part of the 's' nonterminal or as part of the 'reserved-body' nonterminal.

According to the principles explained in #725 and the proposed resolution of #721, it is not desired that a 'reserved-body' starts with a whitespace character; rather, such a whitespace character is meant to be interpreted as part of the preceding 's' nonterminal.

Test case:

.regex   /foo/{xyz}{{hello}}

This patch removes this ambiguity, by disallowing whitespace as the first character of a 'reserved-body' in a reserved-statement.

It thus fixes the first part of #721.

Details:

• In the other occurrences of 'resolved-body' as well (in a 'reserved-annotation' or 'private-use-annotation') the leading whitespace is separated as well. This has no influence on the set of inputs that the 'reserved-annotation' and 'private-use-annotation' nonterminals can match, but highlights that the parser should better trim off this leading whitespace in these places before entering the 'resolved-body' into the data model.
• Two nonterminals 'reserved-body-start' and 'resolved-body-part' are introduced, each referenced once. The purpose is clarity and to follow the common *-start / *-part idiom.

[eemeli]

If we're going to encode in the syntax that the value of reserved-body does not include its leading whitespace, then this seems like a better way of doing so than #730.

Approving, but note my inline suggested simplification, which ought to be applied syntax.md as well.

This does introduce one concern that should be noted:

If someone introduces private syntax {&foo} for which they do not allow for space between the & and foo, a standard MF2 processor, when encountering a privately invalid expression {& foo}, could reserialize it as {&foo}, a privately valid expression.

Similarly, this change would effectively disallow private and future syntax from requiring a space immediately after the initial sigil, as an MF2.0 processor would likely dismiss the space as insignificant.

I'm okay with accepting these limitations.

[eemeli]

This can be simplified:

Suggested change

	reserved-body = reserved-body-start *reserved-body-part
	reserved-body-start = reserved-char / reserved-escape / quoted
	reserved-body-part = [s] 1*(reserved-char / reserved-escape / quoted)
	reserved-body = reserved-body-part *([s] reserved-body-part)
	reserved-body-part = reserved-char / reserved-escape / quoted

[bhaible]

Thanks for the suggestion. I like it, and I'll apply it if there is consensus. (I had introduced the reserved-body-start nonterminal following @aphillips comment #730 (comment) .)

[aphillips]

Let's apply this here and below.

[catamorphism]

The grammar change looks good. It took me ages to understand what this is doing; maybe I'm just slow, but I think it might help future readers to add a note in syntax.md as I suggested.

[catamorphism]

I think it's worth adding some prose here explaining why the grammar is structured this way. It wasn't obvious immediately that it had to be, so it won't be obvious to the reader either. Including the example would be good.

I also think a note should be added to the "Reserved Statements" section of the same doc that explains what the limitations are of reserved-statement that arise because of this change. You could use your example from #721.

[aphillips]

Let's use a separate PR to do any text changes.

[aphillips]

I think the group is settled on this one over #730. Let's close #730 and concentrate on finishing this one off.

Adding a note to syntax.md is probably useful.

[bhaible]

The simplification suggested by @eemeli is now included.