Add markup as designed (#574)

* Add markup as designed

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>

* Apply suggestions from code review

* Add separate markup-standalone schema in messages.json

* Apply suggestions from code review

Co-authored-by: Richard Gibson <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Richard Gibson <[email protected]>

Author: 4 people

spec/data-model/README.md

@@ -117,31 +117,40 @@ String values include all processing of the underlying _text_ values,
 including escape sequence processing.
 `Expression` values wrap each of the _expression_ shapes.
 
-Implementations MUST NOT rely on the set of `Expression` interfaces being exhaustive,
-as future versions of this specification MAY define additional expressions.
+Implementations MUST NOT rely on the set of `Expression` and 
+`Markup` interfaces defined in this document being exhaustive.
+Future versions of this specification might define additional
+expressions or markup.
 
 ```ts
 type Pattern = Array<string | Expression>;
 
-type Expression = LiteralExpression | VariableExpression | FunctionExpression |
-                  UnsupportedExpression;
+type Expression =
+  | LiteralExpression
+  | VariableExpression
+  | FunctionExpression
+  | UnsupportedExpression;
 
 interface LiteralExpression {
+  type: "expression";
   arg: Literal;
   annotation?: FunctionAnnotation | UnsupportedAnnotation;
 }
 
 interface VariableExpression {
+  type: "expression";
   arg: VariableRef;
   annotation?: FunctionAnnotation | UnsupportedAnnotation;
 }
 
 interface FunctionExpression {
+  type: "expression";
   arg?: never;
   annotation: FunctionAnnotation;
 }
 
 interface UnsupportedExpression {
+  type: "expression";
   arg?: never;
   annotation: UnsupportedAnnotation;
 }
@@ -172,17 +181,13 @@ interface VariableRef {
 ```
 
 A `FunctionAnnotation` represents a _function_ _annotation_.
-In a `FunctionAnnotation`,
-the `kind` corresponds to the starting sigil of a _function_:
-`'open'` for `+`, `'close'` for `-`, and `'value'` for `:`.
-The `name` does not include this starting sigil.
+The `name` does not include the `:` starting sigil.
 
 Each _option_ is represented by an `Option`.
 
 ```ts
 interface FunctionAnnotation {
   type: "function";
-  kind: "open" | "close" | "value";
   name: string;
   options?: Option[];
 }
@@ -214,11 +219,44 @@ that the implementation attaches to that _annotation_.
 ```ts
 interface UnsupportedAnnotation {
   type: "unsupported-annotation";
-  sigil: "!" | "@" | "#" | "%" | "^" | "&" | "*" | "<" | ">" | "/" | "?" | "~";
+  sigil: "!" | "@" | "%" | "^" | "&" | "*" | "+" | "<" | ">" | "?" | "~";
   source: string;
 }
 ```
 
+## Markup
+
+A `Markup` object is either `MarkupOpen`, `MarkupStandalone`, or `MarkupClose`,
+which are differentiated by `kind`.
+The `name` in these does not include the starting sigils `#` and `/` 
+or the ending sigil `/`.
+The optional `options` for open and standalone markup use the same `Option`
+as `FunctionAnnotation`.
+
+```ts
+type Markup = MarkupOpen | MarkupStandalone | MarkupClose;
+
+interface MarkupOpen {
+  type: "markup";
+  kind: "open";
+  name: string;
+  options?: Option[];
+}
+
+interface MarkupStandalone {
+  type: "markup";
+  kind: "standalone";
+  name: string;
+  options?: Option[];
+}
+
+interface MarkupClose {
+  type: "markup";
+  kind: "close";
+  name: string;
+}
+```
+
 ## Extensions
 
 Implementations MAY extend this data model with additional interfaces,

spec/data-model/message.dtd

@@ -21,7 +21,7 @@
 <!ELEMENT key (#PCDATA)>
 <!ATTLIST key default (true | false) "false">
 
-<!ELEMENT pattern (#PCDATA | expression)*>
+<!ELEMENT pattern (#PCDATA | expression | markup)*>
 
 <!ELEMENT expression (
   ((literal | variable), (functionAnnotation | unsupportedAnnotation)?) |
@@ -35,12 +35,17 @@
 <!ATTLIST variable name NMTOKEN #REQUIRED>
 
 <!ELEMENT functionAnnotation (option)*>
-<!ATTLIST functionAnnotation
-  kind (open | close | value) #REQUIRED
-  name NMTOKEN #REQUIRED
->
+<!ATTLIST functionAnnotation name NMTOKEN #REQUIRED>
+
 <!ELEMENT option (literal | variable)>
 <!ATTLIST option name NMTOKEN #REQUIRED>
 
 <!ELEMENT unsupportedAnnotation (#PCDATA)>
 <!ATTLIST unsupportedAnnotation sigil CDATA #REQUIRED>
+
+<!-- A <markup kind="close"> MUST NOT contain any <option> elements -->
+<!ELEMENT markup (option)*>
+<!ATTLIST markup
+  kind (open | standalone | close) #REQUIRED
+  name NMTOKEN #REQUIRED
+>

spec/data-model/message.json

@@ -40,12 +40,8 @@
       "type": "object",
       "properties": {
         "type": { "const": "function" },
-        "kind": { "enum": ["open", "close", "value"] },
         "name": { "type": "string" },
-        "options": {
-          "type": "array",
-          "items": { "$ref": "#/$defs/option" }
-        }
+        "options": { "$ref": "#/$defs/options" }
       },
       "required": ["type", "kind", "name"]
     },
@@ -70,32 +66,36 @@
     "literal-expression": {
       "type": "object",
       "properties": {
+        "type": { "const": "expression" },
         "arg": { "$ref": "#/$defs/literal" },
         "annotation": { "$ref": "#/$defs/annotation" }
       },
-      "required": ["arg"]
+      "required": ["type", "arg"]
     },
     "variable-expression": {
       "type": "object",
       "properties": {
+        "type": { "const": "expression" },
         "arg": { "$ref": "#/$defs/variable" },
         "annotation": { "$ref": "#/$defs/annotation" }
       },
-      "required": ["arg"]
+      "required": ["type", "arg"]
     },
     "function-expression": {
       "type": "object",
       "properties": {
+        "type": { "const": "expression" },
         "annotation": { "$ref": "#/$defs/function-annotation" }
       },
-      "required": ["annotation"]
+      "required": ["type", "annotation"]
     },
     "unsupported-expression": {
       "type": "object",
       "properties": {
+        "type": { "const": "expression" },
         "annotation": { "$ref": "#/$defs/unsupported-annotation" }
       },
-      "required": ["annotation"]
+      "required": ["type", "annotation"]
     },
     "expression": {
       "oneOf": [
@@ -106,6 +106,43 @@
       ]
     },
 
+    "markup-open": {
+      "type": "object",
+      "properties": {
+        "type": { "const": "markup" },
+        "kind": { "const": "open" },
+        "name": { "type": "string" },
+        "options": { "$ref": "#/$defs/options" }
+      },
+      "required": ["type", "kind", "name"]
+    },
+    "markup-standalone": {
+      "type": "object",
+      "properties": {
+        "type": { "const": "markup" },
+        "kind": { "const": "standalone" },
+        "name": { "type": "string" },
+        "options": { "$ref": "#/$defs/options" }
+      },
+      "required": ["type", "kind", "name"]
+    },
+    "markup-close": {
+      "type": "object",
+      "properties": {
+        "type": { "const": "markup" },
+        "kind": { "const": "close" },
+        "name": { "type": "string" }
+      },
+      "required": ["type", "kind", "name"]
+    },
+    "markup": {
+      "oneOf": [
+        { "$ref": "#/$defs/markup-open" },
+        { "$ref": "#/$defs/markup-standalone" },
+        { "$ref": "#/$defs/markup-close" }
+      ]
+    },
+
     "pattern": {
       "type": "array",
       "items": {

spec/formatting.md

@@ -12,7 +12,7 @@ their handling during formatting is specified here as well.
 
 Formatting of a _message_ is defined by the following operations:
 
-- **_Expression Resolution_** determines the value of an _expression_,
+- **_Expression and Markup Resolution_** determines the value of an _expression_ or _markup_,
   with reference to the current _formatting context_.
   This can include multiple steps,
   such as looking up the value of a variable and calling formatting functions.
@@ -83,9 +83,10 @@ At a minimum, it includes:
 
 Implementations MAY include additional fields in their _formatting context_.
 
-## Expression Resolution
+## Expression and Markup Resolution
 
 _Expressions_ are used in _declarations_, _selectors_, and _patterns_.
+_Markup_ is only used in _patterns_.
 
 In a _declaration_, the resolved value of the _expression_ is bound to a _variable_,
 which is available for use by later _expressions_.
@@ -98,7 +99,7 @@ but also the _variable_ to which the resolved value of the _variable-expression_
 
 In _selectors_, the resolved value of an _expression_ is used for _pattern selection_.
 
-In a _pattern_, the resolved value of an _expression_ is used in its _formatting_.
+In a _pattern_, the resolved value of an _expression_ or _markup_ is used in its _formatting_.
 
 The shapes of resolved values are implementation-dependent,
 and different implementations MAY choose to perform different levels of resolution.
@@ -210,17 +211,7 @@ the following steps are taken:
    Implementations are not required to implement _namespaces_ or installable
    _function registries_.
 
-3. Resolve the _options_ to a mapping of string identifiers to values.
-   If _options_ is missing, the mapping will be empty.
-   For each _option_:
-   - Resolve the _identifier_ of the _option_.
-   - If the _option_'s _identifier_ already exists in the resolved mapping of _options_,
-     emit a Duplicate Option Name error.
-   - If the _option_'s right-hand side successfully resolves to a value,
-     bind the _identifier_ of the _option_ to the resolved value in the mapping.
-   - Otherwise, bind the _identifier_ of the _option_ to an unresolved value in the mapping.
-     Implementations MAY later remove this value before calling the _function_.
-     (Note that an Unresolved Variable error will have been emitted.)
+3. Perform _option resolution_.
 
 4. Call the function implementation with the following arguments:
 
@@ -247,6 +238,37 @@ the following steps are taken:
    If the call fails or does not return a valid value,
    emit a Resolution error and use a _fallback value_ for the _expression_.
 
+#### Option Resolution
+
+The result of resolving _option_ values is a mapping of string identifiers to values.
+
+For each _option_:
+
+- Resolve the _identifier_ of the _option_.
+- If the _option_'s _identifier_ already exists in the resolved mapping of _options_,
+   emit a Duplicate Option Name error.
+- If the _option_'s right-hand side successfully resolves to a value,
+   bind the _identifier_ of the _option_ to the resolved value in the mapping.
+- Otherwise, bind the _identifier_ of the _option_ to an unresolved value in the mapping.
+   Implementations MAY later remove this value before calling the _function_.
+   (Note that an Unresolved Variable error will have been emitted.)
+
+Errors MAY be emitted during _option resolution_,
+but it always resolves to some mapping of string identifiers to values.
+This mapping can be empty.
+
+### Markup Resolution
+
+Unlike _functions_, the resolution of _markup_ is not customizable.
+
+The resolved value of _markup_ includes the following fields:
+
+- The type of the markup: open, standalone, or close
+- The _identifier_ of the _markup_
+- For _markup-open_ and _markup_standalone_,
+  the resolved _options_ values after _option resolution_.
+
+The resolution of _markup_ MUST always succeed.
 
 ### Fallback Resolution
 
@@ -304,7 +326,7 @@ The _fallback value_ depends on the contents of the _expression_:
   > the message formats to `{$arg}`.
 
 - _function_ _expression_ with no _operand_:
-  the _function_ starting sigil followed by its _identifier_
+  U+003A COLON `:` followed by the _function_ _identifier_
 
   > Examples:
   > In a context where `:func` fails to resolve, `{:func}` resolves to the _fallback value_ `:func`.
@@ -599,7 +621,7 @@ one {{Category match}}
 ## Formatting
 
 After _pattern selection_,
-each _text_ and _expression_ part of the selected _pattern_ is resolved and formatted.
+each _text_ and _placeholder_ part of the selected _pattern_ is resolved and formatted.
 
 _Formatting_ is a mostly implementation-defined process,
 as it depends on the implementation's shape for resolved values
@@ -618,13 +640,18 @@ appropriate data type or structure. Some examples of these include:
 - A string with associated attributes for portions of its text.
 - A flat sequence of objects corresponding to each resolved value.
 - A hierarchical structure of objects that group spans of resolved values,
-  such as sequences delimited by "open" and "close" _function_ _annotations_.
+  such as sequences delimited by _markup-open_ and _markup-close_ _placeholders_.
 
 Implementations SHOULD provide _formatting_ result types that match user needs,
 including situations that require further processing of formatted messages.
 Implementations SHOULD encourage users to consider a formatted localised string
 as an opaque data structure, suitable only for presentation.
 
+When formatting to a string, the default representation of all _markup_
+MUST be an empty string.
+Implementations MAY offer functionality for customizing this,
+such as by emitting XML-ish tags for each _markup_.
+
 ### Examples
 
 _This section is non-normative._