YAML 1.2 schemas support (failsafe, JSON, core, custom)

[MOZGIII]

YAML 1.2 defines support for schemas, which, in turn, specify how you'd parse the data.

See https://yaml.org/spec/1.2.2/#chapter-10-recommended-schemas

It would be nice to have a high-level support for those schemas at the parser level.

I'm thinking something like load::<Failsafe>(mystr) (where we have a type-level distinction between the schemas) or load(mystr, schemas::FAILSAFE) (where we have no type-level distinction between the schemas - cause they're a set of tags and means to resolve them maybe it'd work). I wouldn't want an enum-based API load(mystr, YamlSchema::Core) unless it was accompanied with a way to pass your own schema as in load::<MySchema>(mystr, MySchema::default()) - so, the former would really be load::<YamlSchema>(mystr, YamlSchema::Core).

Refs:

• In JS world, this exists: https://github.com/eemeli/yaml; they have support for schemas https://eemeli.org/yaml/#schema-options - although a but crude, since there's no support for custom schemas

[moonheart08]

To extend on this, a safe schema to use for parsing for the likes of serde would be to not do any schema parsing at a YAML level at all. Pre-parsing values by trying to guess their schema is disasterous, and good YAML parsers from other languages (like YamlDotNet) avoid the practice.

In other words, a 'sanity' schema where foo: 1 is "foo": "1", regardless of what the parser thinks it "looks like". This also avoids accidental excess marshalling (i.e. foo: 1 where the underlying schema being parsed really does expect a string, not an integer.)

Using saphyr-parser directly lets you work around this currently, but that's arguably too low level (resolving aliases yourself, for example)

Not being able to do this (ignoring YAML's bad "default" schema, which is supposed to be decided by the program per spec anyway) makes it hard to interop with some real-world applications like https://github.com/space-wizards/space-station-14. Currently interop with RobustToolbox/Space Station 14 is very difficult because of this, combined with the poor handling of custom tags (!type:foo +1.1 can be parsed arbitrarily by an RT application, according to the serializer for that type. Currently parsing this correctly without using saphyr-parser is impossible.)

If Saphyr were to take any opinions, requiring the user to specify their schema up front would be a good one, given the YAML spec's own advice of the fallback schema being for generic yaml-operating tools, not any application.

Utilizing a schema that does not try to guess values when working with serde/etc also greatly improves those APIs (avoids problems like, for example, attempting to encode big integers, where simply stringifying them is entirely reasonable but would break without "quotes" with the current approach.)

[MOZGIII]

Damn, that's sad. YAML spec is quite clear about the layering of the parsing logic and abstractions carried on between them - looks like saphyr doesn't quite follow it if we have such issues...

[bourumir-wyngs]

Do I understand correctly this would require to have YAML schema manually prepared before we can parse the document? How big is the share of users thinking to invest that amount of work? Do you mean that schema description in YAML or JSON, kind of XSD is for XML, could be passed as additional parameter together with the YAML to parse, or do you mean something else?

[moonheart08]

Do I understand correctly this would require to have YAML schema manually prepered before we can parse the document? How big is the share of users thinking to invest that amount of work? Do you mean that schema description in YAML or JSON, kind of XSD is for XML, could be passed as additional parameter together with the YAML to parse, or do you mean something else?

For many cases this actually just means "all scalars are unconditionally treated as strings + metadata for how they were quoted" (do no parsing of the data at all), which then allows frameworks like serde or facet to be responsible for representing and understanding the data contained within. You don't actually need explicit schema files, and managing them seems outside of saphyr's scope.

[bourumir-wyngs]

I was always thinking that "schema support" is something beyond "treat all scalars as strings, but also tell the scalar style (quoted, unquoted) ". In any case if you need this, then saphyr-parser crate does precisely that.

[MOZGIII]

To address the question above: the schemas are not in the sense of a particular document (like what shape/content the document will have). The schema here is in terms of YAML language itself.

From the spec (https://yaml.org/spec/1.2.2/#chapter-10-recommended-schemas):

A YAML schema is a combination of a set of tags and a mechanism for resolving non-specific tags.

This is not about something like XDS / JSON Schema.