ion_rs::serde::ser::ValueSerializer to be more type-preserving?

[barries]

I'm not sure if this is in-scope, but our product and team would get a lot of value out of being able to reify ion_rs::serde-serialized values with almost complete type information. This would allow us to write values or log streams in ion or i0n format and, for example, (a) pretty-print them in a rusty format, (b) allow analysts and diagnosticians to deserialize them into Rust, Lua, Javascript, Python, etc. generic "object DOM"s or application-specific datatypes with higher fidelity, (c) import them into ad-hoc databases with schemas very true to the originating program's, (d) convert them to other type-preserving formats like YAML w/ tags (internal use only, lol--no desire for more RoR security fails), or even (e) process them with something like a future Ion-extended/ported-to-Ion jaq or spath.

My current use case is to allow readers--human and automated--of our log files (via ion cat, pretty printed in a rusty format, or deserialized into their programming or database platform of choice) to reliably discriminate the types in useful ways. Both the type names and having a consistent structure of the values can be very helpful. For example discriminating between things like TupleStruct(1, 1) vec!(1, 2), [1, 2], and (1, 1) helps them map what they see in the log to the actual code and data in our app; that would increase speed of comprehension, reduce misinterpretation, and avoid errors (especially when deserializing in programs or to import into databases) due to unexpected corner cases.

The present (circa 1.0.0-rc.11) ValueSerializer is so very close to being type-preserving enough and I think it would provide a lot more value with a slight embellishment-via-annotations. If that's too radical, especially if it breaks existing consumers depending on pre-1.0.0 ion_rs code, perhaps an alternative, more type-preserving ion_rs::serde::ser serializer, in the spirit of ron, might be widely useful.

Here are two possible ways of doing it, depending on how to discriminate tuples vs arrays. I don't particularly like either the extra tuple:: annotation or the subverting of the sexpr's executable data connotation into a plain old data type; both feel a bit hacky to me, but I don't see a third way. There are probably better ones (forgive any minor errors--I've not prototyped this, and not all of this is necessarily feasible given Serde's peculiarities, so YMMV).

Rust	Ion (tuple::)	Ion (sexpr)
()	null	null
1	1	1
"a"	"a"	"a"
"a".to_string()	String::"a"	String::"a"
[]	[]	[]
(1u32, ...)	tuple::[1, ...]	(1 ...)
[1u32, ...]	[1, ...]	[1, ...]
MyUnitStruct()	MyUnitStruct::null	MyUnitStruct::null
MyNewTypeStruct(1)	MyNewTypeStruct::1	MyNewTypeStruct::1
MyNewTypeStruct((1))	MyNewTypeStruct::[tuple::[1]]	MyNewTypeStruct::[(1)]
MyTupleStruct(1, ...)	MyTupleStruct::tuple::[1, ...]	MyTupleStruct::(1 ...)
MyEmptyStruct{}	MyEmptyStruct::{}	MyEmptyStruct::{}
MyStruct{ a: 1, ...}	MyStruct::{a: 1, ...}	MyStruct::{a: 1, ...}
MyStruct{ a: MyNewTypeStruct(1), ...}	MyStruct::{a: MyNewTypeStruct::1, ...}	MyStruct::{a: MyNewTypeStruct::1, ...}
MyEnum::MyVariant	MyEnum::'MyVariant'	MyEnum::'MyVariant'
MyEnum::MyUnitVariant()	MyEnum::MyUnitVariant::null	MyEnum::MyUnitVariant::null
MyEnum::MyNewTypeVariant(1)	MyEnum::MyNewTypeVariant::1	MyEnum::MyNewTypeVariant::1
MyEnum::MyTupleVariant(1, ...)	MyEnum::MyTupleVariant::tuple::[1, ...]	MyEnum::MyTupleVariant::(1 ...)
MyEnum::MyEmptyStructVariant{}	MyEnum::MyEmptyStructVariant::{}	MyEnum::MyEmptyStructVariant::{}
MyEnum::MyStructVariant{a: 1, ...}	MyEnum::MyStructVariant::{ a: 1, ...}	MyEnum::MyStructVariant::{ a: 1, ...}

I'm not quite sure how Option<>s should work--should they be special, because serde allows them to be and an increasing number of languages support them as vocabulary types, or should they be consistent with other enums (including Result<>, which are as endemic as Opion<>s but not treated specially in Serde--but erring on the side of consistent, perhaps:

Rust	Ion (tuple::)	Ion (sexpr)
'None`	Option::null	Option::null
Some("a")	Option::Some::["a"]	Option::Some::["a"]
Some("a".to_string())	Option::Some::[String::"a"]	Option::Some::[String::"a"]
Some(())	Option::Some::null	Option::Some::null
Some((1))	Option::Some::[tuple::1]	Option::Some::(1)
Some((1, 2))	Option::Some::[tuple::[1, 2]]	Option::Some::(1 2)
Some(MyStruct(1, 2))	Option::Some::[MyStruct::[1, 2]]	Option::Some::[MyStruct::[1, 2]]
Some(MyStruct{ a: 1, ...})	Option::Some::[MyStruct::{ a: 1, ...}]	Option::Some::[MyStruct::{ a: 1, ...}]

[zslayton]

This is good feedback, thanks! The current ValueSerializer aims to make round-tripping data from Rust -> Ion -> Rust straightforward without being overly verbose. That said, I can understand wanting the ability to opt into a maximally explicit serde flavor. I'm not confident it would be easy to do this via a config option in the existing ValueSerializer; I suspect the best way to achieve it would be creating a second implementation of serde. Unfortunately, I don't think this is something we'll be able to tackle in very near future. If you'd like to take a crack at it, I'd be happy to advise.

[barries]

I'll try copying ValueSerializer to the caller side of life and seeing what potholes exist between it and the above.