Dealing with raw json

[valentin-panalyt]

I have a question similar to #306 and I wonder if there is a better solution to that after 5 years.

Basically, sometimes I get a json structure that contains a field (let's call it raw_payload) that contains more raw json without a schema. I want to just forward that raw_payload and store it in the database and/or return it in my API. I don't care what exact json is inside that field, but it needs to be valid json.

Example 1:

{
    "foo": 1,
    "bar": 2,
    "raw_payload": [{"random": "stuff"}]
}

Example 2:

{
    "foo": 1,
    "bar": 2,
    "raw_payload": {"even": {"more": "random", "stuff": [1,2,3] } }
}

I'd like to map that into case class Data(foo: Int, bar: Int, raw_payload: SomeRawJsonType)

What's currently the best approach to achieve that?

[plokhotnyuk]

Yes, currently the approach is the same: you need a data structure that will hold bytes of raw JSON and a custom codec that will use JsonReader.readRawValAsBytes and JsonWriter.writeRawVal, like here. If you need a validation you can "reparse" the array with raw JSON bytes using readFromArrayReentrant immediately in your custom codec or later after parsing of your whole data structure using just readFromArray. The codec for validation is here.

The best option would be adding JsonReader.validate (similar to JsonReader.skip but with validation of whole value according to RFC-8259) and JsonReader.readRawValAsBytesWithValidation (with the same implementation as JsonReader.readRawValAsBytes except calling of JsonReader.validate instead of JsonReader.skip) routines.

Would you like to contribute these enhancements?

Your input could make a significant difference when reused by an additional compile-time configuration option for macros to validate all skipped fields and make parsing even more safer for everyone.