Consider alternatives to jsonschema for validation

[jsoriano]

We are using jsonschema for validation, one of the targets was to try to define a spec that can be easily validated in other languages that have support for jsonschema, but we have needed several customizations that required more Go code, making difficult to use the spec in other languages.

Some options we have to solve the issues found can be the following ones, we probably need a mix of them to address all issues:

• Use some other framework for validation of directory structure.
• Use a more extensible jsonschema library for validation of json/yaml files.
• Fully replicate the same logic in other languages, and implement a solution to certify the different implementations.
• Expose spec validation functionality through a CLI tool.
• Expose spec validation functionality through some portable target such as WebAssembly (POC).

Limitations found:

• jsonschema is good for the structure of json/yaml files, but we also try to use something similar for the structure of directories. This validation of directories requires additional custom logic.
• We need some custom validation of fields in yaml files that go beyond the basic json types, and this is tricky to implement with current library (for example we check that relative paths exist and comply with size limits using a custom formatter, but this is not about format, and formatters cannot be parameterized).
• Current library doesn't have good support for virtual filesystems, what produces issues on Windows, Fix package validation on Windows #358.
• "Semantic" validation requires additional completely custom code.
• We are using jsonschema to validate yaml files, this works fine by now, but there may be tricky cases, and this may limit portability of the spec.
• Custom code is not portable to other languages.

Other things to take into account:

• When reusing the same code in different places, we may need to be less strict on some checks. For example elastic-package can be strict on additionalContents, but if Kibana is strict on that, new packages may not work in old versions of Kibana.

[jsoriano]

https://cuelang.org/ could be an option if we consider to refactor the spec, thanks @florianl for the suggestion!

[jsoriano]

Giving a try to CUE in #391 and https://github.com/jsoriano/cuetest I am finding some problems, summarized in the description of #391.

After these experiments I am starting to consider that maybe it is worth to continue with our variation of jsonschema. We have already needed to fork the library we use (https://github.com/elastic/gojsonschema), we might continue adding features in this fork.

[jsoriano]

I am going to close this by now. We have invested more in jsonschema since I opened this issue:

• Refactors to more clearly separate the code between yaml/jsonschema, and the spec for directories make it clearer that a single solution wouldn't be enough.
• We have forked the gojsonschema library, we can make changes there as we did to better support fs.FS.
• Tried different approaches, like using CUE, but it also had its problems.
• Added support for "patches" to support multiple versions of the spec.