Roadmap

[Stranger6667]

This is a somewhat detailed roadmap for the development of this crate, as I'd like to be transparent about what changes the user may anticipate. I will update it roughly once a month with all the main milestones.

This plan is not set in stone, and some other features could be implemented outside of these milestones (e.g., moving some dependencies behind features).

❤️ If you'd like to help with the development, consider sponsoring me (like Sentry does)

Rework Compiler

Right now, Registry owns all documents, which implies moves / clones to user-provided documents + uses some unsafe for self-references. However, if Registry is split into storage + resolution context, then user-provided documents could be borrowed (so the registry will only store fetched ones), + unsafe is gone.

Additionally, it will help with ValidationOptions, so it also does not hold owned registry / resources. They are only needed during compilation, so there is no need to own them (unless it is fetched via a retriever).

Schema clones during compilation should be gone by this point.

Generic JSON

Reworking the compiler will simplify direct integration of PyO3 data structures into the compilation process without the need to convert them into serde_json::Value (it is way easier to just borrow things for the compilation time). Though this area requires a bit more refining, generic JSON input would be easier to implement after this step.

In Python bindings, sometimes the overhead data (de)-serialization during compiling a validator / validation is up to 80%, and having generic input will greatly speed up everything.

Website

As I did it with css-inline.org, the crate could already be integrated into a webpage via WASM. A custom retriever via web-sys should enable even more flexibility. Some JS bindings will be needed at this point for a nice integration.

Performance, performance, performance

I have tons of ideas about it, as my main use case is to speed up the generation of instances that match a schema (in hypothesis-jsonschema & schemathesis), hence I am going to focus on the is_valid performance because the exact errors don't matter in this case.

[jpmckinney]

Along with this change, I'd like to expose custom resolvers to Python bindings + hopefully with async too, but not 100% sure about this.

I presently use python-jsonschema, where I use custom resolvers for a few reasons:

1. The default resolver follows file:// URLs. If an application dereferences and renders a user-provided schema, this can be used to read any JSON file on the filesystem to which the web process has access. I therefore use a custom resolver that only resolves HTTP and HTTPS URLs (this is vulnerable to server-side request forgery, but I'm less worried about that).
2. I work with JSON Schema where users are allowed to "patch" the default schema, using JSON Merge Patch (RFC 7386). I use python-jsonschema's registry to make references to the default schema resolve to the patched schema.

It would be really nice to be able to control (1).

For (2), would a workaround be to provide only dereferenced schema to jsonschema_rs?

[Stranger6667]

Hey @jpmckinney

Thank you for bringing this up!

1. Good to know this exact use case, I think that exposing resolvers should not be an issue and could be implemented similarly to how custom format validators work right now.
2. I am not 100% sure about a workaround (if you could provide an example, it would help), however, as this project practically followed the same registry-based design for references, I think we can expose the same functionality to Python bindings and it should be enough for your use case. It also seems like the feature is quite related to Validating individual definitions #432 & Support validating Open API sub-schemas #452

[Stranger6667]

@jpmckinney I realized now that both things you mention are available in Python bindings. Custom retrievers AND registries (they can use retrievers too). Hopefully, it could help in your use case, the API is somewhat similar to Python's jsonschema