-
Notifications
You must be signed in to change notification settings - Fork 11
implement jsonschema #53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
atckr
wants to merge
2
commits into
main
Choose a base branch
from
tphan/jsonschema
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| [package] | ||
| name = "extension" | ||
| version = "0.1.0" | ||
| edition = "2018" | ||
|
|
||
| [lib] | ||
| crate-type = ["cdylib"] | ||
|
|
||
| [dependencies] | ||
| wit-bindgen-rust = { git = "https://github.com/bytecodealliance/wit-bindgen.git", rev = "60e3c5b41e616fee239304d92128e117dd9be0a7" } | ||
| boon = { version = "0.5.1" } | ||
| serde_json = { version = "1" } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| # ss_jsonschema: JSON Schema support for SingleStoreDB | ||
|
|
||
| **ss_jsonschema** adds [JSON schema](https://json-schema.org/) validation support for json data types. Since this is a Wasm extension, this will take `string` as input and parse as Json in the validation process. See examples below. | ||
|
|
||
| ## Contents | ||
|
|
||
| ### `match_schema (schema: string, instance: string) -> bool` | ||
| Returns true if the given instance matches the schema provided | ||
|
|
||
| ### `is_valid_schema (schema: string) -> bool` | ||
| Returns true if the given schema is valid | ||
|
|
||
| ## Building | ||
| ``` | ||
| # Install the WASI cargo extension. | ||
| cargo install cargo-wasi | ||
|
|
||
| # Compile the Wasm module. | ||
| cargo wasi build --release | ||
| ``` | ||
| ## Deployment to SingleStoreDB | ||
|
|
||
| To install these functions using the MySQL client, use the following commands. This command assumes you have built the Wasm module and your current directory is the root of this Git repo. Replace `$DBUSER`, `$DBHOST`, `$DBPORT`, and `$DBNAME` with, respectively, your database username, hostname, port, and the name of the database where you want to deploy the functions. | ||
|
|
||
| ```bash | ||
| cat <<EOF | mysql -u $DBUSER -h $DBHOST -P $DBPORT -D $DBNAME -p | ||
| CREATE FUNCTION match_schema AS WASM FROM LOCAL INFILE "target/wasm32-wasi/release/extension.wasm" WITH WIT FROM LOCAL INFILE "extension.wit"; | ||
| CREATE FUNCTION is_valid_schema AS WASM FROM LOCAL INFILE "target/wasm32-wasi/release/extension.wasm" WITH WIT FROM LOCAL INFILE "extension.wit"; | ||
| ``` | ||
|
|
||
| Alternatively, you can install these functions using [pushwasm](https://github.com/singlestore-labs/pushwasm) with the following command lines. As above, be sure to substitute the environment variables with values of your own. | ||
| ```bash | ||
| pushwasm udf --force --prompt --name match_schema \ | ||
| --wasm target/wasm32-wasi/release/extension.wasm \ | ||
| --wit extension.wit \ | ||
| --abi canonical \ | ||
| --conn "mysql://$DBUSER@$DBHOST:$DBPORT/$DBNAME" | ||
| pushwasm udf --force --prompt --name is_valid_schema \ | ||
| --wasm target/wasm32-wasi/release/extension.wasm \ | ||
| --wit extension.wit \ | ||
| --abi canonical \ | ||
| --conn "mysql://$DBUSER@$DBHOST:$DBPORT/$DBNAME" | ||
| ``` | ||
|
|
||
| ## Clean | ||
| ``` | ||
| cargo clean | ||
| ``` | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Simple example | ||
| ```sql | ||
| SELECT match_schema('{"type": "number"}' , '123'); | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| +--------------------------------------------+ | ||
| | match_schema('{"type": "number"}' , '123') | | ||
| +--------------------------------------------+ | ||
| | 1 | | ||
| +--------------------------------------------+ | ||
| ``` | ||
|
|
||
| ```sql | ||
| SELECT match_schema('{"type": "number"}' , '"hi"'); | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| +---------------------------------------------+ | ||
| | match_schema('{"type": "number"}' , '"hi"') | | ||
| +---------------------------------------------+ | ||
| | 0 | | ||
| +---------------------------------------------+ | ||
| ``` | ||
|
|
||
|
|
||
| ```sql | ||
| SELECT is_valid_schema('{"type" : "integer"}'); | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| +-----------------------------------------+ | ||
| | is_valid_schema('{"type" : "integer"}') | | ||
| +-----------------------------------------+ | ||
| | 1 | | ||
| +-----------------------------------------+ | ||
| ``` | ||
|
|
||
| ```sql | ||
| SELECT is_valid_schema('{"type" : "int"}'); | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| +-------------------------------------+ | ||
| | is_valid_schema('{"type" : "int"}') | | ||
| +-------------------------------------+ | ||
| | 0 | | ||
| +-------------------------------------+ | ||
| ``` | ||
|
|
||
| ### More complex example | ||
| ```sql | ||
| set @schema='{ | ||
| "type": "object", | ||
| "properties": { | ||
| "foo": { | ||
| "type": "string" | ||
| } | ||
| }, | ||
| "required": ["foo"], | ||
| "additionalProperties": false | ||
| }'; | ||
|
|
||
| SELECT match_schema(@schema, '{"foo" : "bar"}'); | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| +------------------------------------------+ | ||
| | match_schema(@schema, '{"foo" : "bar"}') | | ||
| +------------------------------------------+ | ||
| | 1 | | ||
| +------------------------------------------+ | ||
| ``` | ||
|
|
||
| ```sql | ||
| SELECT match_schema(@schema, '{"notfoo" : "bar"}'); | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| +---------------------------------------------+ | ||
| | match_schema(@schema, '{"notfoo" : "bar"}') | | ||
| +---------------------------------------------+ | ||
| | 0 | | ||
| +---------------------------------------------+ | ||
| ``` | ||
|
|
||
| ```sql | ||
| SELECT match_schema(@schema, '{"foo" : "bar", "morefoo": "morebar"}'); | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| +----------------------------------------------------------------+ | ||
| | match_schema(@schema, '{"foo" : "bar", "morefoo": "morebar"}') | | ||
| +----------------------------------------------------------------+ | ||
| | 0 | | ||
| +----------------------------------------------------------------+ | ||
| ``` | ||
|
|
||
| ## Acknowledgement | ||
| Rust [boon](https://crates.io/crates/boon) library and [JSON Schema](https://json-schema.org/) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| match-schema: func(schema: string, instance: string) -> bool | ||
| is-valid-schema: func(schema: string) -> bool |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| wit_bindgen_rust::export!("extension.wit"); | ||
| use boon::{Compiler, Schemas}; | ||
| use serde_json::Value; | ||
|
|
||
| struct Extension; | ||
|
|
||
| const DEFAULT_SCHEMA_URL: &str = "http://tmp/schema.json"; | ||
|
|
||
| impl extension::Extension for Extension { | ||
| fn match_schema(schema_str: String, instance_str: String) -> bool { | ||
| let schema: Value = serde_json::from_str(schema_str.as_str()).unwrap(); | ||
| let instance: Value = serde_json::from_str(instance_str.as_str()).unwrap(); | ||
|
|
||
| let mut schemas = Schemas::new(); | ||
| let mut compiler = Compiler::new(); | ||
| compiler.add_resource(DEFAULT_SCHEMA_URL, schema).unwrap(); | ||
| let sch_index = compiler.compile(DEFAULT_SCHEMA_URL, &mut schemas).unwrap(); | ||
| let result = schemas.validate(&instance, sch_index); | ||
| if result.is_err() { | ||
| false | ||
| } | ||
| else { | ||
| true | ||
| } | ||
| } | ||
|
|
||
| fn is_valid_schema(schema_str: String) -> bool { | ||
| let schema: Value = serde_json::from_str(schema_str.as_str()).unwrap(); | ||
| let mut compiler = Compiler::new(); | ||
| let is_valid = compiler.add_resource(DEFAULT_SCHEMA_URL, schema); | ||
| if is_valid.is_err() { | ||
| false | ||
| } | ||
| else { | ||
| true | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // sanity tests | ||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
|
|
||
| #[test] | ||
| fn test_from_string_naive() { | ||
| let naive_schema = r#"{"type": "object"}"#.to_string(); | ||
| let naive_input = r#"{"foo": "bar"}"#.to_string(); | ||
| assert_eq!(<Extension as extension::Extension>::match_schema(naive_schema, naive_input), true); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_from_string_number() { | ||
| let number_schema = r#"{"type": "number"}"#.to_string(); | ||
| let number_str_pass = "123".to_string(); | ||
| let number_str_fail = r#""meow""#.to_string(); | ||
|
|
||
| assert_eq!(<Extension as extension::Extension>::match_schema(number_schema.clone(), number_str_pass), true); | ||
| assert_eq!(<Extension as extension::Extension>::match_schema(number_schema, number_str_fail), false); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_from_string_required_field() { | ||
| let required_schema = r#"{ | ||
| "type": "object", | ||
| "properties": { | ||
| "foo": { | ||
| "type": "string" | ||
| } | ||
| }, | ||
| "required": ["foo"], | ||
| "additionalProperties": false | ||
| }"#.to_string(); | ||
| let required_input_pass = r#"{"foo": "bar"}"#.to_string(); | ||
| let required_input_fail = r#"{"hi" : "bar"}"#.to_string(); | ||
| let required_input_fail_2 = r#"{"foo": "bar", "hi": "bar"}"#.to_string(); | ||
| assert_eq!(<Extension as extension::Extension>::match_schema(required_schema.clone(), required_input_pass), true); | ||
| assert_eq!(<Extension as extension::Extension>::match_schema(required_schema.clone(), required_input_fail), false); | ||
| assert_eq!(<Extension as extension::Extension>::match_schema(required_schema, required_input_fail_2), false); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_is_valid_schema() { | ||
| let valid_schema = r#"{"type": "integer"}"#.to_string(); | ||
| let not_valid_schema = r#"{"type": "int"}"#.to_string(); | ||
| assert_eq!(<Extension as extension::Extension>::is_valid_schema(valid_schema), true); | ||
| assert_eq!(<Extension as extension::Extension>::is_valid_schema(not_valid_schema), false); | ||
| } | ||
| } | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: this block could be abbreviated to just:
!result.is_err()Same with the one below.