Skip to content

Commit 7f1ec11

Browse files
authored
Support Draft 2020-12 and refactor schema retrieval (#931)
* Fix ref * Refactor * Refactor * Refactor * Refactor * Fix test * Refactor * Add test * Ensure correct meta schema * Support dynamic ref * Fix anchor * Refactor * Update docs * Refactor * Fix * Refactor * Update test suite * Refactor * Fix * Support custom meta schema * Refactor * Support format assertion configuration * Refactor * Refactor * Refactor * Update docs * Allow yaml to be optional * Support vocabularies * Refactor * Refactor * Refactor * Remove deprecations * Update uri schema loader * Refactor * Refactor * Add get instance customizer * Add content validation * Update compatibility doc * Refactor enumObject is failing * Fix * Refactor * Refactor package * Refactor config * validation context set in constructor * Add javadoc * Only normalize standard json schema dialects * Refactor * update comment * Fix dynamic ref circular dependency * Update docs and readme * Update example * Fix 877 * Fix rebase * Add tests * Shift package * Add convenience methods for validation * Refactor * Update docs * Update docs * Update docs * Refactor schema loaders * Update format routines and tests and fixes * Update doc * Update doc * Fix spacing
1 parent f3825f3 commit 7f1ec11

File tree

167 files changed

+4890
-2921
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

167 files changed

+4890
-2921
lines changed

README.md

Lines changed: 150 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,19 @@
1212
[![Javadocs](http://www.javadoc.io/badge/com.networknt/json-schema-validator.svg)](https://www.javadoc.io/doc/com.networknt/json-schema-validator)
1313

1414

15-
This is a Java implementation of the [JSON Schema Core Draft v4, v6, v7, v2019-09 and v2020-12(partial)](http://json-schema.org/latest/json-schema-core.html) specification for JSON schema validation. In addition, it also works for OpenAPI 3.0 request/response validation with some [configuration flags](doc/config.md). For users who want to collect information from a JSON node based on the schema, the [walkers](doc/walkers.md) can help. The default JSON parser is the [Jackson](https://github.com/FasterXML/jackson) that is the most popular one. As it is a key component in our [light-4j](https://github.com/networknt/light-4j) microservices framework to validate request/response against OpenAPI specification for [light-rest-4j](http://www.networknt.com/style/light-rest-4j/) and RPC schema for [light-hybrid-4j](http://www.networknt.com/style/light-hybrid-4j/) at runtime, performance is the most important aspect in the design.
15+
This is a Java implementation of the [JSON Schema Core Draft v4, v6, v7, v2019-09 and v2020-12](http://json-schema.org/latest/json-schema-core.html) specification for JSON schema validation.
16+
17+
In addition, it also works for OpenAPI 3.0 request/response validation with some [configuration flags](doc/config.md). For users who want to collect information from a JSON node based on the schema, the [walkers](doc/walkers.md) can help. The default JSON parser is the [Jackson](https://github.com/FasterXML/jackson) that is the most popular one. As it is a key component in our [light-4j](https://github.com/networknt/light-4j) microservices framework to validate request/response against OpenAPI specification for [light-rest-4j](http://www.networknt.com/style/light-rest-4j/) and RPC schema for [light-hybrid-4j](http://www.networknt.com/style/light-hybrid-4j/) at runtime, performance is the most important aspect in the design.
18+
19+
## JSON Schema Draft Specification Compatibility
20+
21+
Information on the compatibility support for each version, including known issues, can be found in the [Compatibility with JSON Schema versions](doc/compatibility.md) document.
22+
23+
## Upgrading to new versions
24+
25+
Information on notable or breaking changes when upgrading the library can be found in the [Upgrading to new versions](doc/upgrading.md) document. This library can contain breaking changes in minor version releases.
26+
27+
For the latest version, please check the [Releases](https://github.com/networknt/json-schema-validator/releases) page.
1628

1729
## Why this library
1830

@@ -42,23 +54,67 @@ The OpenAPI 3.0 specification is using JSON schema to validate the request/respo
4254

4355
Following the design principle of the Light Platform, this library has minimum dependencies to ensure there are no dependency conflicts when using it.
4456

45-
Here are the dependencies:
57+
The following are the dependencies that will automatically be included when this library is included.
4658

4759
```xml
4860
<dependency>
61+
<!-- Used for logging -->
62+
<groupId>org.slf4j</groupId>
63+
<artifactId>slf4j-api</artifactId>
64+
<version>${version.slf4j}</version>
65+
</dependency>
66+
67+
<dependency>
68+
<!-- Used to process JSON -->
4969
<groupId>com.fasterxml.jackson.core</groupId>
5070
<artifactId>jackson-databind</artifactId>
5171
<version>${version.jackson}</version>
5272
</dependency>
5373

5474
<dependency>
55-
<groupId>org.slf4j</groupId>
56-
<artifactId>slf4j-api</artifactId>
57-
<version>${version.slf4j}</version>
75+
<!-- Used to process YAML -->
76+
<groupId>com.fasterxml.jackson.dataformat</groupId>
77+
<artifactId>jackson-dataformat-yaml</artifactId>
78+
<version>${version.jackson}</version>
79+
</dependency>
80+
81+
<dependency>
82+
<!-- Used to validate RFC 3339 date and date-time -->
83+
<groupId>com.ethlo.time</groupId>
84+
<artifactId>itu</artifactId>
85+
<version>${version.itu}</version>
5886
</dependency>
5987
```
6088

61-
**Note**: Up to version [1.0.81](https://github.com/networknt/json-schema-validator/blob/1.0.81/pom.xml#L99), the dependency `org.apache.commons:commons-lang3` was included as a runtime dependency. Starting with [1.0.82](https://github.com/networknt/json-schema-validator/releases/tag/1.0.82) it is not required anymore.
89+
The following are the optional dependencies that may be required for certain options.
90+
91+
These are not automatically included and setting the relevant option without adding the library will result in a `ClassNotFoundException`.
92+
93+
```xml
94+
<!-- This is required when setting setEcma262Validator(true) -->
95+
<dependency>
96+
<!-- Used to validate ECMA 262 regular expressions -->
97+
<groupId>org.jruby.joni</groupId>
98+
<artifactId>joni</artifactId>
99+
<version>${version.joni}</version>
100+
<optional>true</optional>
101+
</dependency>
102+
```
103+
104+
The YAML dependency can be excluded if this is not required. Attempting to process schemas or input that are YAML will result in a `ClassNotFoundException`.
105+
106+
```xml
107+
<dependency>
108+
<groupId>com.networknt</groupId>
109+
<artifactId>json-schema-validator</artifactId>
110+
<exclusions>
111+
<exclusion>
112+
<groupId>com.fasterxml.jackson.dataformat</groupId>
113+
<artifactId>jackson-dataformat-yaml</artifactId>
114+
</exclusion>
115+
</exclusions>
116+
</dependency>
117+
```
62118

63119
#### Community
64120

@@ -68,37 +124,110 @@ This library is very active with a lot of contributors. New features and bug fix
68124

69125
The library supports Java 8 and up. If you want to build from the source code, you need to install JDK 8 locally. To support multiple version of JDK, you can use [SDKMAN](https://www.networknt.com/tool/sdk/)
70126

71-
## Dependency
127+
## Usage
128+
129+
### Adding the dependency
72130

73131
This package is available on Maven central.
74132

75-
Maven:
133+
#### Maven:
76134

77135
```xml
78136
<dependency>
79137
<groupId>com.networknt</groupId>
80138
<artifactId>json-schema-validator</artifactId>
81-
<version>1.0.87</version>
82-
83-
<!-- Only required for versions < 1.0.82. See README.md -->
84-
<exclusions>
85-
<exclusion>
86-
<groupId>org.apache.commons</groupId>
87-
<artifactId>commons-lang3</artifactId>
88-
</exclusion>
89-
</exclusions>
139+
<version>1.2.0</version>
90140
</dependency>
91141
```
92142

93-
Gradle:
143+
#### Gradle:
94144

95145
```java
96146
dependencies {
97-
implementation(group: 'com.networknt', name: 'json-schema-validator', version: '1.0.87');
147+
implementation(group: 'com.networknt', name: 'json-schema-validator', version: '1.2.0');
98148
}
99149
```
100150

101-
For the latest version, please check the [release](https://github.com/networknt/json-schema-validator/releases) page.
151+
### Validating inputs against a schema
152+
153+
The following example demonstrates how inputs is validated against a schema. It comprises the following steps.
154+
155+
* Creating a schema factory with the default schema dialect and how the schemas can be retrieved.
156+
* Configuring mapping the `$id` to a retrieval URI using `schemaMappers`.
157+
* Configuring how the schemas are loaded using the retrieval URI using `schemaLoaders`.
158+
For instance a `Map<String, String> schemas` containing a mapping of retrieval URI to schema data as a `String` can by configured using `builder.schemaLoaders(schemaLoaders -> schemaLoaders.schemas(schemas))`. This also accepts a `Function<String, String> schemaRetrievalFunction`.
159+
* Creating a configuration for controlling validator behavior.
160+
* Loading a schema from a schema location along with the validator configuration.
161+
* Using the schema to validate the data along with setting any execution specific configuration like for instance the locale or whether format assertions are enabled.
162+
163+
```java
164+
// This creates a schema factory that will use Draft 2012-12 as the default if $schema is not specified in the schema data. If $schema is specified in the schema data then that schema dialect will be used instead and this version is ignored.
165+
JsonSchemaFactory jsonSchemaFactory = JsonSchemaFactory.getInstance(VersionFlag.V202012, builder ->
166+
// This creates a mapping from $id which starts with https://www.example.org/ to the retrieval URI classpath:schema/
167+
builder.schemaMappers(schemaMappers -> schemaMappers.mapPrefix("https://www.example.org/", "classpath:schema/"))
168+
);
169+
170+
SchemaValidatorsConfig config = new SchemaValidatorsConfig();
171+
// By default JSON Path is used for reporting the instance location and evaluation path
172+
config.setPathType(PathType.JSON_POINTER);
173+
// By default the JDK regular expression implementation which is not ECMA 262 compliant is used
174+
// Note that setting this to true requires including the optional joni dependency
175+
// config.setEcma262Validator(true);
176+
177+
// Due to the mapping the schema will be retrieved from the classpath at classpath:schema/example-main.json. If the schema data does not specify an $id the absolute IRI of the schema location will be used as the $id.
178+
JsonSchema schema = jsonSchemaFactory.getSchema(SchemaLocation.of("https://www.example.org/example-main.json"), config);
179+
String input = "{\r\n"
180+
+ " \"main\": {\r\n"
181+
+ " \"common\": {\r\n"
182+
+ " \"field\": \"invalidfield\"\r\n"
183+
+ " }\r\n"
184+
+ " }\r\n"
185+
+ "}";
186+
187+
Set<ValidationMessage> assertions = schema.validate(input, InputFormat.JSON, executionContext -> {
188+
// By default since Draft 2019-09 the format keyword only generates annotations and not assertions
189+
executionContext.getConfig().setFormatAssertionsEnabled(true);
190+
});
191+
```
192+
193+
### Validating a schema against a meta schema
194+
195+
The following example demonstrates how a schema is validated against a meta schema.
196+
197+
This is actually the same as validating inputs against a schema except in this case the input is the schema and the schema used is the meta schema.
198+
199+
```java
200+
JsonSchemaFactory jsonSchemaFactory = JsonSchemaFactory.getInstance(VersionFlag.V202012, builder ->
201+
// This creates a mapping to load the meta schema from the library classpath instead of remotely
202+
// This is better for performance and the remote may choose not to service the request
203+
// For instance Cloudflare will block requests that have older Java User-Agent strings eg. Java/1.
204+
builder.schemaMappers(schemaMappers ->
205+
schemaMappers.mapPrefix("https://json-schema.org", "classpath:").mapPrefix("http://json-schema.org", "classpath:"))
206+
);
207+
208+
SchemaValidatorsConfig config = new SchemaValidatorsConfig();
209+
// By default JSON Path is used for reporting the instance location and evaluation path
210+
config.setPathType(PathType.JSON_POINTER);
211+
// By default the JDK regular expression implementation which is not ECMA 262 compliant is used
212+
// Note that setting this to true requires including the optional joni dependency
213+
// config.setEcma262Validator(true);
214+
215+
// Due to the mapping the meta schema will be retrieved from the classpath at classpath:draft/2020-12/schema.
216+
JsonSchema schema = jsonSchemaFactory.getSchema(SchemaLocation.of(SchemaId.V202012), config);
217+
String input = "{ \n"
218+
+ " \"type\": \"object\", \n"
219+
+ " \"properties\": { \n"
220+
+ " \"key\": { \n"
221+
+ " \"title\" : \"My key\", \n"
222+
+ " \"type\": \"invalidtype\" \n"
223+
+ " } \n"
224+
+ " }\n"
225+
+ "}";
226+
Set<ValidationMessage> assertions = schema.validate(input, InputFormat.JSON, executionContext -> {
227+
// By default since Draft 2019-09 the format keyword only generates annotations and not assertions
228+
executionContext.getConfig().setFormatAssertionsEnabled(true);
229+
});
230+
```
102231

103232
## [Quick Start](doc/quickstart.md)
104233

@@ -110,9 +239,7 @@ For the latest version, please check the [release](https://github.com/networknt/
110239

111240
## [YAML Validation](doc/yaml.md)
112241

113-
## [Schema Mapping](doc/schema-map.md)
114-
115-
## [Customized URIFetcher](doc/cust-fetcher.md)
242+
## [Customizing Schema Retrieval](doc/schema-retrieval.md)
116243

117244
## [Customized MetaSchema](doc/cust-meta.md)
118245

@@ -130,15 +257,6 @@ For the latest version, please check the [release](https://github.com/networknt/
130257

131258
## [Validating RFC 3339 durations](doc/duration.md)
132259

133-
134-
## Known issues
135-
136-
I have just updated the test suites from the [official website](https://github.com/json-schema-org/JSON-Schema-Test-Suite) as the old ones were copied from another Java validator. Now there are several issues that need to be addressed. All of them are edge cases, in my opinion, but need to be investigated. As my old test suites were inherited from another Java JSON Schema Validator, I guess other Java Validator would have the same issues as these issues are in the Java language itself.
137-
138-
[#7](https://github.com/networknt/json-schema-validator/issues/7)
139-
140-
[#5](https://github.com/networknt/json-schema-validator/issues/5)
141-
142260
## Projects
143261

144262
The [light-rest-4j](https://github.com/networknt/light-rest-4j), [light-graphql-4j](https://github.com/networknt/light-graphql-4j) and [light-hybrid-4j](https://github.com/networknt/light-hybrid-4j) use this library to validate the request and response based on the specifications. If you are using other frameworks like Spring Boot, you can use the [OpenApiValidator](https://github.com/mservicetech/openapi-schema-validation), a generic OpenAPI 3.0 validator based on the OpenAPI 3.0 specification.

doc/compatibility.md

Lines changed: 63 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,13 @@
1+
## Compatibility with JSON Schema versions
2+
3+
This implementation does not currently generate annotations.
4+
5+
The `pattern` validator by default uses the JDK regular expression implementation which is not ECMA-262 compliant and is thus not compliant with the JSON Schema specification. The library can however be configured to use a ECMA-262 compliant regular expression implementation.
6+
7+
### Known Issues
8+
* The `anyOf` applicator currently returns immediately on matching a schema. This results in the `unevaluatedItems` and `unevaluatedProperties` keywords potentially returning an incorrect result as the rest of the schemas in the `anyOf` aren't processed.
9+
* The `unevaluatedItems` keyword does not currently consider `contains`.
10+
111

212
### Legend
313

@@ -8,27 +18,27 @@
818
| 🔴 | Not implemented |
919
| 🚫 | Not defined |
1020

11-
### Compatibility with JSON Schema versions
21+
### Keywords Support
1222

1323
| Keyword | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 | Draft 2020-12 |
1424
|:---------------------------|:-------:|:-------:|:-------:|:-------------:|:-------------:|
15-
| $anchor | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
16-
| $dynamicAnchor | 🚫 | 🚫 | 🚫 | 🚫 | 🔴 |
17-
| $dynamicRef | 🚫 | 🚫 | 🚫 | 🚫 | 🔴 |
18-
| $id | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 |
25+
| $anchor | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
26+
| $dynamicAnchor | 🚫 | 🚫 | 🚫 | 🚫 | 🟢 |
27+
| $dynamicRef | 🚫 | 🚫 | 🚫 | 🚫 | 🟢 |
28+
| $id | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
1929
| $recursiveAnchor | 🚫 | 🚫 | 🚫 | 🟢 | 🚫 |
2030
| $recursiveRef | 🚫 | 🚫 | 🚫 | 🟢 | 🚫 |
21-
| $ref | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 |
22-
| $vocabulary | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
31+
| $ref | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
32+
| $vocabulary | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
2333
| additionalItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
2434
| additionalProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
2535
| allOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
2636
| anyOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
2737
| const | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
2838
| contains | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
29-
| contentEncoding | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
30-
| contentMediaType | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
31-
| contentSchema | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
39+
| contentEncoding | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
40+
| contentMediaType | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
41+
| contentSchema | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
3242
| definitions | 🟢 | 🟢 | 🟢 | 🚫 | 🚫 |
3343
| defs | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
3444
| dependencies | 🟢 | 🟢 | 🟢 | 🚫 | 🚫 |
@@ -67,7 +77,49 @@
6777
| uniqueItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
6878
| writeOnly | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
6979

70-
### Semantic Validation (Format)
80+
#### Content Encoding
81+
82+
Since Draft 2019-09, the `contentEncoding` keyword does not generate assertions. As the implementation currently does not collect annotations this only generates assertions in Draft 7.
83+
84+
#### Content Media Type
85+
86+
Since Draft 2019-09, the `contentMediaType` keyword does not generate assertions. As the implementation currently does not collect annotations this only generates assertions in Draft 7.
87+
88+
#### Content Schema
89+
90+
The `contentSchema` keyword does not generate assertions. As the implementation currently does not collect annotations this doesn't do anything.
91+
92+
#### Pattern
93+
94+
By default the `pattern` keyword uses the JDK regular expression implementation validating regular expressions.
95+
96+
This is not ECMA-262 compliant and is thus not compliant with the JSON Schema specification. This is however the more likely desired behavior as other logic will most likely be using the default JDK regular expression implementation to perform downstream processing.
97+
98+
The library can be configured to use a ECMA-262 compliant regular expression validator which is implemented using [joni](https://github.com/jruby/joni). This can be configured by setting `setEcma262Validator` to `true`.
99+
100+
This also requires adding the `joni` dependency.
101+
102+
```xml
103+
<dependency>
104+
<!-- Used to validate ECMA 262 regular expressions -->
105+
<groupId>org.jruby.joni</groupId>
106+
<artifactId>joni</artifactId>
107+
<version>${version.joni}</version>
108+
</dependency>
109+
```
110+
111+
### Format
112+
113+
Since Draft 2019-09 the `format` keyword only generates annotations by default and does not generate assertions.
114+
115+
This can be configured on a schema basis by using a meta schema with the appropriate vocabulary.
116+
117+
| Version | Vocabulary | Value |
118+
|:----------------------|---------------------------------------------------------------|-------------------|
119+
| Draft 2019-09 | `https://json-schema.org/draft/2019-09/vocab/format` | `true` |
120+
| Draft 2020-12 | `https://json-schema.org/draft/2020-12/vocab/format-assertion`| `true`/`false` |
121+
122+
This behavior can be overridden to generate assertions on a per-execution basis by setting the `setFormatAssertionsEnabled` to `true`.
71123

72124
| Format | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 | Draft 2020-12 |
73125
|:----------------------|:-------:|:-------:|:-------:|:-------------:|:-------------:|

doc/config.md

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -45,14 +45,6 @@ The default value is true in the SchemaValidatorsConfig object.
4545

4646
For more details, please refer to this [issue](https://github.com/networknt/json-schema-validator/issues/183).
4747

48-
49-
* uriMappings
50-
51-
Map of public, typically internet-accessible schema URLs to alternate locations; this allows for offline validation of schemas that refer to public URLs. This is merged with any mappings the sonSchemaFactory
52-
may have been built.
53-
54-
The type for this variable is `Map<String, String>`.
55-
5648
* javaSemantics
5749

5850
When set to true, use Java-specific semantics rather than native JavaScript semantics.

0 commit comments

Comments
 (0)