Skip to content

Commit f09740a

Browse files
fduttonFaron Dutton
fdutton
and
Faron Dutton
authored
Resolves improper anchoring of patternProperties (#783)
* Resolves improper anchoring of patternProperties Fixes #782 * Resolves improper anchoring of regular expressions in both Joni and JDK engines. Resolves #495 and #782 --------- Co-authored-by: Faron Dutton <[email protected]>
1 parent 77cd232 commit f09740a

File tree

6 files changed

+274
-75
lines changed

6 files changed

+274
-75
lines changed

‎doc/compatibility.md

Lines changed: 84 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,76 +1,97 @@
11

22
### Legend
33

4-
Symbol | Meaning |
5-
:-----:|---------|
6-
🟢 | Fully implemented
7-
🟡 | Partially implemented
8-
🔴 | Not implemented
9-
🚫 | Not defined in Schema Version.
4+
| Symbol | Meaning |
5+
|:------:|:----------------------|
6+
| 🟢 | Fully implemented |
7+
| 🟡 | Partially implemented |
8+
| 🔴 | Not implemented |
9+
| 🚫 | Not defined |
1010

1111
### Compatibility with JSON Schema versions
1212

13-
Validation Keyword/Schema | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 |
14-
---------------- |:--------------:|:-------: |:-------: |:-------------:|
15-
$ref | 🟢 | 🟢 | 🟢 | 🟢
16-
additionalProperties | 🟢 | 🟢 | 🟢 | 🟢
17-
additionalItems | 🟢 | 🟢 | 🟢 | 🟢
18-
allOf | 🟢 | 🟢 | 🟢 | 🟢
19-
anyOf | 🟢 | 🟢 | 🟢 | 🟢
20-
const | 🚫 | 🟢 | 🟢 | 🟢
21-
contains | 🚫 | 🟢 | 🟢 | 🟢
22-
contentEncoding | 🚫 | 🚫 | 🔴 | 🔴
23-
contentMediaType | 🚫 | 🚫 | 🔴 | 🔴
24-
dependencies | 🟢 | 🟢 |🟢 | 🟢
25-
enum | 🟢 | 🟢 | 🟢 | 🟢
26-
exclusiveMaximum (boolean) | 🟢 | 🚫 | 🚫 | 🚫
27-
exclusiveMaximum (numeric) | 🚫 | 🟢 | 🟢 | 🟢
28-
exclusiveMinimum (boolean) | 🟢 | 🚫 | 🚫 | 🚫
29-
exclusiveMinimum (numeric) | 🚫 | 🟢 | 🟢 | 🟢
30-
items | 🟢 | 🟢 | 🟢 | 🟢
31-
maximum | 🟢 | 🟢 | 🟢 | 🟢
32-
maxItems | 🟢 | 🟢 | 🟢 | 🟢
33-
maxLength | 🟢 | 🟢 | 🟢 | 🟢
34-
maxProperties | 🟢 | 🟢 | 🟢 | 🟢
35-
minimum | 🟢 | 🟢 | 🟢 | 🟢
36-
minItems | 🟢 | 🟢 | 🟢 | 🟢
37-
minLength | 🟢 | 🟢 | 🟢 | 🟢
38-
minProperties | 🟢 | 🟢 | 🟢 | 🟢
39-
multipleOf | 🟢 | 🟢 | 🟢 | 🟢
40-
not | 🟢 | 🟢 | 🟢 | 🟢
41-
oneOf | 🟢 | 🟢 | 🟢 | 🟢
42-
pattern | 🟢 | 🟢 | 🟢 | 🟢
43-
patternProperties | 🟢 | 🟢 | 🟢 | 🟢
44-
properties | 🟢 | 🟢 | 🟢 | 🟢
45-
propertyNames | 🚫 | 🔴 | 🔴 | 🔴
46-
required | 🟢 | 🟢 | 🟢 | 🟢
47-
type | 🟢 | 🟢 | 🟢 | 🟢
48-
uniqueItems | 🟢 | 🟢 | 🟢 | 🟢
13+
| Keyword | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 | Draft 2020-12 |
14+
|:---------------------------|:-------:|:-------:|:-------:|:-------------:|:-------------:|
15+
| $anchor | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
16+
| $dynamicAnchor | 🚫 | 🚫 | 🚫 | 🚫 | 🔴 |
17+
| $dynamicRef | 🚫 | 🚫 | 🚫 | 🚫 | 🔴 |
18+
| $id | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 |
19+
| $recursiveAnchor | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
20+
| $recursiveRef | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
21+
| $ref | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 |
22+
| $vocabulary | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
23+
| additionalItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
24+
| additionalProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
25+
| allOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
26+
| anyOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
27+
| const | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
28+
| contains | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
29+
| contentEncoding | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
30+
| contentMediaType | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
31+
| contentSchema | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
32+
| definitions | 🟢 | 🟢 | 🟢 | 🚫 | 🚫 |
33+
| defs | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
34+
| dependencies | 🟢 | 🟢 | 🟢 | 🚫 | 🚫 |
35+
| dependentRequired | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
36+
| dependentSchemas | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
37+
| enum | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
38+
| exclusiveMaximum (boolean) | 🟢 | 🚫 | 🚫 | 🚫 | 🚫 |
39+
| exclusiveMaximum (numeric) | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
40+
| exclusiveMinimum (boolean) | 🟢 | 🚫 | 🚫 | 🚫 | 🚫 |
41+
| exclusiveMinimum (numeric) | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
42+
| if-then-else | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
43+
| items | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
44+
| maxContains | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
45+
| minContains | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
46+
| maximum | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
47+
| maxItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
48+
| maxLength | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
49+
| maxProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
50+
| minimum | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
51+
| minItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
52+
| minLength | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
53+
| minProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
54+
| multipleOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
55+
| not | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
56+
| oneOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
57+
| pattern | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
58+
| patternProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
59+
| prefixItems | 🚫 | 🚫 | 🚫 | 🚫 | 🟢 |
60+
| properties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
61+
| propertyNames | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
62+
| readOnly | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
63+
| required | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
64+
| type | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
65+
| unevaluatedItems | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
66+
| unevaluatedProperties | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
67+
| uniqueItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
68+
| writeOnly | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
4969

5070
### Semantic Validation (Format)
5171

52-
Format | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 |
53-
-------|---------|---------|---------|---------------|
54-
date |🚫 | 🚫 | 🟢 | 🟢
55-
date-time | 🟢 | 🟢 | 🟢 | 🟢
56-
duration | 🚫 | 🚫 | 🔴 | 🔴
57-
email | 🟢 | 🟢 | 🟢 | 🟢
58-
hostname | 🟢 | 🟢 | 🟢 | 🟢
59-
idn-email | 🚫 | 🚫 | 🔴 | 🔴
60-
idn-hostname | 🚫 | 🚫 | 🔴 | 🔴
61-
ipv4 | 🟢 | 🟢 | 🟢 | 🟢
62-
ipv6 | 🟢 | 🟢 | 🟢 | 🟢
63-
iri | 🚫 | 🚫 | 🔴 | 🔴
64-
iri-reference | 🚫 | 🚫 | 🔴 | 🔴
65-
json-pointer | 🚫 | 🔴 | 🔴 | 🔴
66-
relative-json-pointer | 🚫 | 🔴 | 🔴 | 🔴
67-
regex | 🚫 | 🚫 | 🔴 | 🔴
68-
time | 🚫 | 🚫 | 🟢 | 🟢
69-
uri | 🟢 | 🟢 | 🟢 | 🟢
70-
uri-reference | 🚫 | 🔴 | 🔴 | 🔴
71-
uri-template | 🚫 | 🔴 | 🔴 | 🔴
72-
uuid | 🚫 | 🚫 | 🟢 | 🟢
72+
| Format | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 | Draft 2020-12 |
73+
|:----------------------|:-------:|:-------:|:-------:|:-------------:|:-------------:|
74+
| date | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
75+
| date-time | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
76+
| duration | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
77+
| email | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
78+
| hostname | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
79+
| idn-email | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
80+
| idn-hostname | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
81+
| ipv4 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
82+
| ipv6 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
83+
| iri | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
84+
| iri-reference | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
85+
| json-pointer | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
86+
| relative-json-pointer | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
87+
| regex | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
88+
| time | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
89+
| uri | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
90+
| uri-reference | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
91+
| uri-template | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
92+
| uuid | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
7393

7494
### Footnotes
7595
1. Note that the validation are only optional for some of the keywords/formats.
7696
2. Refer to the corresponding JSON schema for more information on whether the keyword/format is optional or not.
97+

‎src/main/java/com/networknt/schema/regex/JDKRegularExpression.java

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,23 @@
11
package com.networknt.schema.regex;
22

3+
import java.util.regex.Matcher;
34
import java.util.regex.Pattern;
45

56
class JDKRegularExpression implements RegularExpression {
67
private final Pattern pattern;
8+
private final boolean hasStartAnchor;
9+
private final boolean hasEndAnchor;
710

811
JDKRegularExpression(String regex) {
912
this.pattern = Pattern.compile(regex);
13+
this.hasStartAnchor = '^' == regex.charAt(0);
14+
this.hasEndAnchor = '$' == regex.charAt(regex.length() - 1);
1015
}
1116

1217
@Override
1318
public boolean matches(String value) {
14-
return this.pattern.matcher(value).matches();
19+
Matcher matcher = this.pattern.matcher(value);
20+
return matcher.find() && (!this.hasStartAnchor || 0 == matcher.start()) && (!this.hasEndAnchor || matcher.end() == value.length());
1521
}
1622

1723
}

‎src/test/java/com/networknt/schema/AbstractJsonSchemaTestSuite.java

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@
5252
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
5353

5454
public abstract class AbstractJsonSchemaTestSuite extends HTTPServiceSupport {
55-
protected static final TypeReference<List<TestCase>> testCaseType = new TypeReference<List<TestCase>>() {};
55+
protected static final TypeReference<List<TestCase>> testCaseType = new TypeReference<List<TestCase>>() { /* intentionally empty */};
5656
protected static final Map<String, VersionFlag> supportedVersions = new HashMap<>();
5757
static {
5858
supportedVersions.put("draft2019-09", VersionFlag.V201909);
@@ -99,7 +99,7 @@ private DynamicNode buildContainer(VersionFlag defaultVersion, TestCase testCase
9999
String msg = e.getMessage();
100100
if (msg.endsWith("' is unrecognizable schema")) {
101101
return dynamicContainer(testCase.getDisplayName(), unsupportedMetaSchema(testCase));
102-
};
102+
}
103103
throw e;
104104
}
105105
}
@@ -109,7 +109,7 @@ private JsonSchemaFactory buildValidatorFactory(VersionFlag defaultVersion, Test
109109
JsonSchemaFactory base = JsonSchemaFactory.getInstance(specVersion);
110110
return JsonSchemaFactory
111111
.builder(base)
112-
.objectMapper(mapper)
112+
.objectMapper(this.mapper)
113113
.addUriTranslator(URITranslator.combine(
114114
URITranslator.prefix("https://", "http://"),
115115
URITranslator.prefix("http://json-schema.org", "resource:")
@@ -124,21 +124,21 @@ private DynamicNode buildTest(JsonSchemaFactory validatorFactory, TestSpec testS
124124

125125
SchemaValidatorsConfig config = new SchemaValidatorsConfig();
126126
config.setTypeLoose(typeLoose);
127-
config.setEcma262Validator(true);
127+
config.setEcma262Validator(TestSpec.RegexKind.JDK != testSpec.getRegex());
128128
testSpec.getStrictness().forEach(config::setStrict);
129129
URI testCaseFileUri = URI.create("classpath:" + toForwardSlashPath(testSpec.getTestCase().getSpecification()));
130130
JsonSchema schema = validatorFactory.getSchema(testCaseFileUri, testSpec.getTestCase().getSchema(), config);
131131

132132
return dynamicTest(testSpec.getDescription(), () -> executeAndReset(schema, testSpec));
133133
}
134134

135-
private String toForwardSlashPath(Path file) {
135+
private static String toForwardSlashPath(Path file) {
136136
return file.toString().replace('\\', '/');
137137
}
138138

139139
// For 2019-09 and later published drafts, implementations that are able to
140140
// detect the draft of each schema via $schema SHOULD be configured to do so
141-
private VersionFlag detectVersion(TestCase testCase, VersionFlag defaultVersion) {
141+
private static VersionFlag detectVersion(TestCase testCase, VersionFlag defaultVersion) {
142142
return Stream.of(
143143
detectOptionalVersion(testCase.getSchema()),
144144
detectVersionFromPath(testCase.getSpecification())
@@ -152,7 +152,7 @@ private VersionFlag detectVersion(TestCase testCase, VersionFlag defaultVersion)
152152
// For draft-07 and earlier, draft-next, and implementations unable to
153153
// detect via $schema, implementations MUST be configured to expect the
154154
// draft matching the test directory name
155-
private Optional<VersionFlag> detectVersionFromPath(Path path) {
155+
private static Optional<VersionFlag> detectVersionFromPath(Path path) {
156156
return StreamSupport.stream(path.spliterator(), false)
157157
.map(Path::toString)
158158
.map(supportedVersions::get)
@@ -168,7 +168,7 @@ private void executeAndReset(JsonSchema schema, TestSpec testSpec) {
168168
}
169169
}
170170

171-
private void executeTest(JsonSchema schema, TestSpec testSpec) {
171+
private static void executeTest(JsonSchema schema, TestSpec testSpec) {
172172
Set<ValidationMessage> errors = schema.validate(testSpec.getData());
173173

174174
if (testSpec.isValid()) {
@@ -246,7 +246,7 @@ private List<Path> findTestCases(String basePath) {
246246

247247
private Stream<TestCase> loadTestCases(Path testCaseFile) {
248248
try (InputStream in = new FileInputStream(testCaseFile.toFile())) {
249-
return mapper.readValue(in, testCaseType)
249+
return this.mapper.readValue(in, testCaseType)
250250
.stream()
251251
.peek(testCase -> testCase.setSpecification(testCaseFile))
252252
.filter(this::enabled);
@@ -258,7 +258,7 @@ private Stream<TestCase> loadTestCases(Path testCaseFile) {
258258
}
259259
}
260260

261-
private Iterable<? extends DynamicNode> unsupportedMetaSchema(TestCase testCase) {
261+
private static Iterable<? extends DynamicNode> unsupportedMetaSchema(TestCase testCase) {
262262
return Collections.singleton(
263263
dynamicTest("Detected an unsupported schema", () -> {
264264
String schema = testCase.getSchema().asText();

‎src/test/java/com/networknt/schema/TestSpec.java

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,11 @@ public class TestSpec {
8181
*/
8282
private final boolean typeLoose;
8383

84+
/**
85+
* Identifies the regular expression engine to use for this test-case.
86+
*/
87+
private final RegexKind regex;
88+
8489
/**
8590
* The TestCase that contains this TestSpec.
8691
*/
@@ -107,7 +112,8 @@ public TestSpec(
107112
@JsonProperty("strictness") Map<String, Boolean> strictness,
108113
@JsonProperty("validationMessages") Set<String> validationMessages,
109114
@JsonProperty("isTypeLoose") Boolean isTypeLoose,
110-
@JsonProperty("disabled") Boolean disabled
115+
@JsonProperty("disabled") Boolean disabled,
116+
@JsonProperty(value = "regex", defaultValue = "unspecified") RegexKind regex
111117
) {
112118
this.description = description;
113119
this.comment = comment;
@@ -116,6 +122,7 @@ public TestSpec(
116122
this.validationMessages = validationMessages;
117123
this.disabled = Boolean.TRUE.equals(disabled);
118124
this.typeLoose = Boolean.TRUE.equals(isTypeLoose);
125+
this.regex = regex;
119126
if (null != strictness) {
120127
this.strictness.putAll(strictness);
121128
}
@@ -211,4 +218,13 @@ public boolean isTypeLoose() {
211218
return typeLoose;
212219
}
213220

221+
public RegexKind getRegex() {
222+
return this.regex;
223+
}
224+
225+
public static enum RegexKind {
226+
@JsonProperty("unspecified") UNSPECIFIED,
227+
@JsonProperty("ecma-262") JONI,
228+
@JsonProperty("jdk") JDK
229+
}
214230
}

‎src/test/resources/draft2020-12/issue495.json

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
[
2+
{
3+
"description": "issue495 using ECMA-262",
4+
"regex": "ecma-262",
5+
"schema": {
6+
"$schema": "https://json-schema.org/draft/2020-12/schema",
7+
"pattern": "^[a-z]{1,10}$",
8+
"unevaluatedProperties": false
9+
},
10+
"tests": [
11+
{
12+
"description": "an expected property name",
13+
"data": { "aaa": 3 },
14+
"valid": true
15+
},
16+
{
17+
"description": "trailing newline",
18+
"data": { "aaa\n": 3 },
19+
"valid": false
20+
},
21+
{
22+
"description": "embedded newline",
23+
"data": { "aaa\nbbb": 3 },
24+
"valid": false
25+
},
26+
{
27+
"description": "leading newline",
28+
"data": { "\nbbb": 3 },
29+
"valid": false
30+
}
31+
]
32+
},
33+
{
34+
"description": "issue495 using Java Pattern",
35+
"regex": "jdk",
36+
"schema": {
37+
"$schema": "https://json-schema.org/draft/2020-12/schema",
38+
"pattern": "^[a-z]{1,10}$",
39+
"unevaluatedProperties": false
40+
},
41+
"tests": [
42+
{
43+
"description": "an expected property name",
44+
"data": { "aaa": 3 },
45+
"valid": true
46+
},
47+
{
48+
"description": "trailing newline",
49+
"data": { "aaa\n": 3 },
50+
"valid": false
51+
},
52+
{
53+
"description": "embedded newline",
54+
"data": { "aaa\nbbb": 3 },
55+
"valid": false
56+
},
57+
{
58+
"description": "leading newline",
59+
"data": { "\nbbb": 3 },
60+
"valid": false
61+
}
62+
]
63+
}
64+
]

0 commit comments

Comments
 (0)