Skip to content

Commit fe25ba9

Browse files
gregsdennisgregsdennis
committed
updated readme; fixed $schema typo; tests for 2020-12
1 parent 87c184d commit fe25ba9

File tree

5 files changed

+141
-4
lines changed

5 files changed

+141
-4
lines changed

‎output-tests/README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,14 @@
11
These tests are intended to validate that implementations are correctly generating output in accordance with the specification.
22

3-
Output was initially specified with draft 2019-09. It remained unchanged for draft 2020-12 <!-- do we need explicit tests for 2002-12? -->, but will receive an update with the next release.
3+
Output was initially specified with draft 2019-09. It remained unchanged for draft 2020-12, but will receive an update with the next release.
4+
5+
***NOTE** Although the formats didn't change between 2019-09 and 2020-12, the tests are replicated for 2020-12 because the `$schema` is different and implementations may (but shouldn't) produce different output.*
46

57
## Organization
68

7-
The tests are organized into two categories: content and structure.
9+
The tests are organized by specification release and then into two categories: content and structure.
810

9-
Content tests verify that the keywords are producing the correct annotations. Since there are no requirements on the content of error messages, there's not much that can be verified for them, if they're even generated. These tests need to extensively cover the annotation behaviors of each keyword. The expected output format for these tests is `basic`.
11+
Content tests verify that the keywords are producing the correct annotations and/or error messages. Since there are no requirements on the content of error messages, there's not much that can be verified for them, but it is possible to identify when a error message _could_ be present. Primarily, these tests need to extensively cover the annotation behaviors of each keyword. The only output format needed for these tests is `basic`.
1012

1113
Structure tests verify that the structures of the various formats (i.e. `flag`, `basic`, `detailed`, `verbose`) are correct. These tests don't need to cover each keyword; rather they need to sufficiently cover the various aspects of building the output structures by using whatever keywords are necessary to do so.
1214

‎output-tests/draft2019-09/content/general.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
{
33
"description": "failed validation produces no annotations",
44
"schema": {
5-
"$schema": "https://json-schema.org/draft/2020-12/schema",
5+
"$schema": "https://json-schema.org/draft/2019-09/schema",
66
"$id": "https://json-schema.org/tests/content/draft2019-09/general/1",
77
"type": "string",
88
"readOnly": true

‎output-tests/draft2020-12/content/general.json

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
[
2+
{
3+
"description": "failed validation produces no annotations",
4+
"schema": {
5+
"$schema": "https://json-schema.org/draft/2020-12/schema",
6+
"$id": "https://json-schema.org/tests/content/draft2020-12/general/1",
7+
"type": "string",
8+
"readOnly": true
9+
},
10+
"tests": [
11+
{
12+
"description": "readOnly annotation is dropped",
13+
"data": 1,
14+
"output": {
15+
"basic": {
16+
"type": "object",
17+
"properties": {
18+
"errors": {
19+
"type": "array",
20+
"items": {
21+
"properties": {
22+
"annotation": false
23+
}
24+
}
25+
},
26+
"annotations": false
27+
},
28+
"required": ["errors"]
29+
}
30+
}
31+
}
32+
]
33+
}
34+
]

‎output-tests/draft2020-12/content/readOnly.json

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
[
2+
{
3+
"description": "readOnly generates its value as an annotation",
4+
"schema": {
5+
"$schema": "https://json-schema.org/draft/2020-12/schema",
6+
"$id": "https://json-schema.org/tests/content/draft2020-12/readOnly/1",
7+
"readOnly": true
8+
},
9+
"tests": [
10+
{
11+
"description": "readOnly is true",
12+
"data": 1,
13+
"output": {
14+
"basic": {
15+
"type": "object",
16+
"properties": {
17+
"annotations": {
18+
"type": "array",
19+
"contains": {
20+
"type": "object",
21+
"properties": {
22+
"keywordLocation": {"const": "/readOnly"},
23+
"absoluteKeywordLocation": {"const": "https://json-schema.org/tests/content/draft2020-12/readOnly/1#/readOnly"},
24+
"instanceLocation": {"const": ""},
25+
"annotation": {"const": true}
26+
},
27+
"required": ["keywordLocation", "absoluteKeywordLocation", "instanceLocation", "annotation"]
28+
}
29+
},
30+
"errors": false
31+
},
32+
"required": ["annotations"]
33+
}
34+
}
35+
}
36+
]
37+
}
38+
]

‎output-tests/draft2020-12/content/type.json

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
[
2+
{
3+
"description": "validating type",
4+
"schema": {
5+
"$schema": "https://json-schema.org/draft/2020-12/schema",
6+
"$id": "https://json-schema.org/tests/content/draft2020-12/type/1",
7+
"type": "string"
8+
},
9+
"tests": [
10+
{
11+
"description": "incorrect type must be reported, but a message is not required",
12+
"data": 1,
13+
"output": {
14+
"basic": {
15+
"type": "object",
16+
"properties": {
17+
"errors": {
18+
"contains": {
19+
"type": "object",
20+
"properties": {
21+
"keywordLocation": {"const": "/type"},
22+
"absoluteKeywordLocation": {"const": "https://json-schema.org/tests/content/draft2020-12/type/1#/type"},
23+
"instanceLocation": {"const": ""},
24+
"annotation": false,
25+
"error": {"type": "string"}
26+
},
27+
"required": ["keywordLocation", "absoluteKeywordLocation", "instanceLocation"]
28+
}
29+
}
30+
},
31+
"required": ["errors"]
32+
}
33+
}
34+
},
35+
{
36+
"description": "correct type yields an output unit",
37+
"data": 1,
38+
"output": {
39+
"basic": {
40+
"type": "object",
41+
"properties": {
42+
"annotations": {
43+
"contains": {
44+
"type": "object",
45+
"properties": {
46+
"valid": {"const": true},
47+
"keywordLocation": {"const": "/type"},
48+
"absoluteKeywordLocation": {"const": "https://json-schema.org/tests/content/draft2020-12/type/1#/type"},
49+
"instanceLocation": {"const": ""},
50+
"annotation": false,
51+
"error": false
52+
},
53+
"required": ["keywordLocation", "absoluteKeywordLocation", "instanceLocation"]
54+
}
55+
}
56+
},
57+
"required": ["annotations"]
58+
}
59+
}
60+
}
61+
]
62+
}
63+
]

0 commit comments

Comments
 (0)