JSON Schema 2019-09 support

[keegan-caruso]

Overview

This PR adds further support for JSON Schema draft 2019-09.
The changes focus on three main areas:

1. Vocabulary-based keyword filtering
2. $recursiveRef resolution
3. Improved $ref handling with embedded schemas

This is a very large change, happy to break it up however would be easiest to review.

Key Changes

1. Vocabulary Support ($vocabulary)

Files: vocabularies.ts (new), jsonSchemaService.ts, jsonParser.ts, jsonLanguageTypes.ts

JSON Schema 2019-09 introduced vocabularies, which allow meta-schemas to declare which keyword sets are active. This enables custom meta-schemas to disable certain validation keywords.

Implementation

• Added new Vocabularies type (Set<string>) to represent active vocabulary URIs
• Created isKeywordEnabled() function that checks if a keyword should be processed based on active vocabularies
• The schema service now fetches the meta-schema (via $schema) and extracts $vocabulary declarations
• Validation functions now call enabled(keyword) before processing each keyword
• Core keywords ($id, $ref, $schema, etc.) are always enabled per the spec

Example

A meta-schema with only applicator vocabulary (no validation vocabulary) will skip minimum, required, type, etc.

2. $recursiveRef and $recursiveAnchor Support

Files: jsonParser.ts, jsonSchema.ts

These keywords (2019-09) enable extensible recursive schemas—a pattern where a base schema can be extended while maintaining recursion to the extending schema.

Recursive Ref Implementation

• Added schemaStack and schemaRoots parameters to the validate() function to track schema traversal
• When $recursiveRef is encountered:
  1. Find the nearest schema resource root (schema with $id)
  2. If it has $recursiveAnchor: true, find the first $recursiveAnchor: true in the stack
  3. Otherwise, use the current resource root or document root
  4. Validate against the resolved target schema
• Fixed $recursiveAnchor type from string to boolean | string to handle both spec-correct and real-world schemas

3. Improved $ref Resolution with Embedded $id Schemas

Files: jsonSchemaService.ts

Schemas can contain embedded sub-schemas with their own $id, creating new URI scopes. This PR properly resolves $ref against the correct base URI.

Key Fixes

Function	Purpose
traverseWithBaseTracking()	Maintains the current base URI as it traverses schemas with embedded $id values
registerEmbeddedSchemas()	Pre-registers all embedded schemas so they can be resolved as external refs
collectAnchors()	Now stops at embedded $id boundaries since anchors are scoped to their document

4. Scope Isolation for $ref with Sibling Keywords

Files: jsonSchemaService.ts

In 2019-09+, $ref can have sibling keywords, but unevaluatedProperties/unevaluatedItems in the referenced schema shouldn't see properties evaluated by those siblings.

Scope Isolation Implementation

• Added needsScopeIsolation() to detect when isolation is needed
• When needed, siblings are moved into a separate schema and combined with allOf:

{
  "allOf": [
    { /* $ref'd schema */ },
    { /* sibling keywords */ }
  ]
}

5. dependentSchemas Integration with unevaluatedProperties

Files: jsonParser.ts

Properties validated by dependentSchemas were not being tracked as "evaluated," causing false positives with unevaluatedProperties: false.

Fix: Added validationResult.mergeProcessedProperties() after validating dependentSchemas.

6. patternProperties Handling Fix

Files: jsonParser.ts

Previously, patternProperties was processed incrementally, which caused issues when a property matched multiple patterns.

Fix: Collect all pattern matches first, validate against all matching patterns, then mark as processed.

7. New Format Validators

Files: jsonParser.ts

Added validators for two new 2019-09 formats:

Format	Description	Example
duration	ISO 8601 duration	P1Y2M3DT4H5M6S
uuid	UUID format	550e8400-e29b-41d4-a716-446655440000

8. Type Definition Updates

Files: jsonSchema.ts

Property	Old Type	New Type
$recursiveAnchor	string	boolean | string
$vocabulary	any	{ [uri: string]: boolean }

Testing

New Test Files

• vocabularies.test.ts: Unit tests for isKeywordEnabled() covering all vocabulary combinations

Updated Test Files

• parser.test.ts: Added tests for:

  • $recursiveRef
  • dependentSchemas + unevaluatedProperties
  • duration and uuid formats

• schema.test.ts: Added tests for:

  • Vocabulary integration
  • Unsupported feature warnings

• jsonSchemaTestSuite.test.ts:

  • Added schema request service to load remote schemas from test suite
  • Reduced skipped tests from ~100 to ~35

Tests Now Passing

The following test categories now pass:

• $id resolution
• patternProperties
• unevaluatedProperties
• $recursiveRef

Unsupported Features

The following 2020-12 features are detected and produce warnings:

• $dynamicRef
• $dynamicAnchor

Breaking Changes

None. All changes are backward compatible.

Remaining Skipped Tests

Most remaining skipped tests fall into two categories:

1. Remote refs: Tests requiring HTTP fetches to localhost:1234 (partially addressed)
2. $dynamicRef/$dynamicAnchor: 2020-12 features not yet implemented

[keegan-caruso]

cc @aeschli

[jdesrosiers]

Looks good. This is a huge step toward being fully compliant with the JSON Schema spec, but there's definitely a few pieces still missing. I expect that this isn't intended to implement everything, but it's way closer than what we have now.

The following list is all the things I tested. I tested it with these changes running in vscode. Most of the issues are related to identifying which keywords should be recognized/ignored depending on the version of JSON Schema in use. It respects all implemented keywords regardless of what version of JSON Schema the schema uses. That didn't surprise me. I know this implementations doesn't make those distinctions, but I did notice in the code that you tried to add support for those things by implementing the $vocabulary keyword. However, I didn't see any indication that it was working at all. I see there are tests, but when I try those same scenarios in vscode, I don't see $vocabulary being respected.

The other thing I noticed is that in 2019-09, the format keyword is supposed to be an annotation only. You're allowed to provide a way to enable format validation, but it's not supposed to be enabled by default. I understand if the intention is enable this by default, but technically that's not correct.

In 2019-09, you can also enable/disable format using $vocabulary. false means annotation-only and true means validate. This can be overridden by configuration, but those are the defaults. It would be nice to see that working fully.

• ❌ Relative reference with $id
  • This has never worked properly, not just with 2019-09 changes. I'm sure it wasn't intended for this to be in scope for these changes, but it would be nice to get this working properly at some point. I don't think it would be too hard compared to everything else you've achieved in this PR.
• ✔️ Reference an $anchor in the same schema
• ✔️ Reference an $anchor in an external schema
• ✔️ Reference an $anchor in an embedded schema
• ❌ $id fragments aren't anchors in a 2020-12 schema
• ❌ $anchor isn't an anchor in a draft-07/6/4 schema
• ✔️ embedded schemas
• ✔️ nested embedded schemas
• ✔️ recursive references
• ✔️ $ref with siblings
• ✔️ dependentRequired works in 2019-09
• ❌ dependentRequired shouldn't work in draft-07
• ❌ dependentSchemas works in 2019-09
  • Works, but the error is on the wrong node. It appears on the property key, not the property value.
• ❌ dependentSchemas shouldn't work in draft-07
• ❌ dependencies shouldn't work in 2019-09
• ✔️ unevaluatedItems
• ✔️ unevaluatedProperties
• ✔️ unevalautedProperties with dependentSchemas
• ✔️ minContains/maxContains
• ❌ format doesn't validate by default
• ❌ Enable format validation using $vocabulary
• ❌ Disable vocab using $vocabulary

One more thing. I'm not sure if there was something missing in my testing setup or what, but I wasn't seeing vscode recognize the changes in my schemas while editing. I had to refresh (Ctrl+R) in order for the schema changes to be recognized in the JSON document that linked it. If that's related to these changes, that could be a concern, but I expect it's just my setup.

Thanks for this work! It will be huge for JSON Schema to get this and 2020-12 support in vscode.

[keegan-caruso]

@jdesrosiers

• I thought dependentRequired shouldn't work in draft-07, instead I thought it still used dependencies. e.g. https://json-schema.org/understanding-json-schema/reference/conditionals
• dependencies working in 2019-09: I see it, I was using the vocabulary check for this, but dependencies is not in vocabularies since that is a 2019-09 or greater feature.
• I believe I addressed your comment around format assertions but let me know if anything needs adjusted.
• Adjusted id/$id/$anchor handling
• I had an error in extractVocabularies where I wasn't including them all, this might have been what you were seeing with VS Code?

[jdesrosiers]

I think there was some confusion because I wasn't clear enough in my checklist. I haven't had a chance to test this round of changes yet, but I want to make sure there aren't any miscommunications.

• I thought dependentRequired shouldn't work in draft-07, instead I thought it still used dependencies.

That's correct. dependentRequired should be ignored in a draft-07 schema. I'm not seeing dependentRequired be ignored when testing with a draft-07 schema.

dependencies working in 2019-09: I see it, I was using the vocabulary check for this, but dependencies is not in vocabularies since that is a 2019-09 or greater feature.

dependencies should be ignored in draft-2019-09. My tests showed that it was not ignored. I think these are all related to the vocabulary check. Maybe I'm missing something, but it doesn't appear that the vocabulary check is working at all.

[jdesrosiers]

I believe I addressed your comment around format assertions but let me know if anything needs adjusted.

I'm still seeing format being evaluated. Example,

{
  "$schema": "./my-schema.json",
  "date": "not a date" // <-- Should be ok, but has a validation error
}

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "type": "object",
  "properties": {
    "date": {
      "type": "string",
      "format": "date" // <-- Should be annotation-only in 2019-09
    }
  }
}

Adjusted id/$id/$anchor handling

Mostly looks right except for one case.

{
  "$schema": "./my-schema.json",
  "foo": 42 // <-- This is showing that the value should be a string, but the reference shouldn't have worked.
}

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "type": "object",
  "properties": {
    "foo": { "$ref": "#foo" } // <-- The reference target shouldn't exist
  },
  "$defs": {
    "": {
      "$id": "#foo", // <-- Should not create an anchor in draft-2019-09
      "type": "string"
    }
  }
}

I had an error in extractVocabularies where I wasn't including them all, this might have been what you were seeing with VS Code?

I think this is working better. I'm seeing draft-2019-09 schemas allowing only draft-2019-09 keywords. But, in draft-07 schemas, all keywords seem to work including draft-2019-09 keywords.

Also, the vocabulary filtering only seems to work for built-in schemas like draft-2019-09. I'm not seeing it work for custom meta-schemas. Not sure if that was intended or not, but it looks like you wrote tests for it so I expected it to work.

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$vocabulary": {
    "https://json-schema.org/draft/2019-09/vocab/core": true,
    "https://json-schema.org/draft/2019-09/vocab/applicator": true
  },

  "$ref": "https://json-schema.org/draft/2019-09/schema"
}

{
  "$schema": "./my-dialect.json",

  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "minimum": 0 } // <-- "minimum" should do nothing
  },
  "required": ["name"] // <-- "required" should do nothing
}

{ // <-- I'm seeing an error from "required" for "name" not being included, but "required" should be ignored.
  "$schema": "./my-schema.json",
  "age": -3 // <-- I'm seeing an error from "minimum" that should be ignored
}

My checklist so far:

• ❌ Relative reference with $id
  • This has never worked properly, not just with 2019-09 changes. I'm sure it wasn't intended for this to be in scope for these changes, but it would be nice to get this working properly at some point. I don't think it would be too hard compared to everything else you've achieved in this PR.
• ✔️ Reference an $anchor in the same schema
• ✔️ Reference an $anchor in an external schema
• ✔️ Reference an $anchor in an embedded schema
• ❌ $id fragments aren't anchors in a 2019-09 schema
• ✔️ $anchor isn't an anchor in a draft-07/6/4 schema
• ✔️ embedded schemas
• ✔️ nested embedded schemas
• ✔️ recursive references
• ✔️ $ref with siblings
• ✔️ dependentRequired works in 2019-09
• ❌ dependentRequired shouldn't work in draft-07
• ✔️ dependencies shouldn't work in 2019-09
• ❌ dependentSchemas works in 2019-09
  • This works, but the error is still not quite in the right place. It's now on the value, but it's just on the opening curly brace, not the whole object like dependentRequired.
• ❌ dependentSchemas shouldn't work in draft-07
• ❌ dependencies with a schema should work in draft-07
  • This works, but the error is not in the right place. It's on the value, but it's just on the opening curly brace, not the whole object like dependencies with an array.
• ✔️ unevaluatedItems
• ✔️ unevaluatedProperties
• ✔️ unevalautedProperties with dependentSchemas
• ✔️ minContains/maxContains
• ❌ format shouldn't validate by default
• ❌ Enable format validation using $vocabulary
• ❌ Disable vocab using $vocabulary