JSON schema validation with multiple conditions

[emanuelenasoae]

I am experimenting with json schema validation and I stumbled across an issue

I have a schema that is using the oneOf Keyword and I am using List format.
I read this blog post in order to try to understand how to read the output: https://json-schema.org/blog/posts/interpreting-output#output-includes-why

For a simple schema in List format, things are clear.

But when I add the oneOf keyword in order to reference two different local schemas, I am struggling to read the validation result.

In the blog post it says: Here, it becomes obvious that we can't just scan for errors because we have to consider where those errors are coming from. The error in the last output node only pertains to the /if subschema, which (as mentioned before) doesn't affect the validation result.

How should I take into consideration where those errors are coming from? Is there a common way of reading the validation result and make sure I am not reading a "failed validation" which is actually coming from an if statement? I mean, I need to make sure that the oneOf keyword works fine, but but when I navigate through the validation results, I might find and report "false positives" because as it says here, The output includes why.

[gregsdennis]

One tactic that I've found works well is to use the hierarchical ("verbose", maybe) format and follow the valid: falses. As you go deeper into the tree, you can ignore the valid: trues because they're not the problem (unless you have a not keyword).

This process will highlight the problems you need to fix. Any "false positives" will occur in failed subschemas from within passing subschemas.

Schema

{
  "anyOf": [
    {"type": "integer"},
    {"type": "string"}
  ]
}

Instance: "dog"

Output:

{
  "valid": true,
  "evaluationPath": "",
  "schemaLocation": "https://json-everything.net/f14e9c0a25#",
  "instanceLocation": "",
  "details": [
    {
      "valid": false,
      "evaluationPath": "/anyOf/0",
      "schemaLocation": "https://json-everything.net/f14e9c0a25#/anyOf/0",
      "instanceLocation": "",
      "errors": {
        "type": "Value is \"string\" but should be \"integer\""
      }
    },
    {
      "valid": true,
      "evaluationPath": "/anyOf/1",
      "schemaLocation": "https://json-everything.net/f14e9c0a25#/anyOf/1",
      "instanceLocation": ""
    }
  ]
}

The important part is that valid: true. You don't need to look for errors. If you do, you'll find one in the first subschema. But it doesn't matter that that subschema failed because its parent passed. This subschema result is a false positive.

It's not hard to then imagine this structure more deeply in a larger schema that has an overall failing result. If you just simply scan for errors in a flat structure, you'll find the error from this subschema. But if you had followed the valid: falses in the hierarchical structure, you'd have filtered this one out because it's parent result is valid: true.

I hope all of that makes sense.

[gregsdennis]

It'll depend on the implementation, schema, and instance. For example, my JsonSchema.Net builds the Hierarchical by default, so moving to List incurs a small transformation cost. Additionally, a recursive schema could perform differently for a deep vs shallow JSON instance.

[emanuelenasoae]

Coming back to this as something still doesn't work as expected..

So the above proposed solution works for avoiding any "false positives" that will occur in failed subschemas from within passing subschemas.
So I switched to hierarchical and I'm only going deeper if the isValid is false. The problem is that at some point it finds isValid: false and errors !=null for a node of the oneOf..

{
  "type": "test1",
  "data": {
    "input": {
      "cont": "test",
      "filename": "test"
    },
    "output": {
      "container": "test2",
      "filename": "test2"
    },
    "events": [
      {
        "type": "type.value2",
        "data": {
          "input": {
            "container": "test3",
            "filename": "test4"
          },
          "output": {
            "container": "test5",
            "filename": "test6"
          }
        }
      }
    ]
  }
}

there is part of a bigger "events" array, the above is a value from that array
this structure is validated by a oneOf keyword. as part of that, the type can eiter have value "test" or "test1"

required values for input are "container" and "filename"

as a test, in the data node I changed "container" with "cont"..because of this change, my results now include the following errors:

1. events/1 : expected 1 matching subschemas but found 0
2. data/events/1/data/input: required properties "container" are not present
3. data/events/1/type: the string value is not match for the indicated regular expression

I can understand the first and second error, buy why does it complain about the data/events/1/type? It's value is "test1" and is oneOf the accepted ones.
When navigating through the hierarchy, the result of the data/events/1/type has isValid: false and it contains that error.
Since it's part of a oneOf, the expectation would be that this node would have a parent with isValid: true so that it won't go in and read the error..
Looks like somehow the change to "cont" allowed the code to reach the the "false positive" error.

if it help, I added below the code used to read the validation messages

private List<(string instanceLocation, string message)> GetAllErrors(EvaluationResults result)
{
    var errors = new List<(string, string)>();

    if (!result.IsValid)
    {
        if (result.HasErrors && result.Errors != null)
        {
            foreach (var error in result.Errors)
            {
                errors.Add((result.InstanceLocation.ToString(), error.Value));
            }
        }

        if (result.Details != null && result.Details.Count > 0)
        {
            foreach (var detail in result.Details)
            {
                errors.AddRange(GetAllErrors(detail)); // Recurse only into failed evaluation results
            }
        }
    }
    return errors;
}

[emanuelenasoae]

@gregsdennis
So basically I have:

• one failed oneOf subschema (the false positive one, failed because of the type)
• one failed oneOf subschema (this one failed not because of the type, but because of the "cont" instead of "container"
• the parent subschema now has the isValid: false because both of it's subschemas are invalid
  Result: both the false positive error and the actual error are being read/ returned in the final list of validation errors...

[gregsdennis]

This is what I meant by the validator not being able to infer intent. When a oneOf with two subschemas has both subschemas fail, it can't know which of the subschemas was intended to pass, so it reports errors on both of them. In this case, it's not a false positive. Fixing the error for either subschema will make it pass. You have to decide which one was supposed to pass and address that one.

[emanuelenasoae]

Thanks for your answers!