Minimal validator

[Fredi-raspall]

Addresses #1174

• Adds a simple validator, as a rust binary.
• The intent is to compile this validator as wasm/wasi and use it from fabricator.
• The validator expects a GatewayAgent CRD from stdin and produces a JSON-encoded response in stdout.
• The output JSON describes if the CRD could be validated or otherwise the problem encountered.
• The "errors" include issues that the user may not be responsible for, but which prevent the validation from happening.
• The errors are organized mimicking the sequence of steps done by dataplane when validating a config. Roughly:
  • io (specific to this tool)
  • deserialize from json (we should not see these in prod, but if CRD is manually created)
  • metadata issues (these should be integration bugs)
  • conversion to config (e.g. bad values vni = 0 or > 2^24 )
  • config errors (these are the ones the user is responsible for).
    • these type of errors contain as a list/sequence
    • however, currently we only return one element because dataplane validation early exits when the first issue is encountered.

I will open a separate PR to:
* refine the feedback and the information provided by some of the errors.
* move some checks from conversion to validation, as they pertain there.
* see if an exhaustive (non-early-exited) validation can be added.

Sample deserialization errors
In these errors, a list of lines (context) near the issue is returned. The line number is not too exact (depending on the error type it is offset by +/-1 or 2). We should seldom (if ever) see deserialization errors when the input JSON is machine generated. So this is mostly for testing manually crafted configs.

{
  "DeserializeError": {
    "hint": "trailing comma at line 8 column 3",
    "line": 8,
    "column": 3,
    "category": "Syntax",
    "context": {
      "3": "  \"metadata\": {",
      "4": "    \"generation\": 1,",
      "5": "    \"name\": \"gateway-1\",",
      "6": "    \"namespace\": \"fab\",",
      "7": "  },",
      "8": "  \"spec\": {",
      "9": "  \"agentVersion\": \"v0.32.0\",",
      "10": "  \"communities\": {",
      "11": "    \"0\": \"65000:800\",",
      "12": "    \"1\": \"65000:801\","
    }
  }
}

{
  "DeserializeError": {
    "hint": "invalid number at line 150 column 15",
    "line": 150,
    "column": 15,
    "category": "Syntax",
    "context": {
      "145": "    },",
      "146": "    \"VPC-2\": {",
      "147": "      \"internalID\": \"BBBBB\",",
      "148": "      \"subnets\": null,",
      "149": "      \"vni\": 000",
      "150": "    },",
      "151": "    \"VPC-3\": {",
      "152": "      \"internalID\": \"CCCCC\",",
      "153": "      \"subnets\": null,",
      "154": "      \"vni\": 2000"
    }
  }
}

{
  "DeserializeError": {
    "hint": "invalid type: string \"65000-BAD\", expected u32 at line 20 column 22",
    "line": 20,
    "column": 22,
    "category": "Data",
    "context": {
      "15": "    \"4\": \"65000:804\",",
      "16": "    \"5\": \"65000:805\"",
      "17": "  },",
      "18": "  \"gateway\": {",
      "19": "    \"asn\": \"65000-BAD\",",
      "20": "    \"groups\": [",
      "21": "      {",
      "22": "        \"name\": \"gw-group-1\",",
      "23": "        \"priority\": 10",
      "24": "      },"
    }
  }
}

{
  "DeserializeError": {
    "hint": "invalid value: integer `-1500`, expected u32 at line 79 column 20",
    "line": 79,
    "column": 20,
    "category": "Data",
    "context": {
      "74": "    },",
      "75": "    \"protocolIP\": \"7.0.0.100/32\",",
      "76": "    \"vtepIP\": \"7.0.0.100/32\",",
      "77": "    \"vtepMAC\": \"02:aa:bb:cc:dd:ee\",",
      "78": "    \"vtepMTU\": -1500,",
      "79": "    \"workers\": 1",
      "80": "  },",
      "81": "  \"groups\": {",
      "82": "    \"gw-group-1\": {",
      "83": "      \"members\": ["
    }
  }
}

Sample Metadata errors

{
  "MetadataError": "Missing gateway name"
}

{
  "MetadataError": "Missing generation Id"
}

Sample Conversion errors

{
  "ConversionError": "Invalid Gateway Agent object: Could not create VPC: Bad VPC Id"
}

{
  "ConversionError": "Could not parse value: Invalid CIDR format: 192.168.90.0/34: Invalid Prefix: 192.168.90.0/34"
}

{
  "ConversionError": "Invalid Gateway Agent object: Could not create VPC: '0' is not a valid VNI"
}

Sample Configuration errors

{
  "Configuration": {
    "errors": [
      "A VPC peering object refers to non-existent VPC 'VPC-1'"
    ]
  }
}

Success case

"Success"

[Frostman]

@Fredi-raspall, we need to agree on the output format; the current one would be very difficult to consume to produce a user-friendly message and difficult to parse on top of it. I'd suggest to at least flatten it a bit into list of errors and move the error type into field type instead of it being a key and always have an object as a root, e.g.:

{
  "success": true
}

or

{
  "errors": [
{
   "type": "DeserializeError",
    "message": "invalid value: integer `-1500`, expected u32 at line 79 column 20",
    "line": 79,
    "column": 20,
    "category": "Data",
    "context": {
      "74": "    },",
      "75": "    \"protocolIP\": \"7.0.0.100/32\",",
      "76": "    \"vtepIP\": \"7.0.0.100/32\",",
      "77": "    \"vtepMAC\": \"02:aa:bb:cc:dd:ee\",",
      "78": "    \"vtepMTU\": -1500,",
      "79": "    \"workers\": 1",
      "80": "  },",
      "81": "  \"groups\": {",
      "82": "    \"gw-group-1\": {",
      "83": "      \"members\": ["
    }
},
{
  "type": "ConversionError",
  "message": "Invalid Gateway Agent object: Could not create VPC: '0' is not a valid VNI"
},
{
  "type": "ConfigurationError",
  "message": "blah blah 1"
},
{
  "type": "ConfigurationError",
  "message": "blah blah 2"
},
{
  "type": "ConfigurationError",
  "message": "blah blah 3"
},
  ]
}

I additionally suggest renaming hint to message and making it mandatory for all errors to have a message.

[Frostman]

newline missing

[Fredi-raspall]

@Fredi-raspall, we need to agree on the output format; the current one would be very difficult to consume to produce a user-friendly message and difficult to parse on top of it. I'd suggest to at least flatten it a bit into list of errors and move the error type into field type instead of it being a key and always have an object as a root, e.g.:

Ok, I'll see what I can do. Note that some of the errors are exclusive. I.e., if we fail to deserialize, we'll not be able to validate the config. So you'll never see "DeserializeError" along with "ConversionError".

I additionally suggest renaming hint to message and making it mandatory for all errors to have a message.

Ok!

[Fredi-raspall]

@Frostman , I've pushed a new version with a simplified output.

Samples

{
  "success": true,
  "errors": []
}

{
  "success": false,
  "errors": [
    {
      "type": "Metadata",
      "message": "Invalid generation Id"
    }
  ]
}

{
  "success": false,
  "errors": [
    {
      "type": "Deserialization",
      "subtype": "Syntax",
      "message": "trailing comma at line 8 column 3"
    }
  ]
}

{
  "success": false,
  "errors": [
    {
      "type": "Deserialization",
      "subtype": "Data",
      "message": "invalid value: integer `-2000`, expected u32 at line 155 column 18"
    }
  ]
}

{
  "success": false,
  "errors": [
    {
      "type": "Configuration",
      "message": "A VPC peering object refers to non-existent VPC 'VPC-1'"
    }
  ]
}

I wonder if we should add an optional "rule" field (or "rule-violation") that explains the user the reason why a configuration may be rejected in general terms. E.g.

{
  "success": false,
  "errors": [
    {
      "type": "Configuration",
      "message": "The peering configuration of VPC VPC-1 is not correct:  peering X is bad"
      "rule": "A VPC can only peer with ... blah, blah. 
      Suggestion: either you do X or you do Y. Both are not currently supported."
    }
  ]
}

[Frostman]

Tried the latest binary @Fredi-raspall shared in Slack, and it looks ok. I think we can merge in that state and main work would be to make all validation messages as user friendly as possible.