Implement '_invalid_with' data validation option (#2849)

Some attributes (for example, 'set' in a routing policy entry) cannot
be used with other attributes, or when another attribute has a specific
value. This commit adds '_invalid_with' data validation option and
uses it in routing policy entry definition to resolve #2841

Author: ipspace

docs/dev/validation.md

@@ -122,11 +122,17 @@ All attributes defined with a dictionary (**mode** in the above example, but not
 * **true_value** -- value to use when the parameter is set to *True*
 * **_requires** -- a list of modules that must be enabled in global- or node context to allow the use of this attribute. See `vrfs` in `modules/vrf.yml` and `vlans` in `modules/vlan.yml` for more details.
 * **_required** (bool) -- the attribute must be present in the parent dictionary[^CRQ]
+* **_valid_with** -- a list or dictionary of attributes that can be used with this attribute ([example](inter-attribute-examples)).
+* **_invalid_with** -- specifies attributes that cannot be used together with this attribute. Can be:
+
+  - A list of attribute names (this attribute cannot be used when any of the listed attributes are present)
+  - A dictionary where keys are attribute names and values specify that the conflict occurs when the conflicting attribute has (one of the) the specified value(s)
+
 * **_alt_types** -- [alternate data types](validation-alt-types)
 
 [^CRQ]: This does not make the parent dictionary mandatory, but if it's present, it must have the required attribute. Use a chain of `_required` attributes if you want to enforce the presence of an attribute deep in the data structure.
 
-See [](validation-definition-examples) for more details.
+See [](validation-definition-examples) and [](inter-attribute-examples) for more details.
 
 ### Further Data Type Validation Options
 
@@ -275,6 +281,40 @@ mpls:
       6pe: { type: list, true_value: [ ibgp ] }
 ```
 
+(inter-attribute-examples)=
+### Attribute Dependency Validation Examples
+
+The **delete.community.list** in a routing policy entry cannot be used with any other **delete.community** attributes. **‌**
+
+```
+      delete:
+        community:
+          standard: list
+          extended: list
+          large: list
+          list:
+            type: str
+            _valid_with: [ ]
+
+```
+
+The **set** or **delete** attributes of a routing policy cannot be used when the **action** attribute is set to **deny** (because the route is dropped anyway):
+
+```
+  attributes:
+    rp_entry:                       # Define routing policy entry
+      _description: Routing policy entry
+      action:
+        type: str
+        valid_values: [ permit, deny ]
+      set:
+        _invalid_with: { action: deny }
+        ...
+      delete:
+        _invalid_with: { action: deny }
+        ...
+```
+
 (validation-shortcut-type)=
 ## Shortcut Data Type Definitions
 

netsim/data/validate.py

@@ -424,28 +424,59 @@ def check_valid_with(
   if log.debug_active('validate'):
     print(f'check_valid_with {path} {data} against {data_name} {data_type}')
 
-  attr_list = list(data.keys())                             # Get the list of data attributes
   _keys = data_type.get('_keys',data_type)                  # Get the definition of keys
   report_list: typing.List[str] = []                        # List of already-reported incompatibilities
 
-  for attr in attr_list:
-    if not isinstance(_keys.get(attr,None),Box):            # Skip invalid attributes or simple types
-      continue
+  # Get the list of data attributes with complex types
+  #
+  data_attr = [ k for k in data.keys() if not isinstance(k,str) or not k.startswith('_') ]
+  cplx_attr = [ k for k in data_attr if isinstance(_keys.get(k,None),Box) ]
+
+  # Check 'valid with' restrictions
+  #
+  for attr in cplx_attr:
     if '_valid_with' not in _keys[attr]:                    # Attribute has no restrictions
       continue
     if attr in report_list:                                 # Incompatibility has already been reported
       continue
-    inv_list = [ x for x in attr_list
+    inv_list = [ x for x in data_attr
                   if x != attr and
-                     x not in _keys[attr]._valid_with and
-                     not x.startswith('_') ]                # Build a list of incompatible attributes
+                     x not in _keys[attr]._valid_with ]     # Build a list of incompatible attributes
     if inv_list:                                            # ... and report them
       log.error(
         f'Attribute(s) {",".join(inv_list)} cannot be used with attribute {attr} in {path}',
         category=log.IncorrectAttr,
         module=module)
       report_list = report_list + inv_list
 
+  # Check 'invalid with' restrictions
+  #
+  for attr in cplx_attr:
+    if '_invalid_with' not in _keys[attr]:                  # Attribute has no incompatibility specs
+      continue
+    if attr in report_list:                                 # Incompatibility has already been reported
+      continue
+
+    invalid_with = _keys[attr]._invalid_with                # Get the incompatible attribute specs
+    cplx_invalid = isinstance(invalid_with,Box)             # And figure out whether it's a simple list or a box
+
+    for inv_attr in invalid_with:                           # Now iterate over potential incompatible attributes
+      if inv_attr not in data:                              # Incompatible attribute not in data structure
+        continue                                            # ... we're OK, move on
+    
+      # Get the optional list of incompatible values
+      inv_values = invalid_with[inv_attr] if cplx_invalid else None
+      if inv_values is None:
+        log.error(
+          f"Attribute '{attr}' cannot be used together with '{inv_attr}' in {path}",
+          category=log.IncorrectAttr,
+          module=module)
+      elif data[inv_attr] in inv_values if isinstance(inv_values,list) else data[inv_attr] == inv_values:
+        log.error(
+          f"Attribute '{attr}' cannot be used together with '{inv_attr}' set to '{data[inv_attr]}' in {path}",
+          category=log.IncorrectAttr,
+          module=module)
+
 """
 validate_item -- validate a single item from an object:
 

netsim/modules/routing.yml

@@ -92,6 +92,7 @@ _top:                               # Modification of global defaults
         min_value: 1
         max_value: 32767
       set:
+        _invalid_with: { action: deny }
         locpref:
           type: int
           min_value: 0
@@ -124,6 +125,7 @@ _top:                               # Modification of global defaults
             type: str
             _valid_with: [ ]
       delete:
+        _invalid_with: { action: deny }
         community:
           standard: list
           extended: list

tests/coverage/errors/attr-invalid-with.log

@@ -0,0 +1,2 @@
+IncorrectAttr in routing: Attribute 'set' cannot be used together with 'action' set to 'deny' in topology.routing.policy.p1[1]
+Fatal error in netlab: Cannot proceed beyond this point due to errors, exiting

tests/coverage/errors/attr-invalid-with.yml

@@ -0,0 +1,16 @@
+# Test the "normalize and merge" routing policy processing
+#
+
+module: [routing]
+defaults.device: none
+
+routing.policy:
+  p1:
+  - locpref: 10
+    action: deny
+  - set.med: 100
+
+nodes:
+  r1:
+    routing.policy:
+      p1:                   # Copy from global