Add 'color', 'width' and 'fill' attributes to GraphViz and D2 graphs (#2569)

* Add 'color', 'width' and 'fill' to shared attributes, so D2 graphs
  can use the 'graph' values
* Map the well-known graph attributes to Graph Language (GL)-
  specific node/edge attributes (D2 and GraphViz)
* Use the 'style_map' graph settings to map well-known attributes to
  GL-specific attributes, giving the users infinite extensibility.
* Add support for GraphViz node and edge formatting attributes

Author: ipspace

docs/outputs/d2.md

@@ -11,7 +11,7 @@ A single formatting modifier can be used to specify the graph type:
 * **bgp** -- Include autonomous systems, nodes, and BGP sessions. With the **rr** option (specified with `netlab create -o graph:bgp:rr`), RR-client sessions are drawn as directed arrows.
 
 ```{tip}
-The network topology graph description contains nodes and links, but no placement information. D2 is pretty good at figuring out how to draw the required graph, but it pays off to test out the [layout engines](https://d2lang.com/tour/layouts). Changing the [order of links](outputs-d2-style-attributes) might also unclutter the diagrams.
+The network topology graph description contains nodes and links, but no placement information. D2 is pretty good at figuring out how to draw the required graph, but it pays off to test out the [layout engines](https://d2lang.com/tour/layouts). Changing the [order of links](outputs-d2-link-node-attributes) might also unclutter the diagrams.
 ```
 
 ## Modifying Graph Attributes
@@ -27,15 +27,20 @@ Graphing routines use **[default](topo-defaults)** topology settings to modify t
 
 [^DG]: The results look disgusting. If you find a better way to get it done, please submit a PR. Thank you!
 
-(outputs-d2-style-attributes)=
+(outputs-d2-link-node-attributes)=
 ## Modifying Link and Node Attributes
 
 You can use the **d2** link and node attributes to change the style of individual nodes or links. The following attributes are built into _netlab_ (but see also [](outputs-d2-styles)):
 
 * **d2.color** -- line color (*stroke* in D2 lingo)
-* **d2.width** -- line width (*stroke-width*) in D2 lingo)
+* **d2.fill** -- fill color (*fill* in D2 lingo)
+* **d2.width** -- line width (*stroke-width* in D2 lingo)
 
-You can also use the **d2.linkorder** link attribute to change the order of links in the D2 graph description file, which can sometimes improve the diagrams' appearance. Links with lower **d2.linkorder** value (default: 100) appear earlier in the list of links.
+You can also use the **d2.linkorder** link attribute to change the order of links in the D2 graph description file, which can sometimes improve the diagrams' appearance. Links with lower **d2.linkorder** values (default: 100) appear earlier in the list of links.
+
+```{tip}
+The generic link- and node attributes can also be specified as **graph._attribute_** (for example, **graph.color**) values to use the same settings for GraphViz and D2 graphs.
+```
 
 ## Modifying Shape and Connection Attributes
 
@@ -133,10 +138,10 @@ You can define your own link/node style attributes:
 defaults.outputs.d2.attributes.node.background: str
 ```
 
-* Define a mapping between your attribute and D2 style attribute within the **defaults.outputs.d2.styles** dictionary. For example, your **d2.background** attribute maps into D2 **style.fill** attribute:
+* Define a mapping between your attribute and D2 style attribute within the **defaults.outputs.d2.style_map** dictionary. For example, your **d2.background** attribute maps into D2 **style.fill** attribute:
 
 ```
-defaults.outputs.d2.styles.background: fill
+defaults.outputs.d2.style_map.background: fill
 ```
 
 [^AD]: See [](dev-attribute-validation) and [](dev-valid-data-types) for more details.

docs/outputs/graph.md

@@ -21,7 +21,22 @@ Graphing routines use **[default](topo-defaults)** topology settings to modify t
 * **outputs.graphs.node_address_label** (default: *True*) -- add node loopback IP addresses or IP addresses of the first interface (for hosts) to node labels.
 * **outputs.graph.rr_sessions** (default: *False*) -- draw IBGP sessions between BGP route reflectors and clients as directional connections.
 
-You can also change the formatting of individual graph objects with the **outputs.styles._object_** defaults:
+(outputs-graph-link-node-attributes)=
+## Modifying Link and Node Attributes
+
+You can use the **graph** link and node attributes to change the style of individual nodes or links. The following attributes are built into _netlab_[^XS]:
+
+* **graph.color** -- line color (*color* GraphViz attribute)
+* **graph.fill** -- fill color (*fillcolor* GraphViz attribute)
+* **graph.width** -- line width (*penwidth* GraphViz attribute)
+
+You can also use the **graph.linkorder** link attribute to change the order of links in the D2 graph description file, which can sometimes improve the diagrams' appearance. Links with lower **graph.linkorder** values (default: 100) appear earlier in the list of links.
+
+[^XS]: You can extend the GraphViz styling capabilities and add new **graph** attributes. See [](outputs-d2-styles) for details.
+
+## Object Styles
+
+You can also change the formatting of individual graph objects with the **outputs.graph.styles._object_** defaults:
 
 | Object | Description |
 |--------|-------------|

docs/release/2.0.md

@@ -28,7 +28,7 @@ Release 2.0.1 includes [bug](bug-fixes-2.0.1) and [documentation](doc-fixes-2.0.
 * SRv6: [BGP L3VPN support](module-srv6)
 * Use the **routing** module to configure static routes on [**host** devices](node-role-host). VRF-aware devices can use a default route; other devices get more specific routes for address pools and named prefixes.
 * Multiple [EVPN import/export route targets](evpn-vlan-service) allow you to build complex EVPN-based services like *common services* or *hub-and-spoke connectivity*
-* Implement [node/link styles for D2 graphs](outputs-d2-style-attributes)
+* Implement [node/link styles for D2 graphs](outputs-d2-link-node-attributes)
 * Implement 'delete communities matching a list' in [routing policies](generic-routing-policies)
 
 **Retirements**

netsim/outputs/_graph.py

@@ -14,9 +14,28 @@ def set_shared_attributes(topology: Box) -> None:
   global SHARED_GRAPH_ATTRIBUTES
   SHARED_GRAPH_ATTRIBUTES = topology.defaults.outputs.graph.attributes.shared
 
-def build_nodes(topology: Box) -> Box:
+"""
+Utility function: map shared attributes (color, fill, width) into GL-specific attributes
+"""
+def map_style(g_attr: Box, style_map: Box) -> Box:
+  return get_box({ style_map[k]:v for k,v in g_attr.items() if k in style_map })
+
+"""
+Build graph nodes data structure (first step in graph-building)
+
+* Copy shared graph attributes into GL-specific ones
+* Collect nodes into graph.nodes dict
+* Collect BGP AS into graph.bgp dict
+* Return the new graph data structure
+"""
+def build_nodes(topology: Box, g_type: str) -> Box:
+  global SHARED_GRAPH_ATTRIBUTES
   maps = Box({},default_box=True,box_dots=True)
   for name,n in topology.nodes.items():
+    if g_type != 'graph':
+      for kw in SHARED_GRAPH_ATTRIBUTES:
+        if kw in n.get('graph',{}) and kw not in n.get(g_type,{}):
+          n[g_type][kw] = n.graph[kw]
     maps.nodes[name] = n
 
   if 'bgp' in topology.get('module',[]):
@@ -55,6 +74,12 @@ def add_groups(maps: Box, graph_groups: list, topology: Box) -> None:
       maps.clusters[g_name].nodes[node] = topology.nodes[node]
       placed_hosts.append(node)
 
+"""
+Create graph clusters:
+
+* Use 'groups' when available
+* Use BGP AS otherwise (if the topology is using BGP)
+"""
 def graph_clusters(graph: Box, topology: Box, settings: Box) -> None:
   if 'groups' in settings:
     must_be_list(
@@ -134,7 +159,7 @@ def topo_edges(graph: Box, topology: Box, settings: Box,g_type: str) -> None:
 
 def topology_graph(topology: Box, settings: Box,g_type: str) -> Box:
   set_shared_attributes(topology)
-  graph = build_nodes(topology)
+  graph = build_nodes(topology,g_type)
   graph_clusters(graph,topology,settings)
   topo_edges(graph,topology,settings,g_type)
   log.exit_on_error()
@@ -171,7 +196,7 @@ def bgp_graph(topology: Box, settings: Box, g_type: str, rr_sessions: bool) -> t
       module='graph')
     return None
 
-  graph = build_nodes(topology)
+  graph = build_nodes(topology,g_type)
   graph.clusters = graph.bgp
   graph.edges = []
   bgp_sessions(graph,topology,settings,g_type,rr_sessions)

netsim/outputs/d2.py

@@ -12,7 +12,7 @@
 from . import _TopologyOutput
 from ..utils import files as _files
 from ..utils import log
-from ._graph import topology_graph,bgp_graph
+from ._graph import topology_graph,bgp_graph,map_style
 
 '''
 Copy default settings into a D2 map converting Python dictionaries into
@@ -53,7 +53,7 @@ def d2_node_attr(f : typing.TextIO, n: Box, settings: Box, indent: str = '') ->
 def d2_style(f : typing.TextIO, obj: Box, indent: str) -> None:
   if 'd2' not in obj:
     return
-  d2_style = { STYLE_MAP[k]:v for k,v in obj.d2.items() if k in STYLE_MAP }
+  d2_style = map_style(obj.d2,STYLE_MAP)
   d2_extra = obj.get('d2.format',{})
 
   if d2_style or d2_extra:
@@ -208,7 +208,7 @@ def graph_bgp(topology: Box, fname: str, settings: Box, g_format: typing.Optiona
 '''
 def set_d2_attr(topology: Box) -> None:
   global STYLE_MAP
-  STYLE_MAP = topology.defaults.outputs.d2.styles
+  STYLE_MAP = topology.defaults.outputs.d2.style_map
 
   for n,ndata in topology.nodes.items():
     dev_data = topology.defaults.devices[ndata.device]

netsim/outputs/d2.yml

@@ -40,13 +40,15 @@ ebgp:
   target-arrowhead:
     shape: arrow
 
-styles:
+style_map:
   color: stroke
   width: stroke-width
+  fill: fill
 
 attributes:
   node:
     color: str
+    fill: str
     width: { type: int, min_value: 1, max_value: 32 }
   link:
     color: str

netsim/outputs/graph.py

@@ -12,7 +12,7 @@
 from . import _TopologyOutput
 from ..utils import files as _files
 from ..utils import log
-from ._graph import topology_graph,bgp_graph
+from ._graph import topology_graph,bgp_graph,map_style
 
 def edge_label(f : typing.TextIO, direction: str, data: Box, subnet: bool = True) -> None:
   addr = data.ipv4 or data.ipv6
@@ -29,6 +29,12 @@ def edge_node_net(f : typing.TextIO, link: Box, ifdata: Box, labels: typing.Opti
     edge_label(f,'tail',ifdata,subnet=False)
   f.write(' ]\n')
 
+'''
+Add graph styling information from graph.* link/node attributes
+'''
+STYLE_MAP: Box
+IGNORE_KW: list = ['dir', 'type', 'name']
+
 def get_gv_attr(c_data: Box, o_type: str, settings: Box) -> None:
   c_data.graph.format = settings.styles[o_type] + c_data.graph.format
 
@@ -81,6 +87,8 @@ def gv_end(f : typing.TextIO, fname: str) -> None:
     f.close()
 
 def gv_node(f : typing.TextIO, n: Box, settings: Box, indent: int = 0) -> None:
+  global STYLE_MAP
+
   f.write(f'{" "*indent}"{n.name}" [\n')
   node_ip_str = ""
   if settings.node_address_label:
@@ -90,7 +98,10 @@ def gv_node(f : typing.TextIO, n: Box, settings: Box, indent: int = 0) -> None:
     if node_ip:
       node_ip_str = f'{node_ip}'
 
-  f.write(f'{" "*indent}    label="{n.name} [{n.device}]\\n{node_ip_str}"\n')
+  gv_attr = n.get('graph.format',{})
+  gv_attr.label = f"{n.name} [{n.device}]\\n{node_ip_str}"
+  gv_attr += map_style(n.get('graph',{}),STYLE_MAP)
+  gv_multiline_attr(f,attr=gv_attr,indent=indent + 2)
   f.write(f'{" "*indent}]\n')
 
 def gv_network(f : typing.TextIO, n: Box, settings: Box, indent: int = 0) -> None:
@@ -126,10 +137,12 @@ def gv_nodes(f: typing.TextIO, graph: Box, topology: Box, settings: Box) -> None
         gv_node(f,n_data,settings,indent=2)
 
 def gv_links(f: typing.TextIO, graph: Box, topology: Box, settings: Box) -> None:
+  global STYLE_MAP
   for edge in graph.edges:
     dir = edge.nodes[0].get('attr.dir','--')
     f.write(f' "{edge.nodes[0].node}" -- "{edge.nodes[1].node}"')
-    attr = get_empty_box()
+
+    attr = map_style(edge.attr,STYLE_MAP) + edge.attr.get('format',{})
     for n_data in edge.nodes:
       if 'type' in n_data:
         attr = attr + settings.styles[n_data.type]
@@ -209,6 +222,9 @@ class Graph(_TopologyOutput):
   DESCRIPTION :str = 'Topology graph in graphviz format'
 
   def write(self, topology: Box) -> None:
+    global STYLE_MAP
+    STYLE_MAP = topology.defaults.outputs.graph.style_map
+
     graphfile = self.settings.filename or 'graph.dot'
     output_format = 'topology'
 

netsim/outputs/graph.yml

@@ -37,10 +37,21 @@ styles:
     labelfontsize: 8
     labeldistance: 1.5
 
+style_map:
+  color: color
+  width: penwidth
+  fill: fillcolor
+
 attributes:
+  node:
+    color: str
+    fill: str
+    width: { type: int, min_value: 1, max_value: 32 }
   link:
     type: dict
     _keys:
       linkorder: { type: int, min_value: 1, max_value: 200 }
       type: { type: str, valid_values: ['lan'] }
-  shared: [ linkorder ]
+      color: str
+      width: { type: int, min_value: 1, max_value: 32 }
+  shared: [ linkorder, color, fill, width ]

tests/platform-integration/graph/topo.yml

@@ -6,14 +6,14 @@ groups:
     device: frr
     members: [ fabric ]
   fabric:
-#    d2.color: '#ff8080'
-#    d2.width: 3
+    graph.color: '#ff8080'
+    graph.width: 3
+    graph.fill: '#ffc0c0'
     bgp.as: 65000
     members: [ s1, s2, l1, l2 ]
   host:
     members: [ h1, h2, h3, h4 ]
     device: linux
-#    d2.color: '#e0e0e0'
 
 module: [ ospf, bgp ]
 
@@ -27,6 +27,8 @@ links:
     graph.linkorder: 200
   h2:
     graph.linkorder: 200
+  graph.color: "#FF0000"
+  graph.width: 3
 - interfaces: [ l2, h3 ]
 - interfaces: [ l2, h4 ]
   graph.type: lan