genie learn isis fails to parse alternate configurations

[roguerouter]

Running into issues with the "genie learn isis" command when configurations deviate from what the parser expects to process.

First, when using ip unnumbered interfaces, the parser looks for the key levels which doesn't exist on an unnumbered interface (see below)

Issue while parsing: <class 'genie.libs.parser.nxos.show_isis.ShowIsisInterface'>

Traceback (most recent call last):
File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops
File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/libs/ops/isis/nxos/isis.py", line 397, in learn
self.make()
File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/ops/base/base.py", line 103, in make
self.maker.make(*args, **kwargs)
File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make
File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser
File "src/genie/metaparser/_metaparser.py", line 342, in genie.metaparser._metaparser.MetaParser.parse
File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse
File "src/genie/metaparser/util/schemaengine.py", line 419, in genie.metaparser.util.schemaengine.Schema.validate
genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces', 'Ethernet1/1', 'levels'], ['instance', '100', 'vrf', 'default', 'interfaces', 'Ethernet1/2', 'levels']]

As a result, the parser fails to collect data and stops processing. I switched levels and topologies in the parser to be Optional() values showIsisInterfaceSchema which "appears" to fix the issue but I'm not sure if this potentially breaks other things within genie.

Second, there appears to be an issue across the board with genie learn isis when the default vrf is not used. If a user deploys a non-default vrf (say vrf blue) across the system and associates all interfaces with that vrf, genie learn still expects to see interfaces, or a system/area id configured in the default vrf. As a result, we see failures because no interfaces are associated with the default vrf, only vrf blue. Without an interface configured in the default vrf, ShowIsisInterface parser fails. If you put an interface in the default vrf, then showisisadjacency fails. While this is expected in a configuration without a default vrf, in a situation where default is not configured, genie should ignore and parse the other configured vrfs.

Issue while parsing: <class 'genie.libs.parser.nxos.show_isis.ShowIsisInterface'>

Traceback (most recent call last):
File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops
File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/libs/ops/isis/nxos/isis.py", line 397, in learn
self.make()
File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/ops/base/base.py", line 103, in make
self.maker.make(*args, **kwargs)
File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make
File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser
File "src/genie/metaparser/_metaparser.py", line 342, in genie.metaparser._metaparser.MetaParser.parse
File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse
File "src/genie/metaparser/util/schemaengine.py", line 419, in genie.metaparser.util.schemaengine.Schema.validate
genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces']]

Issue while parsing: <class 'genie.libs.parser.nxos.show_isis.ShowIsisAdjacency'>

Traceback (most recent call last):
File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops
File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/libs/ops/isis/nxos/isis.py", line 397, in learn
self.make()
File "/Users/mleuschn/Builds/system_venv/venv/lib/python3.9/site-packages/genie/ops/base/base.py", line 103, in make
self.maker.make(*args, **kwargs)
File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make
File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser
File "src/genie/metaparser/_metaparser.py", line 342, in genie.metaparser._metaparser.MetaParser.parse
File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse
File "src/genie/metaparser/util/schemaengine.py", line 419, in genie.metaparser.util.schemaengine.Schema.validate
genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces']]

[Taarini]

Hi

Thanks for reporting. Could you please share the device output of the command, the os, platform and the command used.
I will debug and let you the know what the issue is about.

[roguerouter]

Hello,

The commands are being run on a Nexus 9kV running 9.3.6 code. However, I also have a customer who is using physical Nexus 9000 (9336-FX) that is experiencing the same issue. I also believe they have seen this issue on some IOS-XE platforms as well (ISR/ASR routers they ran genie learn isis).

The command being run in genie is "genie learn isis --testbed isis_testbed.yaml"

The first error is experienced when we lack an interface in a default vrf configuration.
genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'interfaces']]

In the case of my system I only have vrf blue configured on this device. You can see the output of "show isis interface vrf all":

`
show isis interface vrf all
IS-IS process: 100 VRF: blue
loopback5, Interface status: protocol-up/link-up/admin-up
IP address: 172.168.0.48, IP subnet: 172.168.0.48/32
IPv6 routing is disabled
Level1
No auth type and keychain
Auth check set
Level2
No auth type and keychain
Auth check set
Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1-2
BFD IPv4 is locally disabled for Interface loopback5
BFD does not support AF IPv4
BFD IPv6 is locally disabled for Interface loopback5
BFD does not support AF IPv6
MTR is disabled
Level Metric
1 1
2 1
Topologies enabled:
L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg
1 0 1 no UP UP yes DN no
2 0 1 no UP UP yes DN no

Ethernet1/1, Interface status: protocol-up/link-up/admin-up
IP unnumbered interface (loopback5)
IPv6 routing is disabled
No auth type and keychain
Auth check set
Index: 0x0002, Local Circuit ID: 0x01, Circuit Type: L1-2
BFD IPv4 is locally disabled for Interface Ethernet1/1
BFD IPv6 is locally disabled for Interface Ethernet1/1
MTR is disabled
Extended Local Circuit ID: 0x1A000000, P2P Circuit ID: 0000.0000.0000.00
Retx interval: 5, Retx throttle interval: 66 ms
LSP interval: 33 ms, MTU: 9216
MTU check OFF on P2P interface
P2P Adjs: 1, AdjsUp: 1, Priority 64
Hello Interval: 10, Multi: 3, Next IIH: 00:00:05
MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID
1 1 1 40 10 00:00:02 ffff.ffff.ffff.ff-ff
2 1 1 40 10 00:00:04 ffff.ffff.ffff.ff-ff
Topologies enabled:
L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg
1 0 40 no UP UP yes DN no
2 0 40 no UP UP yes DN no

Ethernet1/2, Interface status: protocol-up/link-up/admin-up
IP unnumbered interface (loopback5)
IPv6 routing is disabled
No auth type and keychain
Auth check set
Index: 0x0003, Local Circuit ID: 0x01, Circuit Type: L1-2
BFD IPv4 is locally disabled for Interface Ethernet1/2
BFD IPv6 is locally disabled for Interface Ethernet1/2
MTR is disabled
Extended Local Circuit ID: 0x1A000200, P2P Circuit ID: 0000.0000.0000.00
Retx interval: 5, Retx throttle interval: 66 ms
LSP interval: 33 ms, MTU: 9216
MTU check OFF on P2P interface
P2P Adjs: 1, AdjsUp: 1, Priority 64
Hello Interval: 10, Multi: 3, Next IIH: 00:00:05
MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID
1 1 1 40 10 00:00:10 ffff.ffff.ffff.ff-ff
2 1 1 40 10 00:00:08 ffff.ffff.ffff.ff-ff
Topologies enabled:
L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg
1 0 40 no UP UP yes DN no
2 0 40 no UP UP yes DN no

IS-IS process: 100 VRF: default
`

Now, if I add an interface to the default vrf as part of IS-IS process, the Missing key for interface goes away, but instead I receive this error:

genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'default', 'area_address']]

This is due to the lack of net address in the ISIS configuration on the router. As we are not using that vrf we do not want a System ID/NET configured under the isis router process (see router config below) and we need the genie isis parser to be ok with that scenario.

router isis 100 is-type level-1 address-family ipv4 unicast vrf blue net 49.9300.0100.8212.8048.00 address-family ipv4 unicast

If I place a physical interface into the default vrf, add a net in the default vrf and establish adjacency to another IS-IS router, and proceed to run genie learn isis again. The errors related to the default vrf go away but now I am presented with my VRF blue interfaces complaining they are missing the key levels (which is missing because they are IP unnumbered). Note the difference between the Ethernet 1/1 and 1/2 interfaces in vrf blue (ip unnumbered) and the Ethernet 1/3 interface in vrf default (bottom)

genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['instance', '100', 'vrf', 'blue', 'interfaces', 'Ethernet1/1', 'levels'], ['instance', '100', 'vrf', 'blue', 'interfaces', 'Ethernet1/2', 'levels']]

`n9kv-s2(config-router)# show isis interface vrf all
IS-IS process: 100 VRF: blue
loopback5, Interface status: protocol-up/link-up/admin-up
IP address: 172.168.0.48, IP subnet: 172.168.0.48/32
IPv6 routing is disabled
Level1
No auth type and keychain
Auth check set
Level2
No auth type and keychain
Auth check set
Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1-2
BFD IPv4 is locally disabled for Interface loopback5
BFD does not support AF IPv4
BFD IPv6 is locally disabled for Interface loopback5
BFD does not support AF IPv6
MTR is disabled
Level Metric
1 1
2 1
Topologies enabled:
L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg
1 0 1 no UP UP yes DN no
2 0 1 no UP UP yes DN no

Ethernet1/1, Interface status: protocol-up/link-up/admin-up
IP unnumbered interface (loopback5)
IPv6 routing is disabled
No auth type and keychain
Auth check set
Index: 0x0002, Local Circuit ID: 0x01, Circuit Type: L1-2
BFD IPv4 is locally disabled for Interface Ethernet1/1
BFD IPv6 is locally disabled for Interface Ethernet1/1
MTR is disabled
Extended Local Circuit ID: 0x1A000000, P2P Circuit ID: 0000.0000.0000.00
Retx interval: 5, Retx throttle interval: 66 ms
LSP interval: 33 ms, MTU: 9216
MTU check OFF on P2P interface
P2P Adjs: 0, AdjsUp: 0, Priority 64
Hello Interval: 10, Multi: 3, Next IIH: 00:00:08
MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID
1 0 0 40 10 0.339208 ffff.ffff.ffff.ff-ff
2 0 0 40 10 00:00:03 ffff.ffff.ffff.ff-ff
Topologies enabled:
L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg
1 0 40 no UP UP yes DN no
2 0 40 no UP UP yes DN no

Ethernet1/2, Interface status: protocol-up/link-up/admin-up
IP unnumbered interface (loopback5)
IPv6 routing is disabled
No auth type and keychain
Auth check set
Index: 0x0003, Local Circuit ID: 0x01, Circuit Type: L1-2
BFD IPv4 is locally disabled for Interface Ethernet1/2
BFD IPv6 is locally disabled for Interface Ethernet1/2
MTR is disabled
Extended Local Circuit ID: 0x1A000200, P2P Circuit ID: 0000.0000.0000.00
Retx interval: 5, Retx throttle interval: 66 ms
LSP interval: 33 ms, MTU: 9216
MTU check OFF on P2P interface
P2P Adjs: 1, AdjsUp: 1, Priority 64
Hello Interval: 10, Multi: 3, Next IIH: 00:00:08
MT Adjs AdjsUp Metric CSNP Next CSNP Last LSP ID
1 1 1 40 10 00:00:02 ffff.ffff.ffff.ff-ff
2 1 1 40 10 00:00:08 ffff.ffff.ffff.ff-ff
Topologies enabled:
L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg
1 0 40 no UP UP yes DN no
2 0 40 no UP UP yes DN no

IS-IS process: 100 VRF: default
Ethernet1/3, Interface status: protocol-up/link-up/admin-up
IP address: 10.1.1.1, IP subnet: 10.1.1.0/24
IPv6 routing is disabled
Level1
No auth type and keychain
Auth check set
Level2
No auth type and keychain
Auth check set
Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1
BFD IPv4 is locally disabled for Interface Ethernet1/3
BFD IPv6 is locally disabled for Interface Ethernet1/3
MTR is disabled
LSP interval: 33 ms, MTU: 1500
MTU check OFF on LAN interface level-1
MTU check OFF on LAN interface level-2
Level-1 Designated IS: n9kv-s2
Level Metric-0 Metric-2 CSNP Next CSNP Hello Multi Next IIH
1 40 0 10 00:00:06 3 3 0.812277
2 40 0 10 Inactive 10 3 Inactive
Level Adjs AdjsUp Pri Circuit ID Since
1 1 1 64 n9kv-s2.01 * 00:07:04
2 0 0 64 0000.0000.0000.01 never
Topologies enabled:
L MT Metric MetricCfg Fwdng IPV4-MT IPV4Cfg IPV6-MT IPV6Cfg
1 0 40 no UP UP yes DN no
2 0 40 no UP DN no DN no`

The issue with IP unnumbered not limited to a vrf configuration. Even interfaces in the default vrf experience this issue because they are missing the key 'levels' and based on reviewing the parser, would also error on the lack of 'topologies'.

[mfierbaugh]

The parser is also broken for IOS-XR. It seems that the parser also assumes specific configuration and if VRFs, IPv6+v4 topology, SRv6, etc are configured then it fails (albeit a bit differently).
Issue while building the feature

Traceback (most recent call last):
  File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops
  File "/home/cpoc/.local/lib/python3.8/site-packages/genie/libs/ops/isis/iosxr/isis.py", line 313, in learn
    self.make()
  File "/home/cpoc/.local/lib/python3.8/site-packages/genie/ops/base/base.py", line 103, in make
    self.maker.make(*args, **kwargs)
  File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make
  File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser
  File "src/genie/metaparser/_metaparser.py", line 308, in genie.metaparser._metaparser.MetaParser.parse
  File "/home/cpoc/.local/lib/python3.8/site-packages/genie/libs/parser/iosxr/show_isis.py", line 757, in cli
    address_family_dict = topology_dict\
UnboundLocalError: local variable 'topology_dict' referenced before assignment

Note that if I manually fix the undefined var or change the topology/configuration (as originally reported in this issue), there are still many parsing issues with isis. This is against a topology with no VRFs, ipv4 and ipv6 configured.

Traceback (most recent call last):
  File "src/genie/cli/commands/learn.py", line 365, in genie.cli.commands.learn.LearnCommand._retrieve_ops
  File "/Users/mfierbau/Library/Python/3.9/lib/python/site-packages/genie/libs/ops/isis/iosxr/isis.py", line 488, in learn
    self.make()
  File "/Users/mfierbau/Library/Python/3.9/lib/python/site-packages/genie/ops/base/base.py", line 103, in make
    self.maker.make(*args, **kwargs)
  File "src/genie/ops/base/maker.py", line 315, in genie.ops.base.maker.Maker.make
  File "src/genie/ops/base/maker.py", line 427, in genie.ops.base.maker.Maker._call_parser
  File "src/genie/metaparser/_metaparser.py", line 329, in genie.metaparser._metaparser.MetaParser.parse
  File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse
  File "src/genie/metaparser/util/schemaengine.py", line 318, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 288, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 318, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 288, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 390, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 363, in genie.metaparser.util.schemaengine.Schema.validate
  File "src/genie/metaparser/util/schemaengine.py", line 430, in genie.metaparser.util.schemaengine.Schema.validate
genie.metaparser.util.exceptions.SchemaUnsupportedKeyError: instance.RTP_LAB.interface.HundredGigE0/0/0/0.topology.ipv6 unicast: Unsupported keys: ['lsp_rexmit_queue_size']