Skip to content

Commit 7335253

Browse files
shmfrtanyaveksleradisos
authored
Explainability with hcs (#477)
* Initial implementation of the optimized TcpLikeProperties (and HyperCubeSet) holding all connections including src_peers, dst_peers and protocols Signed-off-by: Tanya <[email protected]> * Extended testcase3 to produce connectivity_map Signed-off-by: Tanya <[email protected]> * Fixed a problem in HyperCubeSet (wrongly changing self in _and_aux. Added optimized_denied_ingress_props and optimized_denied_egress_props (in addition to allowed ones). Improved non_captured_conns computation Signed-off-by: Tanya <[email protected]> * Small fix Signed-off-by: Tanya <[email protected]> * Further fix of the hyper cube set Signed-off-by: Tanya <[email protected]> * Further fix of the hyper cube set Signed-off-by: Tanya <[email protected]> * Avoiding redundant and heavy copy of layers. Signed-off-by: Tanya <[email protected]> * General changes from the Optimized_HC_set branch. Signed-off-by: Tanya <[email protected]> * General changes from the Optimized_HC_set branch. Signed-off-by: Tanya <[email protected]> * Avoiding lint warnings Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Avoiding lint warnings Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Added support of IpBlocks in optimized hyper cube set implementation. Signed-off-by: Tanya <[email protected]> * More accurate update of base_peer_set. Signed-off-by: Tanya <[email protected]> * Added dedundant fw_rules creation for testing (to be further removed). Signed-off-by: Tanya <[email protected]> * Initial implementation of Calico optimized connections handling. Fixed protocol handling: 0 is a valid protocol number (HOPOPT). Allowing any protocol in the range [0...255], though ProtocolNameResolver does not contain names of all of the possible 256 protocols. Fixed handling non-captured peers in K8S (cannot be handled as denied). Signed-off-by: Tanya <[email protected]> * Fixed the construction of connectivity graph (when some of src_peer or dst_peers dimensions is all values). Added optimization for fw_rules_map - join different entries having the same values (fw_rules). Signed-off-by: Tanya <[email protected]> * Multiple fixes: 1. To represent No connections, do not build TcpLikeProperties (TcpLikeProperties with no dimensions represent All connections). 2. Support subsets in query in optimized solution. 3. For comparison of optimized solution to the original one, add connections from peers to themselves. 4. More accurate comparison for 'dot' connectivity queries. 5. Generalized convert_named_ports (to not assume dimensions order) 6. Handling the possibility when projection on one dimension is empty. Signed-off-by: Tanya <[email protected]> * Properly handling 'False' represented by TcpLikeProperties. Properly handling HostEPs in optimized TcpLikeProperties. Signed-off-by: Tanya <[email protected]> * Added support to ICMP data in optimized HC set (handled in TCPLikeProperties) Fixed handling of non-captured pods in optimized solution. Added using True/False HC_set (make_all_properties()/make_empty_properties()) Signed-off-by: Tanya <[email protected]> * Added command line flag optimized_run, having 3 possibilities: 'false' - only original run 'true' - only optimized run 'debug'- both runs and comparison of their results. Printing parsing time, queries time and total run time. Signed-off-by: Tanya <[email protected]> * Small fixes; Workaround for the bug in HC set: using mutual contained_in, instead of == Signed-off-by: Tanya <[email protected]> * Small fixes; Signed-off-by: Tanya <[email protected]> * Fixed building tcp_like_properties form connectivity graph Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Optimized the comparison between original and optimized connections (for -opt=debug option) Added more debug prints. Better handling of peer_set copying in TcpLikeProperties. Signed-off-by: Tanya <[email protected]> * Implemented optimized Istio policy handling. Further optimization - calculating ref_ip_blocks only in non-optimized run. Signed-off-by: Tanya <[email protected]> * Added Ingress policy support in the optimized solution. Improved comments. Signed-off-by: Tanya <[email protected]> * Added Istio Ingress policy support in the optimized solution. Signed-off-by: Tanya <[email protected]> * Further optimization: converting HC set directly to fw rules. Signed-off-by: Tanya <[email protected]> * Small bug fix Signed-off-by: Tanya <[email protected]> * Fixed printing peer sets in FWRules. Signed-off-by: Tanya <[email protected]> * More released comparison between original and optimized fw-rules (allowing differences in auto-connections). Signed-off-by: Tanya <[email protected]> * Splitting istio opt properties to tcp and non-tcp properties. Fixed handling non captured peers in Istio policy. Signed-off-by: Tanya <[email protected]> * Fixed creation TcpLikeProperties with empty methods or protocols. Signed-off-by: Tanya <[email protected]> * Optimizing sidecar priorities handling by refinement of sidecar.selected_peers in parse time. Istio sidecar optimized connectivity implementation. Removing unused methods. Better non-captured handling. Signed-off-by: Tanya <[email protected]> * Adding newline at the end of connectivity test expected results. Signed-off-by: Tanya <[email protected]> * Handling exclude_ipv6 print in optimized calculation. Signed-off-by: Tanya <[email protected]> * Fixing initialization of MethodSet in HTTPRoute (None means no methods, MethodSet() means empty method set, which would create empty connections). Signed-off-by: Tanya <[email protected]> * Changed output format of ICMP data. Signed-off-by: Tanya <[email protected]> * Making default the original (not optimized) implementation in run_all_tests Signed-off-by: Tanya <[email protected]> * 1. Merge with master 2. Improved filtering of ipv6 blocks in optimized solution. Signed-off-by: Tanya <[email protected]> * Simplifying and improving make_tcp_like_properties function. Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Removed unised classes ConnectivityGraphPrototype and ConnectivityGraphOptimized. In creating TcpLikeProperties, methods and protocols cannot be None. Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Avoid using creation of TcpLikeProperties directly with init; using make_tcp_like_properties, make_tcp_like_properties_from_dict, make_empty_properties or make_all_properties instead. Use icmp_type and icmp_code full domain intervals instead of None in parameters to TcpLikeProperties creation methods. Removed unused or commented-out code. Fixed project_on_one_dimension to return None (to represent all values) for non-active dimensions. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing ConnectionSet.__str__ to be accurate, since it is used in sorting functions. Signed-off-by: Tanya <[email protected]> * Fixed excluding unused ipv6 blocks in the optimized solution. Signed-off-by: Tanya <[email protected]> * Renamed TcpLikeProperties to ConnectivityProperties. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * track expl data Signed-off-by: Shmulik Froimovich <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * Simplified and cleaned interfaces. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Removed assertion which is no longer correct. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Added a new class ConnectivityCube that manages forth and back translations of all dimensions of ConnectivityProperties, (translations between input format and internal format). Consequently, changed API of ConnectivityProperties methods to use the above new class. Improved documentation. Signed-off-by: Tanya <[email protected]> * Added set_dims method to set multiple dimensions at once. Signed-off-by: Tanya <[email protected]> * Added get_protocol_set_with_single_protocol function to ProtocolSet. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixed connectivity properties unit tests to match the new API. Signed-off-by: Tanya <[email protected]> * Aligned get_cube_dict to return str for all dimensions. Fixed resolving named ports for excluded_named_ports in the optimized solution. Signed-off-by: Tanya <[email protected]> * Removed unused ICMPDAtaSet class and its unit tests. Optimized ConnectivityProperties properties creation for semantic True (all full dimensions). Removed unused methods. Signed-off-by: Tanya <[email protected]> * Cleaner code using the new ConnectivityCube API. Signed-off-by: Tanya <[email protected]> * Added missing copy() in ConnectionSet. Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Added missing copy() in ConnectionSet. Fixed lint errors. Signed-off-by: Tanya <[email protected]> * When running with -opt=debug, printing the original results of ConnectivityMap query. Signed-off-by: Tanya <[email protected]> * Made cleaner interface of ConectivityCube class, using __setitem__, __getitem__ and update functions. Made cleaner interface of creating empty/full/by cube ConnectivityProperties. Signed-off-by: Tanya <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * Made cleaner interface of ConectivityCube class, using __setitem__, __getitem__ and update functions. Made cleaner interface of creating empty/full/by cube ConnectivityProperties. Signed-off-by: Tanya <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * Fixed lint error. Signed-off-by: Tanya <[email protected]> * Small fix Signed-off-by: Tanya <[email protected]> * Added documentation and small code beautifications. Signed-off-by: Tanya <[email protected]> * Improved documentation. Simplified interface by adding ConnectivityCube.make_from_dict method. Signed-off-by: Tanya <[email protected]> * Small fix. Signed-off-by: Tanya <[email protected]> * Moved empty dimension values to DimensionsManager. Fixed project_on_one_dimension method. Signed-off-by: Tanya <[email protected]> * Moved empty dimension values to DimensionsManager. Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Fixed small errors. Signed-off-by: Tanya <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/Parsers/CalicoPolicyYamlParser.py Co-authored-by: Adi Sosnovich <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Removed base_peer_set from ConnectivityProperties and ConnectivityCube. Instead, added a singleton class BasePeerSet that keeps all peers and translates PeerSets to CanonicalIntervalSets and vice versa. Signed-off-by: Tanya <[email protected]> * Removed base_peer_set from ConnectivityProperties and ConnectivityCube. Instead, added a singleton class BasePeerSet that keeps all peers and translates PeerSets to CanonicalIntervalSets and vice versa. Signed-off-by: Tanya <[email protected]> * Added check to BasePeerSet.get_peer_interval_of that all peers are translated to intervals. Signed-off-by: Tanya <[email protected]> * updates after merge with HC branch Signed-off-by: Shmulik Froimovich <[email protected]> * updates after merge with HC branch Signed-off-by: Shmulik Froimovich <[email protected]> * removed unused function Signed-off-by: Shmulik Froimovich <[email protected]> * added some function descriptions Signed-off-by: Shmulik Froimovich <[email protected]> * txt_no_fe_rules format added Signed-off-by: Shmulik Froimovich <[email protected]> * Added OptimizedPolicyConnections class to hold allowed, denied and pass connections and captured peers. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * The BasePeerSet singleton should be reset in the main (for the cases when running multiple tests, for example from run_all_tests) Signed-off-by: Tanya <[email protected]> * Added support to calico PASS rules in optimized solution. Signed-off-by: Tanya <[email protected]> * Added support to calico PASS rules in optimized solution. Signed-off-by: Tanya <[email protected]> * Simplified calico parsing method to avoid lint error. Fixed typo in the code. Signed-off-by: Tanya <[email protected]> * handling ipBlocks and base ip range * Generalized ServiceEntry implementation for optimized solution. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * xml support for explain_all and default-policy fix Signed-off-by: Shmulik Froimovich <[email protected]> * Removed unused functions. Optimized BasePeerSet.get_peer_interval_of method. Signed-off-by: Tanya <[email protected]> * Separated ConnectivityCube class to its own file. Removed unused methods/params/imports. Signed-off-by: Tanya <[email protected]> * Update nca/NetworkConfig/NetworkConfig.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/Resources/NetworkPolicy.py Co-authored-by: Adi Sosnovich <[email protected]> * Added assertions avoiding incorrect comparisons of "src_peers" and "dst_peers" dimensions. Split complex ConnectivityMapQuery.exec method to multiple methods. Signed-off-by: Tanya <[email protected]> * Update nca/CoreDS/Peer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectionSet.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectionSet.py Co-authored-by: Adi Sosnovich <[email protected]> * Added shortcut function ConnectivityProperties.make_conn_props_from_dict. Moved BasePeerSet.reset() to run_args. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Code reuse optimization. Added documentation. Removed unused method. Improved ConnectivityProperties.project_on_one_dimension() method. Signed-off-by: Tanya <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Removed unused functions and imports. Avoid including IpBlocks in non-captured peers. Signed-off-by: Tanya <[email protected]> * Making more accurate default all properties, according to all peers in the current config. Signed-off-by: Tanya <[email protected]> * output_endpoints support. Signed-off-by: Shmulik Froimovich <[email protected]> * support ep modes Signed-off-by: Shmulik Froimovich <[email protected]> * Documentation added Signed-off-by: Shmulik Froimovich <[email protected]> * minor fix Signed-off-by: Shmulik Froimovich <[email protected]> * use Expl' functions only when activated by user Signed-off-by: Shmulik Froimovich <[email protected]> * some lintings Signed-off-by: Shmulik Froimovich <[email protected]> * supporting scheme files Signed-off-by: Shmulik Froimovich <[email protected]> * support only text output formats and '[',']' in peer names Signed-off-by: Shmulik Froimovich <[email protected]> * linting Signed-off-by: Shmulik Froimovich <[email protected]> * small fixes Signed-off-by: Shmulik Froimovich <[email protected]> * small fixes Signed-off-by: Shmulik Froimovich <[email protected]> * adding beautifulsoup4 to requirements Signed-off-by: Shmulik Froimovich <[email protected]> * small fix Signed-off-by: Shmulik Froimovich <[email protected]> * use TCP conns when istio layer is present Signed-off-by: Shmulik Froimovich <[email protected]> * some fixes and improvements. Signed-off-by: Shmulik Froimovich <[email protected]> * extra line Signed-off-by: Shmulik Froimovich <[email protected]> * adding test Signed-off-by: Shmulik Froimovich <[email protected]> * adding test Signed-off-by: Shmulik Froimovich <[email protected]> * changed 'run_all_tests' so opt parameter can be override by tests Signed-off-by: Shmulik Froimovich <[email protected]> * added expected results Signed-off-by: Shmulik Froimovich <[email protected]> * parsing xml without the need of lxml Signed-off-by: Shmulik Froimovich <[email protected]> * parsing xml without the need of lxml Signed-off-by: Shmulik Froimovich <[email protected]> * update expected results (for new xml parser) Signed-off-by: Shmulik Froimovich <[email protected]> * update expected results and make it deterministic Signed-off-by: Shmulik Froimovich <[email protected]> * some fixes and new tests Signed-off-by: Shmulik Froimovich <[email protected]> * temp support in setting peers from the peer container, till fix will be merged to master Signed-off-by: Shmulik Froimovich <[email protected]> * new expected results Signed-off-by: Shmulik Froimovich <[email protected]> * new expected results Signed-off-by: Shmulik Froimovich <[email protected]> * manually adding tests Signed-off-by: Shmulik Froimovich <[email protected]> * manually updating test times Signed-off-by: Shmulik Froimovich <[email protected]> * fixed some testing env agnostic issues Signed-off-by: Shmulik Froimovich <[email protected]> * better handling of ipblocks and some small fixes. Signed-off-by: Shmulik Froimovich <[email protected]> * removing explanation for ipblocks Signed-off-by: Shmulik Froimovich <[email protected]> * adding policies in one place Signed-off-by: Shmulik Froimovich <[email protected]> * adding some documentations and test Signed-off-by: Shmulik Froimovich <[email protected]> * updated expected results Signed-off-by: Shmulik Froimovich <[email protected]> * fixed run_all_tests path problem Signed-off-by: Shmulik Froimovich <[email protected]> * duplicate line removed Signed-off-by: Shmulik Froimovich <[email protected]> * updated testes and some docs. Signed-off-by: Shmulik Froimovich <[email protected]> * little better explanation Signed-off-by: Shmulik Froimovich <[email protected]> * some small fixes Signed-off-by: Shmulik Froimovich <[email protected]> * base_name added Signed-off-by: Shmulik Froimovich <[email protected]> * better documentation for using IP-blocks Signed-off-by: Shmulik Froimovich <[email protected]> * added a ip-block test Signed-off-by: Shmulik Froimovich <[email protected]> * base_name check Signed-off-by: Shmulik Froimovich <[email protected]> * base_name check Signed-off-by: Shmulik Froimovich <[email protected]> * base_name and sync Signed-off-by: Shmulik Froimovich <[email protected]> --------- Signed-off-by: Tanya <[email protected]> Signed-off-by: Shmulik Froimovich <[email protected]> Co-authored-by: Tanya <[email protected]> Co-authored-by: Adi Sosnovich <[email protected]>
1 parent 36661e4 commit 7335253

31 files changed

+9786
-58
lines changed

README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,11 @@ The arguments to `--resource_list` and to `--base_resource_list` should be one o
101101
- `--output_endpoints`\
102102
Choose endpoints type in output (pods/deployments).\
103103
*default:* deployments
104+
- `--explain`\
105+
A pair of node names (comma separated) to explain the policies affecting their connection or lack of it. Relevant only for connectivity query.\
106+
Connections including IP-Blocks will show only the configurations of the node in that connection (since, IP-Blocks does
107+
not have configurations). IP-Blocks should be places in CIDR format as seen in the query results (run the connectivity query first, to see the nodes there).\
108+
e.g. default/pod-A1,default/deployment-B1.
104109
- `--print_ipv6`\
105110
include IPv6 range in the query results even when the policies of the config do not contain any IPv6 addresses.
106111

docs/SchemeFileFormat.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -78,15 +78,15 @@ For example: `my_set/prod_ns/deny_all_policy`. If there are multiple policies na
7878
#### <a name="outputconfig"></a>Output Configuration object
7979
The supported entries in the outputConfiguration object are as follows:
8080

81-
| Field | Description | Value |
82-
|------------------|-----------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------|
83-
| outputFormat | Output format specification. | string [ txt / yaml / csv / md / dot / jpg/ txt_no_fw_rules] |
84-
| outputPath | A file path to redirect output into. | string |
85-
| outputEndpoints | Choose endpoints type in output. | string [ pods / deployments ] |
86-
| subset | A dict object with the defined subset elements to display in the output | [subset](#subset) object |
87-
| fullExplanation | Choose if to print all counterexamples causing the query result in the output | bool |
88-
| excludeIPv6Range | If the policies of the config do not contain any IPv6 addresses, do not include IPv6 range in the query results | bool [default: True] |
89-
81+
| Field | Description | Value |
82+
|------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------|
83+
| outputFormat | Output format specification. | string [ txt / yaml / csv / md / dot / jpg/ txt_no_fw_rules] |
84+
| outputPath | A file path to redirect output into. | string |
85+
| outputEndpoints | Choose endpoints type in output. | string [ pods / deployments ] |
86+
| subset | A dict object with the defined subset elements to display in the output | [subset](#subset) object |
87+
| fullExplanation | Choose if to print all counterexamples causing the query result in the output | bool |
88+
| excludeIPv6Range | If the policies of the config do not contain any IPv6 addresses, do not include IPv6 range in the query results | bool [default: True] |
89+
| explain | A pair of node names (comma separated) to explain the policies affecting their connection or lack of it. Relevant only for connectivityMap query. Connections including IP-Blocks will show only the configurations of the node in that connection (since, IP-Blocks does not have configurations). IP-Blocks should be places in CIDR format as seen in the query results (run the connectivity query first, to see the nodes there). | string [ ns/node1,ns/node2 ] |
9090

9191
#### <a name="subset"></a>Subset object
9292
The supported entries in the subset object are as follows:

nca/FWRules/ConnectivityGraph.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -343,7 +343,7 @@ def get_connectivity_dot_format_str(self, connectivity_restriction=None):
343343

344344
dot_graph = DotGraph(name)
345345
peers_groups = self._get_equals_groups()
346-
# we are going to treat a a peers_group as one peer.
346+
# we are going to treat a peers_group as one peer.
347347
# the first peer in the peers_group is representing the group
348348
# we will add the text of all the peers in the group to this peer
349349
for peers_group, group_connection in peers_groups:

nca/FWRules/FWRule.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -383,7 +383,8 @@ def get_pod_str(self):
383383
"""
384384
:return: string for the field src_pods or dst_pods in representation for txt rule format
385385
"""
386-
return f'[{self._get_pods_names()}]'
386+
sorted_pods_names = ', '.join(sorted(self._get_pods_names().split(', ')))
387+
return f'[{sorted_pods_names}]'
387388

388389
def _get_pods_names(self):
389390
res = ''

nca/FWRules/InteractiveConnectivityGraph.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ def read_input_file(self):
127127
"""
128128
try:
129129
with open(self.input_svg_file) as svg_file:
130-
self.soup = BeautifulSoup(svg_file.read(), 'xml')
130+
self.soup = BeautifulSoup(svg_file.read(), 'html')
131131
except Exception as e:
132132
print(f'Failed to open file: {self.input_svg_file}\n{e} for reading', file=sys.stderr)
133133

nca/FileScanners/GenericTreeScanner.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ class YamlFile:
2121

2222
class ObjectWithLocation:
2323
line_number = 0
24+
path = ''
2425
column_number = 0
2526

2627

@@ -36,13 +37,15 @@ def to_yaml_objects(yaml_node):
3637
if isinstance(yaml_node, yaml.SequenceNode):
3738
res = YamlList()
3839
res.line_number = yaml_node.start_mark.line
40+
res.path = yaml_node.start_mark.name
3941
res.column_number = yaml_node.start_mark.column
4042
for obj in yaml_node.value:
4143
res.append(to_yaml_objects(obj))
4244
return res
4345
if isinstance(yaml_node, yaml.MappingNode):
4446
res = YamlDict()
4547
res.line_number = yaml_node.start_mark.line + 1
48+
res.path = yaml_node.start_mark.name
4649
res.column_number = yaml_node.start_mark.column + 1
4750
for obj in yaml_node.value:
4851
res[obj[0].value] = to_yaml_objects(obj[1])

nca/NetworkConfig/NetworkConfig.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
from nca.CoreDS.ConnectivityProperties import ConnectivityProperties
1010
from nca.Resources.NetworkPolicy import NetworkPolicy, OptimizedPolicyConnections
1111
from .NetworkLayer import NetworkLayersContainer, NetworkLayerName
12+
from nca.Utils.ExplTracker import ExplTracker
1213

1314

1415
@dataclass
@@ -275,6 +276,8 @@ def allowed_connections_optimized(self, layer_name=None):
275276
:return: allowed_conns: all allowed connections for relevant peers.
276277
:rtype: OptimizedPolicyConnections
277278
"""
279+
if ExplTracker().is_active():
280+
ExplTracker().set_peers(self.peer_container.peer_set)
278281
if layer_name is not None:
279282
if layer_name not in self.policies_container.layers:
280283
return self.policies_container.layers.empty_layer_allowed_connections_optimized(self.peer_container,

nca/NetworkConfig/NetworkConfigQuery.py

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
PoliciesAndRulesExplanations, PodsListsExplanations, ConnectionsDiffExplanation, IntersectPodsExplanation, \
2525
PoliciesWithCommonPods, PeersAndConnections, ComputedExplanation
2626
from .NetworkLayer import NetworkLayerName
27+
from nca.Utils.ExplTracker import ExplTracker
2728

2829

2930
class QueryType(Enum):
@@ -805,17 +806,24 @@ def compute_connectivity_output_optimized(self):
805806
"dst_peers": opt_peers_to_compare})
806807
base_peers_num = len(opt_peers_to_compare)
807808
subset_peers = self.compute_subset(opt_peers_to_compare)
809+
all_peers = subset_peers
808810
if len(subset_peers) != base_peers_num:
809811
# remove connections where both of src_peers and dst_peers are out of the subset
810812
subset_conns = ConnectivityProperties.make_conn_props_from_dict({"src_peers": subset_peers}) | \
811813
ConnectivityProperties.make_conn_props_from_dict({"dst_peers": subset_peers})
812814
all_conns_opt &= subset_conns
815+
src_peers, dst_peers = ExplTracker().extract_peers(all_conns_opt)
816+
all_peers = src_peers | dst_peers
813817
all_conns_opt = self.config.filter_conns_by_peer_types(all_conns_opt, opt_peers_to_compare)
818+
expl_conns = all_conns_opt
814819
if self.config.policies_container.layers.does_contain_layer(NetworkLayerName.Istio):
815820
output_res, opt_fw_rules_tcp, opt_fw_rules_non_tcp = \
816821
self.get_props_output_split_by_tcp(all_conns_opt, opt_peers_to_compare)
822+
expl_conns, _ = self.convert_props_to_split_by_tcp(all_conns_opt)
817823
else:
818824
output_res, opt_fw_rules = self.get_props_output_full(all_conns_opt, opt_peers_to_compare)
825+
if ExplTracker().is_active():
826+
ExplTracker().set_connections_and_peers(expl_conns, all_peers)
819827
return output_res, opt_fw_rules, opt_fw_rules_tcp, opt_fw_rules_non_tcp
820828

821829
def exec(self):
@@ -900,7 +908,9 @@ def get_props_output_full(self, props, peers_to_compare):
900908
if self.output_config.outputFormat in ['dot', 'jpg']:
901909
dot_full = self.dot_format_from_props(props, peers_to_compare)
902910
return dot_full, None
903-
# TODO - handle 'txt_no_fw_rules' output format
911+
if self.output_config.outputFormat == 'txt_no_fw_rules':
912+
conns_wo_fw_rules = self.txt_no_fw_rules_format_from_props(props, peers_to_compare)
913+
return conns_wo_fw_rules, None
904914
# handle other formats
905915
formatted_rules, fw_rules = self.fw_rules_from_props(props, peers_to_compare)
906916
return formatted_rules, fw_rules
@@ -966,7 +976,12 @@ def get_props_output_split_by_tcp(self, props, peers_to_compare):
966976
# concatenate the two graphs into one dot file
967977
res_str = dot_tcp + dot_non_tcp
968978
return res_str, None, None
969-
# TODO - handle 'txt_no_fw_rules' output format
979+
if self.output_config.outputFormat in ['txt_no_fw_rules']:
980+
txt_no_fw_rules_tcp = self.txt_no_fw_rules_format_from_props(props_tcp, peers_to_compare, connectivity_tcp_str)
981+
txt_no_fw_rules_non_tcp = self.txt_no_fw_rules_format_from_props(props_non_tcp, peers_to_compare,
982+
connectivity_non_tcp_str)
983+
res_str = txt_no_fw_rules_tcp + txt_no_fw_rules_non_tcp
984+
return res_str, None, None
970985
# handle formats other than dot and txt_no_fw_rules
971986
formatted_rules_tcp, fw_rules_tcp = self.fw_rules_from_props(props_tcp, peers_to_compare, connectivity_tcp_str)
972987
formatted_rules_non_tcp, fw_rules_non_tcp = self.fw_rules_from_props(props_non_tcp, peers_to_compare,
@@ -1033,6 +1048,20 @@ def dot_format_from_props(self, props, peers, connectivity_restriction=None):
10331048
conn_graph.add_edges_from_cube_dict(props.get_connectivity_cube(cube), self.config.peer_container)
10341049
return conn_graph.get_connectivity_dot_format_str(connectivity_restriction)
10351050

1051+
def txt_no_fw_rules_format_from_props(self, props, peers, connectivity_restriction=None):
1052+
"""
1053+
:param ConnectivityProperties props: properties describing allowed connections
1054+
:param PeerSet peers: the peers to consider for dot output
1055+
:param Union[str,None] connectivity_restriction: specify if connectivity is restricted to
1056+
TCP / non-TCP , or not
1057+
:rtype str
1058+
:return the connectivity map in txt_no_fw_rules format, considering connectivity_restriction if required
1059+
"""
1060+
conn_graph = ConnectivityGraph(peers, self.config.get_allowed_labels(), self.output_config)
1061+
for cube in props:
1062+
conn_graph.add_edges_from_cube_dict(props.get_connectivity_cube(cube), self.config.peer_container)
1063+
return conn_graph.get_connections_without_fw_rules_txt_format(connectivity_restriction)
1064+
10361065
def fw_rules_from_connections_dict(self, connections, peers_to_compare, connectivity_restriction=None):
10371066
"""
10381067
:param dict connections: the connections' dict (map from connection-set to peer pairs)

nca/NetworkConfig/NetworkConfigQueryRunner.py

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
from nca.Resources.NetworkPolicy import NetworkPolicy
1212
from .NetworkConfig import NetworkConfig
1313
from . import NetworkConfigQuery
14+
from nca.Utils.ExplTracker import ExplTracker
1415

1516

1617
@dataclass
@@ -40,9 +41,9 @@ def compute_final_results(self, output_format):
4041
extracts the final query results from self variables
4142
from self.query_iterations_output computes the final str output of the query,
4243
other results returned as is from query_result.
43-
:param str output_format: the output format to form the final output
44-
if output format is json, dumps the output list into one-top-leveled string
45-
if output format is yaml, dumps the output list into str of a list of yaml objects
44+
:param str output_format: the output format to form the final output.
45+
if output format is json, dumps the output list into one-top-leveled string.
46+
if output format is yaml, dumps the output list into str of a list of yaml objects.
4647
otherwise, writes the output list items split by \n
4748
:return the results: numerical result, output - str , num of not executed
4849
:rtype: int, str, int
@@ -169,7 +170,12 @@ def _run_query_for_each_config(self):
169170
query_result = QueryResult()
170171
for config in self.configs_array:
171172
query_result.update(self._execute_one_config_query(self.query_name, self._get_config(config)))
172-
return query_result.compute_final_results(self.output_configuration.outputFormat)
173+
expl_out = ''
174+
if ExplTracker().is_active() and self.output_configuration.explain and \
175+
ExplTracker().is_output_format_supported(self.output_configuration.outputFormat):
176+
expl_out = '\n\nExplainability results:\n'+ExplTracker().explain(self.output_configuration.explain.split(','))
177+
numerical_result, output, num_not_executed = query_result.compute_final_results(self.output_configuration.outputFormat)
178+
return numerical_result, output + expl_out, num_not_executed
173179

174180
def _run_query_on_configs_vs_base_config(self, cmd_line_flag):
175181
query_result = QueryResult()

0 commit comments

Comments
 (0)