sanitycheck: do not abort logging on faults

We have been dropping lines after finding a fault which resulted in
missing information in the log. Make sure we continue and only report
failure at the end of the execution.

Signed-off-by: Anas Nashif <[email protected]>

Author: nashif

scripts/sanity_chk/harness.py

@@ -12,6 +12,7 @@ def __init__(self):
         self.tests = {}
         self.id = None
         self.fail_on_fault = True
+        self.fault = False
 
     def configure(self, instance):
         config = instance.test.harness_config
@@ -76,13 +77,16 @@ def handle(self, line):
             self.tests[name] = match.group(1)
 
         if self.RUN_PASSED in line:
-            self.state = "passed"
+            if self.fault:
+                self.state = "failed"
+            else:
+                self.state = "passed"
 
         if self.RUN_FAILED in line:
             self.state = "failed"
 
         if self.fail_on_fault:
             for fault in self.faults:
                 if fault in line:
-                    self.state = "failed"
+                    self.fault = True