Merge pull request #2579 from sebix/fix-2571

lib/test: show warnings and errors in case of mismatch

Author: sebix

CHANGELOG.md

@@ -125,6 +125,7 @@ Please refer to the [NEWS](NEWS.md) for a list of changes which have an affect o
 - `intelmq.tests.lib.test_pipeline.TestAmqp.test_acknowledge`: Also skip on Python 3.11 and 3.12 besides on 3.8 when running on CI (PR#2542 by Sebastian Wagner).
 - Full pytest workflow: Version-independent install of postgres client, for Ubuntu 24.04 (default on GitHub now) test environment compatibility (PR#2557 by Sebastian Wagner).
 - Debian package build workflow: Use artifact upload v4 instead of v3 (PR#2565 by Sebastian Wagner).
+- `intelmq.lib.test.BotTestCase`: Show the warnings and errors of a bot if the amount does not match the expected number (PR#2578 by Kamil Mankowski/Sebastian Wagner, related to #2571).
 
 ### Known issues
 This is short list of the most important known issues. The full list can be retrieved from [GitHub](https://github.com/certtools/intelmq/labels/bug?page=2&q=is%3Aopen+label%3Abug).

Makefile

@@ -1,7 +1,7 @@
-# SPDX-FileCopyrightText: 2023 Filip Pokorný
+# SPDX-FileCopyrightText: 2023-2025 Filip Pokorný, Sebastian Wagner
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
-.PHONY: coverage docs clean
+.PHONY: coverage docs clean test codespell codestyle
 
 coverage:
 	python -m pytest --cov=intelmq -v

intelmq/lib/test.py

@@ -386,9 +386,9 @@ def run_bot(self, iterations: int = 1, error_on_pipeline: bool = False,
             self.assertLoglineEqual(-1, "Bot stopped.", "INFO")
 
         allowed_error_count = max(allowed_error_count, self.allowed_error_count)
-        self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count)
+        self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count, "\n".join(re.findall(' - ERROR - [^\n]*', self.loglines_buffer)))
         allowed_warning_count = max(allowed_warning_count, self.allowed_warning_count)
-        self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count)
+        self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count, "\n".join(re.findall(' - WARNING - [^\n]*', self.loglines_buffer)))
         self.assertNotRegexpMatchesLog("CRITICAL")
         """ If no error happened (incl. tracebacks) we can check for formatting """
         if not self.allowed_error_count: