-
Notifications
You must be signed in to change notification settings - Fork 13
Open
Labels
CoreChange in core componentsChange in core componentsCriticalThis is a critical issue and needs to be handled properly with test case support and design thought.This is a critical issue and needs to be handled properly with test case support and design thought.
Milestone
Description
Describe the bug
The results being logged into the terminal by the result handler, sometimes misses out on the result of the last run even though the test ran completely. For example:
✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
| arb | FAIL | 00:29:14 | NA |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
| Category | Count |
+--------------------------------------+-------+
| Disruptive Tests | 1 |
| Non Disruptive Tests | 0 |
| Disruptive Tests Skipped | 0 |
| Non Disruptive Tests Skipped | 0 |
| Tests Ran | 1 |
| Tests Skipped | 0 |
| Total Tests | 1 |
| Disruptive Tests Pass Percentage | 0.0 |
| Non Disruptive Tests Pass Percentage | 0 |
| Total Pass Percentage | 0.0 |
+--------------------------------------+-------+
Total Time Taken : 00 hours 58 minutes 57 seconds
✔ Tearing down successful.
In the above test run, the second test dist-arb result was not present in the table even though the framework ran that (which is confirmed from the logs and the time taken). Though, the results are logged sometime like:
✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
| arb | FAIL | 00:09:32 | NA |
| dist-arb | FAIL | 00:10:17 | NA |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
| Category | Count |
+--------------------------------------+-------+
| Disruptive Tests | 1 |
| Non Disruptive Tests | 0 |
| Disruptive Tests Skipped | 0 |
| Non Disruptive Tests Skipped | 0 |
| Tests Ran | 1 |
| Tests Skipped | 0 |
| Total Tests | 1 |
| Disruptive Tests Pass Percentage | 0.0 |
| Non Disruptive Tests Pass Percentage | 0 |
| Total Pass Percentage | 0.0 |
+--------------------------------------+-------+
Total Time Taken : 00 hours 20 minutes 01 seconds
✔ Tearing down successful.
It is being suspected that this issue is occurring due to a race condition in the multiprocessing queue..
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
CoreChange in core componentsChange in core componentsCriticalThis is a critical issue and needs to be handled properly with test case support and design thought.This is a critical issue and needs to be handled properly with test case support and design thought.