Skip to content

Core: Race condition in result handler #919

@nik-redhat

Description

@nik-redhat

Describe the bug
The results being logged into the terminal by the result handler, sometimes misses out on the result of the last run even though the test ran completely. For example:

✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
|     arb     |     FAIL    |        00:29:14       |      NA     |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
|               Category               | Count |
+--------------------------------------+-------+
|           Disruptive Tests           |   1   |
|         Non Disruptive Tests         |   0   |
|       Disruptive Tests Skipped       |   0   |
|     Non Disruptive Tests Skipped     |   0   |
|              Tests Ran               |   1   |
|            Tests Skipped             |   0   |
|             Total Tests              |   1   |
|   Disruptive Tests Pass Percentage   |  0.0  |
| Non Disruptive Tests Pass Percentage |   0   |
|        Total Pass Percentage         |  0.0  |
+--------------------------------------+-------+
Total Time Taken : 00 hours 58 minutes 57 seconds
✔ Tearing down successful.

In the above test run, the second test dist-arb result was not present in the table even though the framework ran that (which is confirmed from the logs and the time taken). Though, the results are logged sometime like:

✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
|     arb     |     FAIL    |        00:09:32       |      NA     |
|   dist-arb  |     FAIL    |        00:10:17       |      NA     |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
|               Category               | Count |
+--------------------------------------+-------+
|           Disruptive Tests           |   1   |
|         Non Disruptive Tests         |   0   |
|       Disruptive Tests Skipped       |   0   |
|     Non Disruptive Tests Skipped     |   0   |
|              Tests Ran               |   1   |
|            Tests Skipped             |   0   |
|             Total Tests              |   1   |
|   Disruptive Tests Pass Percentage   |  0.0  |
| Non Disruptive Tests Pass Percentage |   0   |
|        Total Pass Percentage         |  0.0  |
+--------------------------------------+-------+
Total Time Taken : 00 hours 20 minutes 01 seconds
✔ Tearing down successful.

It is being suspected that this issue is occurring due to a race condition in the multiprocessing queue..

Metadata

Metadata

Assignees

Labels

CoreChange in core componentsCriticalThis is a critical issue and needs to be handled properly with test case support and design thought.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions