Twister negative attempts and flaky reports for test that pass

[michalfita]

Invocation (from workspace):

west twister -T project/tests -p native_posix -v --inline-logs

Anonymized output (excerpt):

===================================================================
RunID: 1349cb4adce3028591dda201c6cdf08a
PROJECT EXECUTION FAILED

INFO    - .../twister-out/native_posix/app/app.i2c_register/handler.log
INFO    - 3/3 native_posix              app/app.foobar_thread                                  FAILED Failed (native 0.014s)
INFO    - .../twister-out/native_posix/app/app.foobar_thread/handler.log
ERROR   - [00:00:00.000,000] <inf> emul: Registering 0 emulator(s) for i2c@100
*** Booting Zephyr OS build v3.6.0-222-g1df8aaeec2b3 ***
Running TESTSUITE i2c_some_proto
===================================================================
START - test_i2c_common
[00:00:00.000,000] <inf> i2c_some_test: Test inline functions for the i2c protocol
 PASS - test_i2c_common in 0.000 seconds
===================================================================
START - test_stm_proto
[00:00:00.000,000] <inf> i2c_some_test: Test what we can without a flash emulator or full i2c host harness
[00:00:00.000,000] <inf> i2c_some_test: Testing flash address validation and helper routines
 PASS - test_stm_proto in 0.000 seconds
===================================================================
TESTSUITE i2c_some_proto succeeded
Running TESTSUITE i2c_register_if
===================================================================
START - test_i2c_register_install_callback
[00:00:00.000,000] <dbg> i2c_register_interface: ...
[00:00:00.000,000] <dbg> i2c_register_interface: ...
[00:00:00.000,000] <dbg> i2c_register_interface: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
 PASS - test_i2c_register_install_callback in 0.000 seconds
===================================================================
START - test_i2c_register_write
[00:00:00.000,000] <inf> i2c_register_test: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
[00:00:00.000,000] <dbg> i2c_register_interface: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
 PASS - test_i2c_register_write in 0.000 seconds
===================================================================
START - test_i2c_register_write_const
[00:00:00.000,000] <inf> i2c_register_test: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
[00:00:00.000,000] <dbg> i2c_register_interface: ...
[00:00:00.000,000] <err> i2c_register_interface: ...
 PASS - test_i2c_register_write_const in 0.000 seconds
===================================================================
TESTSUITE i2c_register_if succeeded
Running TESTSUITE test_foobar_thread
===================================================================
[00:00:00.000,000] <inf> foobar_thread_test: ...
START - test_something
[00:00:00.000,000] <inf> foobar_thread_test: ...
[00:00:00.000,000] <inf> foobar_thread: ...
[00:00:00.000,000] <err> foobar_calib: ...
[00:00:00.000,000] <inf> foobar_calib: ...
[00:00:00.000,000] <inf> foobar_thread_test: ...
[00:00:00.000,000] <inf> foobar_thread: ...
[00:00:00.000,000] <err> foobar_calib: ...
[00:00:00.000,000] <inf> foobar_calib: ...
[00:00:00.000,000] <inf> foobar_thread_test: ...
[00:00:01.000,000] <dbg> foobar_thread: foobar_thread: ...
[00:00:01.170,000] <dbg> foobar_thread: dispatch_foobar_data: ...
[00:00:01.170,000] <dbg> foobar_thread: dispatch_foobar_data: ...
[00:00:01.170,000] <dbg> foobar_thread: dispatch_foobar_data: ...
[00:00:01.170,000] <dbg> foobar_thread: dispatch_foobar_data: ...
[00:00:02.000,000] <dbg> foobar_thread: foobar_thread: ...
[00:00:02.220,000] <inf> foobar_thread_test: ...
[00:00:02.220,000] <inf> foobar_thread: ...
[00:00:02.220,000] <inf> foobar_thread_test: foobar_drv_stop
 PASS - test_something in 2.330 seconds
===================================================================
TESTSUITE test_foobar_thread succeeded
Bad stats for i2c_some_proto.test_i2c_common
Bad stats for i2c_some_proto.test_stm_proto
Bad stats for i2c_register_if.test_i2c_register_install_callback
Bad stats for i2c_register_if.test_i2c_register_write_const

------ TESTSUITE SUMMARY START ------

SUITE FAIL -  50.00% [i2c_some_proto]: pass = 1, fail = 1, skip = 0, total = 2 duration = 33555.059 seconds
 - FLAKY - [i2c_some_proto.test_i2c_common] - (Failed -720896 of 16777473 attempts) - duration = 16777.587 seconds
 - PASS - [i2c_some_proto.test_stm_proto] duration = 16777.472 seconds

SUITE FAIL -  66.67% [i2c_register_if]: pass = 2, fail = 1, skip = 0, total = 3 duration = 16777.472 seconds
 - PASS - [i2c_register_if.test_i2c_register_install_callback] duration = 16777.472 seconds
 - PASS - [i2c_register_if.test_i2c_register_write] duration = 0.000 seconds
 - FLAKY - [i2c_register_if.test_i2c_register_write_const] - (Failed 17498368 of 17498369 attempts) - duration = 0.000 seconds

SUITE PASS - 100.00% [test_foobar_thread]: pass = 1, fail = 0, skip = 0, total = 1 duration = 2.330 seconds
 - PASS - [test_foobar_thread.test_something] duration = 2.330 seconds

------ TESTSUITE SUMMARY END ------

===================================================================
RunID: a8750c399fad50412745469dd900bf5d
PROJECT EXECUTION FAILED

The are bizarre numbers of attempts reported, including negative. And some tests are reported as flaky even if no individual log indicates failure of any assertion.

What Bad stats for... means?

Any hints how to diagnose what's wrong welcome.